BPF packet scheduler updates part 1
Subsystems (7)
v10 cover BPF packet scheduler updates part 1
v10:
- add mptcp_storage map type.
- update scheduler API.
v9:
- merge 'Squash to "BPF packet scheduler updates v8"' and 'mptcp:
make the contexts array of sched_data invisible in BPF'
v8:
address Paolo's comments in v7:
- add sched_data into mptcp_sock as new field, not an pointer.
- drop rename __mptcp_set_timeout, export mptcp_set_timeout into BPF context instead.
- add a comment in bpf_burst to explan why not use
"ssk = send_info[SSK_MODE_ACTIVE].ssk;" directly.
- move mptcp_subflow_tcp_sock into bpf_tcp_helpers.h.
v7:
- drop more last_snd, only set it in bpf_rr.
- 1-10: save sched_data at mptcp_sock, for issues #342
- 11-16: add bpf_burst scheduler
- 17-19: add bpf_stale scheduler, for issues #349
v6:
- Only update patches 14, 15. Add a helper bpf_mptcp_subflow_ctx, fix bug in bpf_burst_get_retrans().
v5:
- 1-9: save sched_data at mptcp_sock, for issues #342
- 10-15: add bpf_burst scheduler
- 16-18: add bpf_stale scheduler, for issues #349
v4:
- drop "mptcp: add three helpers for bpf_burst", implement the helpers in the bpf context directly.
v3:
- address Florian's comments in v2.
- split into three more patches.
v2:
- fix this error reported by CI:
KASAN: slab-use-after-free in __mptcp_close_ssk (net/mptcp/protocol.c:2461) - add bpf burst scheduler.
This patchset adds sched_data pointer into mptcp_sock to to save some data at MPTCP and subflows levels.
With these changes, the old patch "mptcp: register default scheduler" in
[1] now works.
https://patchwork.kernel.org/project/mptcp/cover/[email protected]/ [1]
Geliang Tang (13):
Squash to "mptcp: drop last_snd and MPTCP_RESET_SCHEDULER" Squash to "mptcp: add struct mptcp_sched_ops" Squash to "mptcp: add sched_data_set_contexts helper"
mptcp: register default scheduler
Squash to "bpf: Add bpf_mptcp_sched_ops"
Squash to "bpf: Add bpf_mptcp_sched_kfunc_set"
bpf: Add mptcp_storage map type
bpf: Add mptcp_storage helpers
Squash to "selftests/bpf: Add mptcp sched structs" Squash to "selftests/bpf: Add bpf_first scheduler" Squash to "selftests/bpf: Add bpf_bkup scheduler" Squash to "selftests/bpf: Add bpf_rr scheduler" Squash to "selftests/bpf: Add bpf_red scheduler"
include/linux/bpf.h | 2 + include/linux/bpf_types.h | 1 + include/linux/btf_ids.h | 1 + include/net/mptcp.h | 11 +- include/uapi/linux/bpf.h | 40 +++ kernel/bpf/helpers.c | 4 + kernel/bpf/syscall.c | 6 +- kernel/bpf/verifier.c | 14 +- net/mptcp/bpf.c | 245 ++++++++++++++++++ net/mptcp/protocol.c | 10 +- net/mptcp/protocol.h | 7 +- net/mptcp/sched.c | 46 +++- tools/include/uapi/linux/bpf.h | 40 +++ tools/lib/bpf/libbpf.c | 1 + tools/lib/bpf/libbpf_probes.c | 1 + tools/testing/selftests/bpf/bpf_tcp_helpers.h | 20 +- .../selftests/bpf/progs/mptcp_bpf_bkup.c | 17 +- .../selftests/bpf/progs/mptcp_bpf_first.c | 8 +- .../selftests/bpf/progs/mptcp_bpf_red.c | 12 +- .../selftests/bpf/progs/mptcp_bpf_rr.c | 43 ++- 20 files changed, 475 insertions(+), 54 deletions(-)
Signature · 1 line
2.35.3
PATCH v10 1/13 Squash to "mptcp: drop last_snd and MPTCP_RESET_SCHEDULER"
Drop all last_snd.
--- net/mptcp/protocol.c | 9 --------- net/mptcp/protocol.h | 1 - 2 files changed, 10 deletions(-)
net/mptcp/protocol.c +0 -9
@@ -1573,7 +1573,6 @@ void __mptcp_push_pending(struct sock *sk, unsigned int flags)continue ; } do_check_data_fin = true; - msk->last_snd = ssk; } } }
@@ -1614,7 +1613,6 @@ static void __mptcp_subflow_push_pending(struct sock *sk, struct sock *ssk, boolif (ret <=0 )break ; copied += ret; - msk->last_snd = ssk;continue ; }
@@ -1627,7 +1625,6 @@ static void __mptcp_subflow_push_pending(struct sock *sk, struct sock *ssk, boolif (ret <=0 ) keep_pushing = false; copied += ret; - msk->last_snd = ssk; }mptcp_for_each_subflow (msk, subflow) {
@@ -2410,9 +2407,6 @@ static void __mptcp_close_ssk(struct sock *sk, struct sock *ssk,WRITE_ONCE (msk->first,NULL ); out: -if (ssk == msk->last_snd) - msk->last_snd =NULL ; -if (need_push)__mptcp_push_pending (sk,0 ); }
@@ -2592,8 +2586,6 @@ static void __mptcp_retrans(struct sock *sk) }release_sock (ssk); - - msk->last_snd = ssk; } }
@@ -3104,7 +3096,6 @@ static int mptcp_disconnect(struct sock *sk, int flags)* subflow */ mptcp_destroy_common (msk,MPTCP_CF_FASTCLOSE ); - msk->last_snd =NULL ;WRITE_ONCE (msk->flags,0 ); msk->cb_flags =0 ; msk->push_pending =0 ;
net/mptcp/protocol.h +0 -1
@@ -268,7 +268,6 @@ struct mptcp_sock {u64 rcv_data_fin_seq;u64 bytes_retrans;int rmem_fwd_alloc; -struct sock *last_snd;int snd_burst;int old_wspace;u64 recovery_snd_nxt;/* in recovery mode accept up to this seq;
Signature · 1 line
2.35.3
PATCH v10 2/13 Squash to "mptcp: add struct mptcp_sched_ops"
In reply to Geliang Tang- Add subflows in mptcp_sched_data.
- Drop const before msk in data_init() and get_subflow(), since msk->bpf_storage needs to be writable.
- Add const before data in get_subflow().
--- include/net/mptcp.h | 11 ++++++----- 1 file changed, 6 insertions(+), 5 deletions(-)
include/net/mptcp.h +6 -5
@@ -101,21 +101,22 @@ struct mptcp_out_options {struct mptcp_sched_data {bool reinject; +u8 subflows;struct mptcp_subflow_context *contexts[MPTCP_SUBFLOWS_MAX ]; };struct mptcp_sched_ops { -void (*data_init)(const struct mptcp_sock *msk, +void (*data_init)(struct mptcp_sock *msk,struct mptcp_sched_data *data); -int (*get_subflow)(const struct mptcp_sock *msk, -struct mptcp_sched_data *data); +int (*get_subflow)(struct mptcp_sock *msk, +const struct mptcp_sched_data *data);char name[MPTCP_SCHED_NAME_MAX ];struct module *owner;struct list_head list; -void (*init)(const struct mptcp_sock *msk); -void (*release)(const struct mptcp_sock *msk); +void (*init)(struct mptcp_sock *msk); +void (*release)(struct mptcp_sock *msk); }____cacheline_aligned_in_smp ;#ifdef CONFIG_MPTCP
Signature · 1 line
2.35.3
PATCH v10 3/13 Squash to "mptcp: add sched_data_set_contexts helper"
In reply to Geliang Tangmptcp: add sched_data helpers
Add a new helper mptcp_sched_data_set_contexts() to set the subflow pointers array in struct mptcp_sched_data. Add a new helper mptcp_subflow_ctx_by_pos() to get the given pos subflow from the contexts array in struct mptcp_sched_data. They will be invoked by the BPF schedulers to export the subflow pointers to the BPF contexts.
--- net/mptcp/protocol.h | 2 ++ net/mptcp/sched.c | 9 +++++++++ 2 files changed, 11 insertions(+)
net/mptcp/protocol.h +2 -0
@@ -664,6 +664,8 @@ void mptcp_subflow_set_scheduled(struct mptcp_subflow_context *subflow, bool scheduled);void mptcp_sched_data_set_contexts (const struct mptcp_sock *msk,struct mptcp_sched_data *data); +struct mptcp_subflow_context * +mptcp_subflow_ctx_by_pos (const struct mptcp_sched_data *data,unsigned int pos);struct sock *mptcp_subflow_get_send (struct mptcp_sock *msk);struct sock *mptcp_subflow_get_retrans (struct mptcp_sock *msk);int mptcp_sched_get_send (struct mptcp_sock *msk);
net/mptcp/sched.c +9 -0
@@ -108,11 +108,20 @@ void mptcp_sched_data_set_contexts(const struct mptcp_sock *msk,mptcp_subflow_set_scheduled (subflow, false); data->contexts[i++] = subflow; } + data->subflows = i;for (; i <MPTCP_SUBFLOWS_MAX ; i++) data->contexts[i] =NULL ; } +struct mptcp_subflow_context * +mptcp_subflow_ctx_by_pos (const struct mptcp_sched_data *data,unsigned int pos) +{ +if (pos >=MPTCP_SUBFLOWS_MAX ) +return NULL ; +return data->contexts[pos]; +} +int mptcp_sched_get_send (struct mptcp_sock *msk) {struct mptcp_subflow_context *subflow;
Signature · 1 line
2.35.3
PATCH v10 4/13 mptcp: register default scheduler
In reply to Geliang TangThis patch defines the default packet scheduler mptcp_sched_default. Register it in mptcp_sched_init(), which is invoked in mptcp_proto_init(). Skip deleting this default scheduler in mptcp_unregister_scheduler().
Set msk->sched to the default scheduler when the input parameter of mptcp_init_sched() is NULL.
--- net/mptcp/protocol.c | 1 + net/mptcp/protocol.h | 1 + net/mptcp/sched.c | 37 +++++++++++++++++++++++++++++++++++-- 3 files changed, 37 insertions(+), 2 deletions(-)
net/mptcp/protocol.c +1 -0
@@ -3970,6 +3970,7 @@ void __init mptcp_proto_init(void)mptcp_subflow_init ();mptcp_pm_init (); +mptcp_sched_init ();mptcp_token_init ();if (proto_register (&mptcp_prot,1 ) !=0 )
net/mptcp/protocol.h +1 -0
@@ -657,6 +657,7 @@ void mptcp_info2sockaddr(const struct mptcp_addr_info *info,struct mptcp_sched_ops *mptcp_sched_find (const char *name);int mptcp_register_scheduler (struct mptcp_sched_ops *sched);void mptcp_unregister_scheduler (struct mptcp_sched_ops *sched); +void mptcp_sched_init (void );int mptcp_init_sched (struct mptcp_sock *msk,struct mptcp_sched_ops *sched);void mptcp_release_sched (struct mptcp_sock *msk);
net/mptcp/sched.c +35 -2
@@ -16,6 +16,32 @@static DEFINE_SPINLOCK (mptcp_sched_list_lock);static LIST_HEAD (mptcp_sched_list); +static void mptcp_sched_default_data_init (struct mptcp_sock *msk, +struct mptcp_sched_data *data) +{ +} + +static int mptcp_sched_default_get_subflow (struct mptcp_sock *msk, +const struct mptcp_sched_data *data) +{ +struct sock *ssk; + + ssk = data->reinject ?mptcp_subflow_get_retrans (msk) : +mptcp_subflow_get_send (msk); +if (!ssk) +return -EINVAL ; + +mptcp_subflow_set_scheduled (mptcp_subflow_ctx (ssk), true); +return 0 ; +} + +static struct mptcp_sched_ops mptcp_sched_default = { + .data_init = mptcp_sched_default_data_init, + .get_subflow = mptcp_sched_default_get_subflow, + .name ="default" , + .owner =THIS_MODULE , +}; +/* Must be called with rcu read lock held */ struct mptcp_sched_ops *mptcp_sched_find (const char *name) {
@@ -50,16 +76,24 @@ int mptcp_register_scheduler(struct mptcp_sched_ops *sched)void mptcp_unregister_scheduler (struct mptcp_sched_ops *sched) { +if (sched == &mptcp_sched_default) +return ; +spin_lock (&mptcp_sched_list_lock);list_del_rcu (&sched->list);spin_unlock (&mptcp_sched_list_lock); } +void mptcp_sched_init (void ) +{ +mptcp_register_scheduler (&mptcp_sched_default); +} +int mptcp_init_sched (struct mptcp_sock *msk,struct mptcp_sched_ops *sched) {if (!sched) - goto out; + sched = &mptcp_sched_default;if (!bpf_try_module_get (sched, sched->owner))return -EBUSY ;
@@ -70,7 +104,6 @@ int mptcp_init_sched(struct mptcp_sock *msk,pr_debug ("sched=%s" , msk->sched->name); -out:return 0 ; }
Signature · 1 line
2.35.3
PATCH v10 5/13 Squash to "bpf: Add bpf_mptcp_sched_ops"
In reply to Geliang TangAdd write accesses for avg_pacing_rate of struct mptcp_subflow_context in .btf_struct_access. They will be used in the bpf_burst selftests.
Add write accesses for all bit flags of struct mptcp_subflow_context between map_csum_len and data_avail in .btf_struct_access. The stale flag will be used in the bpf_stale selftests.
--- net/mptcp/bpf.c | 6 ++++++ 1 file changed, 6 insertions(+)
net/mptcp/bpf.c +6 -0
@@ -50,6 +50,12 @@ static int bpf_mptcp_sched_btf_struct_access(struct bpf_verifier_log *log,case offsetof(struct mptcp_subflow_context , scheduled): end = offsetofend(struct mptcp_subflow_context, scheduled);break ; + case offsetofend(struct mptcp_subflow_context, map_csum_len): + end = offsetof(struct mptcp_subflow_context , data_avail); +break ; +case offsetof(struct mptcp_subflow_context , avg_pacing_rate): + end = offsetofend(struct mptcp_subflow_context, avg_pacing_rate); +break ;default :bpf_log (log,"no write support to mptcp_subflow_context at off %d\n" , off);return -EACCES ;
Signature · 1 line
2.35.3
PATCH v10 6/13 Squash to "bpf: Add bpf_mptcp_sched_kfunc_set"
In reply to Geliang TangExport mptcp_subflow_ctx_by_pos too.
--- net/mptcp/bpf.c | 1 + 1 file changed, 1 insertion(+)
net/mptcp/bpf.c +1 -0
@@ -166,6 +166,7 @@ struct bpf_struct_ops bpf_mptcp_sched_ops = {BTF_SET8_START (bpf_mptcp_sched_kfunc_ids)BTF_ID_FLAGS (func ,mptcp_subflow_set_scheduled )BTF_ID_FLAGS (func, mptcp_sched_data_set_contexts) +BTF_ID_FLAGS (func, mptcp_subflow_ctx_by_pos)BTF_SET8_END (bpf_mptcp_sched_kfunc_ids) static const struct btf_kfunc_id_set bpf_mptcp_sched_kfunc_set = {
Signature · 1 line
2.35.3
PATCH v10 7/13 bpf: Add mptcp_storage map type
In reply to Geliang TangAdd a new bpf map type MPTCP_STORAGE, define its mptcp_storage_map_ops. Add this storage into struct mptcp_sock to store persistent schedulers data.
--- include/linux/bpf_types.h | 1 + include/uapi/linux/bpf.h | 1 + kernel/bpf/syscall.c | 6 +- kernel/bpf/verifier.c | 1 + net/mptcp/bpf.c | 154 +++++++++++++++++++++++++++++++++ net/mptcp/protocol.h | 3 + tools/include/uapi/linux/bpf.h | 1 + tools/lib/bpf/libbpf.c | 1 + tools/lib/bpf/libbpf_probes.c | 1 + 9 files changed, 167 insertions(+), 2 deletions(-)
include/linux/bpf_types.h +1 -0
@@ -122,6 +122,7 @@ BPF_MAP_TYPE(BPF_MAP_TYPE_XSKMAP, xsk_map_ops) BPF_MAP_TYPE(BPF_MAP_TYPE_SOCKMAP, sock_map_ops) BPF_MAP_TYPE(BPF_MAP_TYPE_SOCKHASH, sock_hash_ops) BPF_MAP_TYPE(BPF_MAP_TYPE_REUSEPORT_SOCKARRAY, reuseport_array_ops) +BPF_MAP_TYPE(BPF_MAP_TYPE_MPTCP_STORAGE, mptcp_storage_map_ops)#endif #endif BPF_MAP_TYPE (BPF_MAP_TYPE_QUEUE , queue_map_ops)
include/uapi/linux/bpf.h +1 -0
@@ -943,6 +943,7 @@ enum bpf_map_type {BPF_MAP_TYPE_BLOOM_FILTER ,BPF_MAP_TYPE_USER_RINGBUF ,BPF_MAP_TYPE_CGRP_STORAGE , + BPF_MAP_TYPE_MPTCP_STORAGE, };/* Note that tracing related programs such as
kernel/bpf/syscall.c +4 -2
@@ -1060,7 +1060,8 @@ static int map_check_btf(struct bpf_map *map, const struct btf *btf, map->map_type !=BPF_MAP_TYPE_SK_STORAGE && map->map_type !=BPF_MAP_TYPE_INODE_STORAGE && map->map_type !=BPF_MAP_TYPE_TASK_STORAGE && - map->map_type != BPF_MAP_TYPE_CGRP_STORAGE) { + map->map_type !=BPF_MAP_TYPE_CGRP_STORAGE && + map->map_type != BPF_MAP_TYPE_MPTCP_STORAGE) { ret = -EOPNOTSUPP; goto free_map_tab; }
@@ -1085,7 +1086,8 @@ static int map_check_btf(struct bpf_map *map, const struct btf *btf, map->map_type !=BPF_MAP_TYPE_SK_STORAGE && map->map_type !=BPF_MAP_TYPE_INODE_STORAGE && map->map_type !=BPF_MAP_TYPE_TASK_STORAGE && - map->map_type != BPF_MAP_TYPE_CGRP_STORAGE) { + map->map_type !=BPF_MAP_TYPE_CGRP_STORAGE && + map->map_type != BPF_MAP_TYPE_MPTCP_STORAGE) { ret = -EOPNOTSUPP; goto free_map_tab; }
kernel/bpf/verifier.c +1 -0
@@ -16577,6 +16577,7 @@ static int check_map_prog_compatibility(struct bpf_verifier_env *env,case BPF_MAP_TYPE_SK_STORAGE :case BPF_MAP_TYPE_TASK_STORAGE :case BPF_MAP_TYPE_CGRP_STORAGE : +case BPF_MAP_TYPE_MPTCP_STORAGE :break ;default : verbose(env,
net/mptcp/bpf.c +154 -0
166 lines not on this page · raw message
net/mptcp/protocol.h +3 -0
@@ -313,6 +313,9 @@ struct mptcp_sock {*/ struct sock *first;struct mptcp_pm_data pm; +#ifdef CONFIG_BPF_SYSCALL +struct bpf_local_storage __rcu *bpf_storage; +#endif struct mptcp_sched_ops *sched;struct {u32 space;/* bytes copied in last measurement window */
tools/include/uapi/linux/bpf.h +1 -0
@@ -943,6 +943,7 @@ enum bpf_map_type {BPF_MAP_TYPE_BLOOM_FILTER ,BPF_MAP_TYPE_USER_RINGBUF ,BPF_MAP_TYPE_CGRP_STORAGE , + BPF_MAP_TYPE_MPTCP_STORAGE, };/* Note that tracing related programs such as
tools/lib/bpf/libbpf.c +1 -0
@@ -168,6 +168,7 @@ static const char * const map_type_name[] = { [BPF_MAP_TYPE_BLOOM_FILTER] = "bloom_filter", [BPF_MAP_TYPE_USER_RINGBUF] = "user_ringbuf", [BPF_MAP_TYPE_CGRP_STORAGE] = "cgrp_storage", + [BPF_MAP_TYPE_MPTCP_STORAGE] = "mptcp_storage", };static const char *const prog_type_name[] = {
tools/lib/bpf/libbpf_probes.c +1 -0
@@ -308,6 +308,7 @@ static int probe_map_create(enum bpf_map_type map_type)case BPF_MAP_TYPE_INODE_STORAGE :case BPF_MAP_TYPE_TASK_STORAGE :case BPF_MAP_TYPE_CGRP_STORAGE : +case BPF_MAP_TYPE_MPTCP_STORAGE : btf_key_type_id =1 ; btf_value_type_id =3 ; value_size =8 ;
Signature · 1 line
2.35.3
PATCH v10 8/13 bpf: Add mptcp_storage helpers
In reply to Geliang TangAdd two helpers bpf_mptcp_storage_get() and bpf_mptcp_storage_delete() for mptcp_storage map type to get or delete a bpf_local_storage from the given msk.
--- include/linux/bpf.h | 2 + include/linux/btf_ids.h | 1 + include/uapi/linux/bpf.h | 39 ++++++++++++++++ kernel/bpf/helpers.c | 4 ++ kernel/bpf/verifier.c | 13 +++++- net/mptcp/bpf.c | 84 ++++++++++++++++++++++++++++++++++ tools/include/uapi/linux/bpf.h | 39 ++++++++++++++++ 7 files changed, 181 insertions(+), 1 deletion(-)
include/linux/bpf.h +2 -0
@@ -2909,6 +2909,8 @@ extern const struct bpf_func_proto bpf_get_retval_proto;extern const struct bpf_func_proto bpf_user_ringbuf_drain_proto;extern const struct bpf_func_proto bpf_cgrp_storage_get_proto;extern const struct bpf_func_proto bpf_cgrp_storage_delete_proto; +extern const struct bpf_func_proto bpf_mptcp_storage_get_proto; +extern const struct bpf_func_proto bpf_mptcp_storage_delete_proto;const struct bpf_func_proto *tracing_prog_func_proto (enum bpf_func_id func_id,const struct bpf_prog *prog);
include/linux/btf_ids.h +1 -0
@@ -267,5 +267,6 @@ MAX_BTF_TRACING_TYPE,extern u32 btf_tracing_ids[];extern u32 bpf_cgroup_btf_id[];extern u32 bpf_local_storage_map_btf_id[]; +extern u32 bpf_mptcp_btf_id[];#endif
include/uapi/linux/bpf.h +39 -0
@@ -5570,6 +5570,43 @@ union bpf_attr {* 0 on success. * * **-ENOENT** if the bpf_local_storage cannot be found. +* +* void *bpf_mptcp_storage_get(struct bpf_map *map, struct mptcp_sock *msk, void *value, u64 flags) +* Description +* Get a bpf_local_storage from the *msk*. +* +* Logically, it could be thought of as getting the value from +* a *map* with *msk* as the **key**. From this +* perspective, the usage is not much different from +* **bpf_map_lookup_elem**\ (*map*, **&**\ *msk*) except this +* helper enforces the key must be a mptcp_sock and the map must also +* be a **BPF_MAP_TYPE_MPTCP_STORAGE**. +* +* Underneath, the value is stored locally at *msk* instead of +* the *map*. The *map* is used as the bpf-local-storage +* "type". The bpf-local-storage "type" (i.e. the *map*) is +* searched against all bpf_local_storage residing at *msk*. +* +* An optional *flags* (**BPF_LOCAL_STORAGE_GET_F_CREATE**) can be +* used such that a new bpf_local_storage will be +* created if one does not exist. *value* can be used +* together with **BPF_LOCAL_STORAGE_GET_F_CREATE** to specify +* the initial value of a bpf_local_storage. If *value* is +* **NULL**, the new bpf_local_storage will be zero initialized. +* Return +* A bpf_local_storage pointer is returned on success. +* +* **NULL** if not found or there was an error in adding +* a new bpf_local_storage. +* +* long bpf_mptcp_storage_delete(struct bpf_map *map, struct mptcp_sock *msk) +* Description +* Delete a bpf_local_storage from the *msk*. +* +* Return +* 0 on success. +* +* **-ENOENT** if the bpf_local_storage cannot be found. */ #define ___BPF_FUNC_MAPPER(FN, ctx...) \ FN(unspec, 0, ##ctx) \
@@ -5784,6 +5821,8 @@ union bpf_attr { FN(user_ringbuf_drain, 209, ##ctx) \ FN(cgrp_storage_get, 210, ##ctx) \ FN(cgrp_storage_delete, 211, ##ctx) \ + FN(mptcp_storage_get, 212, ##ctx) \ + FN(mptcp_storage_delete, 213, ##ctx) \/* */ /* backwards-compatibility macros for users of __BPF_FUNC_MAPPER that don't
kernel/bpf/helpers.c +4 -0
@@ -1768,6 +1768,10 @@ bpf_base_func_proto(enum bpf_func_id func_id)case BPF_FUNC_get_current_ancestor_cgroup_id:return &bpf_get_current_ancestor_cgroup_id_proto;#endif +case BPF_FUNC_mptcp_storage_get: +return &bpf_mptcp_storage_get_proto; +case BPF_FUNC_mptcp_storage_delete: +return &bpf_mptcp_storage_delete_proto;default :break ; }
kernel/bpf/verifier.c +12 -1
@@ -553,7 +553,8 @@ static bool is_storage_get_function(enum bpf_func_id func_id)return func_id == BPF_FUNC_sk_storage_get || func_id == BPF_FUNC_inode_storage_get || func_id == BPF_FUNC_task_storage_get || - func_id == BPF_FUNC_cgrp_storage_get; + func_id == BPF_FUNC_cgrp_storage_get || + func_id == BPF_FUNC_mptcp_storage_get; } static bool helper_multiple_ref_obj_use(enum bpf_func_id func_id,
@@ -8241,6 +8242,11 @@ static int check_map_func_compatibility(struct bpf_verifier_env *env, func_id != BPF_FUNC_map_push_elem) goto error;break ; +case BPF_MAP_TYPE_MPTCP_STORAGE : +if (func_id != BPF_FUNC_mptcp_storage_get && + func_id != BPF_FUNC_mptcp_storage_delete) + goto error; +break ;default :break ; }
@@ -8353,6 +8359,11 @@ static int check_map_func_compatibility(struct bpf_verifier_env *env,if (map->map_type !=BPF_MAP_TYPE_CGRP_STORAGE ) goto error;break ; +case BPF_FUNC_mptcp_storage_get: +case BPF_FUNC_mptcp_storage_delete: +if (map->map_type !=BPF_MAP_TYPE_MPTCP_STORAGE ) + goto error; +break ;default :break ; }
net/mptcp/bpf.c +84 -0
102 lines not on this page · raw message
tools/include/uapi/linux/bpf.h +39 -0
@@ -5570,6 +5570,43 @@ union bpf_attr {* 0 on success. * * **-ENOENT** if the bpf_local_storage cannot be found. +* +* void *bpf_mptcp_storage_get(struct bpf_map *map, struct mptcp_sock *msk, void *value, u64 flags) +* Description +* Get a bpf_local_storage from the *msk*. +* +* Logically, it could be thought of as getting the value from +* a *map* with *msk* as the **key**. From this +* perspective, the usage is not much different from +* **bpf_map_lookup_elem**\ (*map*, **&**\ *msk*) except this +* helper enforces the key must be a mptcp_sock and the map must also +* be a **BPF_MAP_TYPE_MPTCP_STORAGE**. +* +* Underneath, the value is stored locally at *msk* instead of +* the *map*. The *map* is used as the bpf-local-storage +* "type". The bpf-local-storage "type" (i.e. the *map*) is +* searched against all bpf_local_storage residing at *msk*. +* +* An optional *flags* (**BPF_LOCAL_STORAGE_GET_F_CREATE**) can be +* used such that a new bpf_local_storage will be +* created if one does not exist. *value* can be used +* together with **BPF_LOCAL_STORAGE_GET_F_CREATE** to specify +* the initial value of a bpf_local_storage. If *value* is +* **NULL**, the new bpf_local_storage will be zero initialized. +* Return +* A bpf_local_storage pointer is returned on success. +* +* **NULL** if not found or there was an error in adding +* a new bpf_local_storage. +* +* long bpf_mptcp_storage_delete(struct bpf_map *map, struct mptcp_sock *msk) +* Description +* Delete a bpf_local_storage from the *msk*. +* +* Return +* 0 on success. +* +* **-ENOENT** if the bpf_local_storage cannot be found. */ #define ___BPF_FUNC_MAPPER(FN, ctx...) \ FN(unspec, 0, ##ctx) \
@@ -5784,6 +5821,8 @@ union bpf_attr { FN(user_ringbuf_drain, 209, ##ctx) \ FN(cgrp_storage_get, 210, ##ctx) \ FN(cgrp_storage_delete, 211, ##ctx) \ + FN(mptcp_storage_get, 212, ##ctx) \ + FN(mptcp_storage_delete, 213, ##ctx) \/* */ /* backwards-compatibility macros for users of __BPF_FUNC_MAPPER that don't
Signature · 1 line
2.35.3
PATCH v10 9/13 Squash to "selftests/bpf: Add mptcp sched structs"
In reply to Geliang TangUpdate scheduler API.
--- tools/testing/selftests/bpf/bpf_tcp_helpers.h | 14 ++++++++------ 1 file changed, 8 insertions(+), 6 deletions(-)
tools/testing/selftests/bpf/bpf_tcp_helpers.h +8 -6
@@ -240,19 +240,19 @@ struct mptcp_subflow_context {struct mptcp_sched_data {bool reinject; -struct mptcp_subflow_context *contexts[MPTCP_SUBFLOWS_MAX ]; +__u8 subflows; } __attribute__((preserve_access_index));struct mptcp_sched_ops {char name[MPTCP_SCHED_NAME_MAX ]; -void (*init)(const struct mptcp_sock *msk); -void (*release)(const struct mptcp_sock *msk); +void (*init)(struct mptcp_sock *msk); +void (*release)(struct mptcp_sock *msk); -void (*data_init)(const struct mptcp_sock *msk, +void (*data_init)(struct mptcp_sock *msk,struct mptcp_sched_data *data); -int (*get_subflow)(const struct mptcp_sock *msk, -struct mptcp_sched_data *data); +int (*get_subflow)(struct mptcp_sock *msk, +const struct mptcp_sched_data *data);void *owner; };
@@ -269,5 +269,7 @@ extern void mptcp_subflow_set_scheduled(struct mptcp_subflow_context *subflow, bool scheduled) __ksym; extern void mptcp_sched_data_set_contexts(const struct mptcp_sock *msk, struct mptcp_sched_data *data) __ksym; +extern struct mptcp_subflow_context * +mptcp_subflow_ctx_by_pos (const struct mptcp_sched_data *data,unsigned int pos) __ksym;#endif
Signature · 1 line
2.35.3
PATCH v10 10/13 Squash to "selftests/bpf: Add bpf_first scheduler"
In reply to Geliang TangUpdate API.
--- tools/testing/selftests/bpf/progs/mptcp_bpf_first.c | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-)
tools/testing/selftests/bpf/progs/mptcp_bpf_first.c +4 -4
@@ -16,16 +16,16 @@ void BPF_PROG(mptcp_sched_first_release, const struct mptcp_sock *msk) { } -void BPF_STRUCT_OPS (bpf_first_data_init ,const struct mptcp_sock *msk, +void BPF_STRUCT_OPS (bpf_first_data_init ,struct mptcp_sock *msk,struct mptcp_sched_data *data) {mptcp_sched_data_set_contexts (msk, data); } -int BPF_STRUCT_OPS (bpf_first_get_subflow ,const struct mptcp_sock *msk, -struct mptcp_sched_data *data) +int BPF_STRUCT_OPS (bpf_first_get_subflow ,struct mptcp_sock *msk, +const struct mptcp_sched_data *data) { -mptcp_subflow_set_scheduled (data->contexts[0 ], true); +mptcp_subflow_set_scheduled (mptcp_subflow_ctx_by_pos (data,0 ), true);return 0 ; }
Signature · 1 line
2.35.3
PATCH v10 11/13 Squash to "selftests/bpf: Add bpf_bkup scheduler"
In reply to Geliang TangUpdate API.
--- .../selftests/bpf/progs/mptcp_bpf_bkup.c | 17 ++++++++++------- 1 file changed, 10 insertions(+), 7 deletions(-)
tools/testing/selftests/bpf/progs/mptcp_bpf_bkup.c +10 -7
@@ -16,28 +16,31 @@ void BPF_PROG(mptcp_sched_bkup_release, const struct mptcp_sock *msk) { } -void BPF_STRUCT_OPS (bpf_bkup_data_init ,const struct mptcp_sock *msk, +void BPF_STRUCT_OPS (bpf_bkup_data_init ,struct mptcp_sock *msk,struct mptcp_sched_data *data) {mptcp_sched_data_set_contexts (msk, data); } -int BPF_STRUCT_OPS (bpf_bkup_get_subflow ,const struct mptcp_sock *msk, -struct mptcp_sched_data *data) +int BPF_STRUCT_OPS (bpf_bkup_get_subflow ,struct mptcp_sock *msk, +const struct mptcp_sched_data *data) {int nr =0 ; -for (int i =0 ; i <MPTCP_SUBFLOWS_MAX ; i++) { -if (!data->contexts[i]) +for (int i =0 ; i < data->subflows && i <MPTCP_SUBFLOWS_MAX ; i++) { +struct mptcp_subflow_context *subflow; + + subflow =mptcp_subflow_ctx_by_pos (data, i); +if (!subflow)break ; -if (!BPF_CORE_READ_BITFIELD_PROBED (data->contexts[i], backup)) { +if (!BPF_CORE_READ_BITFIELD_PROBED (subflow, backup)) { nr = i;break ; } } -mptcp_subflow_set_scheduled (data->contexts[nr], true); +mptcp_subflow_set_scheduled (mptcp_subflow_ctx_by_pos (data, nr), true);return 0 ; }
Signature · 1 line
2.35.3
PATCH v10 12/13 Squash to "selftests/bpf: Add bpf_rr scheduler"
In reply to Geliang TangUse mptcp_storage instead of msk->last_snd.
--- tools/testing/selftests/bpf/bpf_tcp_helpers.h | 6 ++- .../selftests/bpf/progs/mptcp_bpf_rr.c | 43 ++++++++++++++----- 2 files changed, 38 insertions(+), 11 deletions(-)
tools/testing/selftests/bpf/bpf_tcp_helpers.h +5 -1
@@ -259,7 +259,6 @@ struct mptcp_sched_ops {struct mptcp_sock {struct inet_connection_sock sk; -struct sock *last_snd;__u32 token;struct sock *first;char ca_name[TCP_CA_NAME_MAX ];
@@ -271,5 +270,10 @@ extern void mptcp_sched_data_set_contexts(const struct mptcp_sock *msk, struct mptcp_sched_data *data) __ksym;extern struct mptcp_subflow_context *mptcp_subflow_ctx_by_pos (const struct mptcp_sched_data *data,unsigned int pos) __ksym; +static inline struct sock * +mptcp_subflow_tcp_sock (const struct mptcp_subflow_context *subflow) +{ +return subflow->tcp_sock; +}#endif
tools/testing/selftests/bpf/progs/mptcp_bpf_rr.c +33 -10
73 lines not on this page · raw message
Signature · 1 line
2.35.3
PATCH v10 13/13 Squash to "selftests/bpf: Add bpf_red scheduler"
In reply to Geliang TangUpdate API.
--- tools/testing/selftests/bpf/progs/mptcp_bpf_red.c | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-)
tools/testing/selftests/bpf/progs/mptcp_bpf_red.c +6 -6
@@ -16,20 +16,20 @@ void BPF_PROG(mptcp_sched_red_release, const struct mptcp_sock *msk) { } -void BPF_STRUCT_OPS (bpf_red_data_init ,const struct mptcp_sock *msk, +void BPF_STRUCT_OPS (bpf_red_data_init ,struct mptcp_sock *msk,struct mptcp_sched_data *data) {mptcp_sched_data_set_contexts (msk, data); } -int BPF_STRUCT_OPS (bpf_red_get_subflow ,const struct mptcp_sock *msk, -struct mptcp_sched_data *data) +int BPF_STRUCT_OPS (bpf_red_get_subflow ,struct mptcp_sock *msk, +const struct mptcp_sched_data *data) { -for (int i =0 ; i <MPTCP_SUBFLOWS_MAX ; i++) { -if (!data->contexts[i]) +for (int i =0 ; i < data->subflows && i <MPTCP_SUBFLOWS_MAX ; i++) { +if (!mptcp_subflow_ctx_by_pos (data, i))break ; -mptcp_subflow_set_scheduled (data->contexts[i], true); +mptcp_subflow_set_scheduled (mptcp_subflow_ctx_by_pos (data, i), true); }return 0 ;
Signature · 1 line
2.35.3
Re: Squash to "selftests/bpf: Add bpf_red scheduler": Build Failure
Hi Geliang,
Thank you for your modifications, that's great!
But sadly, our CI spotted some issues with it when trying to build it.
You can find more details there:
https://patchwork.kernel.org/project/mptcp/patch/cae101ed32a7f97ffcd7d4e25e7c96cd85819e87.1687746021.git.geliang.tang@suse.com/
https://github.com/multipath-tcp/mptcp_net-next/actions/runs/5373236226
Status: failure
Initiator: MPTCPimporter
Commits: https://github.com/multipath-tcp/mptcp_net-next/commits/d70b6a579bb6
Feel free to reply to this email if you cannot access logs, if you need some support to fix the error, if this doesn't seem to be caused by your modifications or if the error is a false positive one.
Cheers,
MPTCP GH Action bot
Bot operated by Matthieu Baerts (Tessares)
Re: Squash to "selftests/bpf: Add bpf_red scheduler": Tests Results
In reply to Geliang TangHi Geliang,
Thank you for your modifications, that's great!
Our CI did some validations and here is its report:
- KVM Validation: normal (except selftest_mptcp_join):
- Script error! ❓:
- Task: https://cirrus-ci.com/task/6610932755857408
- Summary: https://api.cirrus-ci.com/v1/artifact/task/6610932755857408/summary/summary.txt
- KVM Validation: normal (only selftest_mptcp_join):
- Script error! ❓:
- Task: https://cirrus-ci.com/task/4781345407238144
- Summary: https://api.cirrus-ci.com/v1/artifact/task/4781345407238144/summary/summary.txt
- KVM Validation: debug (except selftest_mptcp_join):
- Script error! ❓:
- Task: https://cirrus-ci.com/task/5907245314080768
- Summary: https://api.cirrus-ci.com/v1/artifact/task/5907245314080768/summary/summary.txt
- KVM Validation: debug (only selftest_mptcp_join):
- Script error! ❓:
- Task: https://cirrus-ci.com/task/5344295360659456
- Summary: https://api.cirrus-ci.com/v1/artifact/task/5344295360659456/summary/summary.txt
Initiator: Patchew Applier
Commits: https://github.com/multipath-tcp/mptcp_net-next/commits/d70b6a579bb6
If there are some issues, you can reproduce them using the same environment as the one used by the CI thanks to a docker image, e.g.:
$ cd [kernel source code]
$ docker run -v "${PWD}:${PWD}:rw" -w "${PWD}" --privileged --rm -it \
--pull always mptcp/mptcp-upstream-virtme-docker:latest \
auto-debugFor more details:
https://github.com/multipath-tcp/mptcp-upstream-virtme-docker
Please note that despite all the efforts that have been already done to have a stable tests suite when executed on a public CI like here, it is possible some reported issues are not due to your modifications. Still, do not hesitate to help us improve that ;-)
Cheers,
MPTCP GH Action bot
Bot operated by Matthieu Baerts (Tessares)
Re: Squash to "selftests/bpf: Add bpf_red scheduler": Tests Results
Hi Geliang,
Quoted 9 lines · MPTCP CI
Hi Geliang,
Thank you for your modifications, that's great!
Our CI did some validations and here is its report:
- KVM Validation: normal (except
selftest_mptcp_join):
- Script error! ❓:
- Task: https://cirrus-ci.com/task/6610932755857408
Please note that these errors are due to issues in net-next:
https://lore.kernel.org/r/[email protected]/
I just applied a fix in our tree. I also asked Patchew to re-apply the patches on top of the latest export branch but I don't think if this will work.
Please also note that the "Build Failure" email you also got is *not* linked to these errors, they are likely linked to the modifications you did.
https://github.com/multipath-tcp/mptcp_net-next/actions/runs/5373236226
Cheers,
Matt
Signature · 2 lines
Tessares | Belgium | Hybrid Access Solutions www.tessares.net
Re: [PATCH mptcp-next v10 02/13] Squash to "mptcp: add struct mptcp_sched_ops"
In reply to Geliang TangQuoted 4 lines · Geliang Tang
- Add subflows in
mptcp_sched_data.- Drop const before msk in
data_init()andget_subflow(), sincemsk->bpf_storageneeds to be writable.
No need to change msk type, we can keep it as const:
#define MPTCP_SCHED_NAME_MAX 16
#define MPTCP_SUBFLOWS_MAX 8
struct mptcp_sched_data {
bool reinject;
u8 subflows;
struct mptcp_subflow_context *contexts[MPTCP_SUBFLOWS_MAX];
};struct mptcp_sched_ops {
void (*data_init)(const struct mptcp_sock *msk,
struct mptcp_sched_data *data);
int (*get_subflow)(struct mptcp_sock *msk,
const struct mptcp_sched_data *data);
char name[MPTCP_SCHED_NAME_MAX];
struct module *owner;
struct list_head list;
void (*init)(const struct mptcp_sock *msk);
void (*release)(const struct mptcp_sock *msk);
} ____cacheline_aligned_in_smp;Will update in v11.
-Geliang
Quoted 43 lines
- Add const before data in
get_subflow().Signed-off-by Geliang Tang <[email protected]>--- include/net/mptcp.h | 11 ++++++----- 1 file changed, 6 insertions(+), 5 deletions(-)include/net/mptcp.h +6
-5@@ -101,21 +101,22 @@ struct mptcp_out_options {struct mptcp_sched_data {bool reinject; +u8 subflows;struct mptcp_subflow_context *contexts[MPTCP_SUBFLOWS_MAX ]; };struct mptcp_sched_ops { -void (*data_init)(const struct mptcp_sock *msk, +void (*data_init)(struct mptcp_sock *msk,struct mptcp_sched_data *data); -int (*get_subflow)(const struct mptcp_sock *msk, -struct mptcp_sched_data *data); +int (*get_subflow)(struct mptcp_sock *msk, +const struct mptcp_sched_data *data);char name[MPTCP_SCHED_NAME_MAX ];struct module *owner;struct list_head list; -void (*init)(const struct mptcp_sock *msk); -void (*release)(const struct mptcp_sock *msk); +void (*init)(struct mptcp_sock *msk); +void (*release)(struct mptcp_sock *msk); }____cacheline_aligned_in_smp ;#ifdef CONFIG_MPTCP Signature · 1 line
2.35.3
Re: Squash to "selftests/bpf: Add bpf_red scheduler": Build Failure
In reply to Geliang TangHi Geliang,
Thank you for your modifications, that's great!
But sadly, our CI spotted some issues with it when trying to build it.
You can find more details there:
https://patchwork.kernel.org/project/mptcp/patch/cae101ed32a7f97ffcd7d4e25e7c96cd85819e87.1687746021.git.geliang.tang@suse.com/
https://github.com/multipath-tcp/mptcp_net-next/actions/runs/5376448855
Status: failure
Initiator: MPTCPimporter
Commits: https://github.com/multipath-tcp/mptcp_net-next/commits/be983431b4a0
Feel free to reply to this email if you cannot access logs, if you need some support to fix the error, if this doesn't seem to be caused by your modifications or if the error is a false positive one.
Cheers,
MPTCP GH Action bot
Bot operated by Matthieu Baerts (Tessares)
Re: [PATCH mptcp-next v10 08/13] bpf: Add mptcp_storage helpers
In reply to Geliang TangQuoted 233 lines · Geliang Tang
Add two helpers
bpf_mptcp_storage_get()andbpf_mptcp_storage_delete()formptcp_storagemap type to get or delete abpf_local_storagefrom the given msk.Signed-off-by Geliang Tang <[email protected]>--- include/linux/bpf.h | 2 + include/linux/btf_ids.h | 1 + include/uapi/linux/bpf.h | 39 ++++++++++++++++ kernel/bpf/helpers.c | 4 ++ kernel/bpf/verifier.c | 13 +++++- net/mptcp/bpf.c | 84 ++++++++++++++++++++++++++++++++++ tools/include/uapi/linux/bpf.h | 39 ++++++++++++++++ 7 files changed, 181 insertions(+), 1 deletion(-)include/linux/bpf.h +2
-0@@ -2909,6 +2909,8 @@ extern const struct bpf_func_proto bpf_get_retval_proto;extern const struct bpf_func_proto bpf_user_ringbuf_drain_proto;extern const struct bpf_func_proto bpf_cgrp_storage_get_proto;extern const struct bpf_func_proto bpf_cgrp_storage_delete_proto; +extern const struct bpf_func_proto bpf_mptcp_storage_get_proto; +extern const struct bpf_func_proto bpf_mptcp_storage_delete_proto;const struct bpf_func_proto *tracing_prog_func_proto (enum bpf_func_id func_id,const struct bpf_prog *prog);include/linux/btf_ids.h +1
-0@@ -267,5 +267,6 @@ MAX_BTF_TRACING_TYPE,extern u32 btf_tracing_ids[];extern u32 bpf_cgroup_btf_id[];extern u32 bpf_local_storage_map_btf_id[]; +extern u32 bpf_mptcp_btf_id[];#endif include/uapi/linux/bpf.h +39
-0@@ -5570,6 +5570,43 @@ union bpf_attr {* 0 on success. * * **-ENOENT** if the bpf_local_storage cannot be found. +* +* void *bpf_mptcp_storage_get(struct bpf_map *map, struct mptcp_sock *msk, void *value, u64 flags) +* Description +* Get a bpf_local_storage from the *msk*. +* +* Logically, it could be thought of as getting the value from +* a *map* with *msk* as the **key**. From this +* perspective, the usage is not much different from +* **bpf_map_lookup_elem**\ (*map*, **&**\ *msk*) except this +* helper enforces the key must be a mptcp_sock and the map must also +* be a **BPF_MAP_TYPE_MPTCP_STORAGE**. +* +* Underneath, the value is stored locally at *msk* instead of +* the *map*. The *map* is used as the bpf-local-storage +* "type". The bpf-local-storage "type" (i.e. the *map*) is +* searched against all bpf_local_storage residing at *msk*. +* +* An optional *flags* (**BPF_LOCAL_STORAGE_GET_F_CREATE**) can be +* used such that a new bpf_local_storage will be +* created if one does not exist. *value* can be used +* together with **BPF_LOCAL_STORAGE_GET_F_CREATE** to specify +* the initial value of a bpf_local_storage. If *value* is +* **NULL**, the new bpf_local_storage will be zero initialized. +* Return +* A bpf_local_storage pointer is returned on success. +* +* **NULL** if not found or there was an error in adding +* a new bpf_local_storage. +* +* long bpf_mptcp_storage_delete(struct bpf_map *map, struct mptcp_sock *msk) +* Description +* Delete a bpf_local_storage from the *msk*. +* +* Return +* 0 on success. +* +* **-ENOENT** if the bpf_local_storage cannot be found. */ #define ___BPF_FUNC_MAPPER(FN, ctx...) \ FN(unspec, 0, ##ctx) \@@ -5784,6 +5821,8 @@ union bpf_attr { FN(user_ringbuf_drain, 209, ##ctx) \ FN(cgrp_storage_get, 210, ##ctx) \ FN(cgrp_storage_delete, 211, ##ctx) \ + FN(mptcp_storage_get, 212, ##ctx) \ + FN(mptcp_storage_delete, 213, ##ctx) \/* */ /* backwards-compatibility macros for users of __BPF_FUNC_MAPPER that don't kernel/bpf/helpers.c +4
-0@@ -1768,6 +1768,10 @@ bpf_base_func_proto(enum bpf_func_id func_id)case BPF_FUNC_get_current_ancestor_cgroup_id:return &bpf_get_current_ancestor_cgroup_id_proto;#endif +case BPF_FUNC_mptcp_storage_get: +return &bpf_mptcp_storage_get_proto; +case BPF_FUNC_mptcp_storage_delete: +return &bpf_mptcp_storage_delete_proto;default :break ; }kernel/bpf/verifier.c +12
-1@@ -553,7 +553,8 @@ static bool is_storage_get_function(enum bpf_func_id func_id)return func_id == BPF_FUNC_sk_storage_get || func_id == BPF_FUNC_inode_storage_get || func_id == BPF_FUNC_task_storage_get || - func_id == BPF_FUNC_cgrp_storage_get; + func_id == BPF_FUNC_cgrp_storage_get || + func_id == BPF_FUNC_mptcp_storage_get; } static bool helper_multiple_ref_obj_use(enum bpf_func_id func_id,@@ -8241,6 +8242,11 @@ static int check_map_func_compatibility(struct bpf_verifier_env *env, func_id != BPF_FUNC_map_push_elem) goto error;break ; +case BPF_MAP_TYPE_MPTCP_STORAGE : +if (func_id != BPF_FUNC_mptcp_storage_get && + func_id != BPF_FUNC_mptcp_storage_delete) + goto error; +break ;default :break ; }@@ -8353,6 +8359,11 @@ static int check_map_func_compatibility(struct bpf_verifier_env *env,if (map->map_type !=BPF_MAP_TYPE_CGRP_STORAGE ) goto error;break ; +case BPF_FUNC_mptcp_storage_get: +case BPF_FUNC_mptcp_storage_delete: +if (map->map_type !=BPF_MAP_TYPE_MPTCP_STORAGE ) + goto error; +break ;default :break ; }net/mptcp/bpf.c +64
-0@@ -201,6 +201,17 @@ static void bpf_mptcp_storage_unlock(void)migrate_enable (); } +static bool bpf_mptcp_storage_trylock (void ) +{ +migrate_disable (); +if (unlikely (this_cpu_inc_return (bpf_mptcp_storage_busy) !=1 )) { +this_cpu_dec (bpf_mptcp_storage_busy); +migrate_enable (); +return false; + } +return true; +} +static struct bpf_local_storage __rcu **mptcp_storage_ptr (void *owner) {struct mptcp_sock *msk = owner;@@ -305,6 +316,59 @@ static long bpf_mptcp_storage_delete_elem(struct bpf_map *map, void *key)return ret; } +/* Called by bpf_mptcp_storage_get*() helpers */ +static void *__bpf_mptcp_storage_get (struct bpf_map *map, +struct mptcp_sock *msk,void *value, +u64 flags,gfp_t gfp_flags,bool nobusy) +{ +struct bpf_local_storage_data *sdata; + + sdata =mptcp_storage_lookup (msk, map, nobusy); +if (sdata) +return sdata->data; + +if ((flags &BPF_LOCAL_STORAGE_GET_F_CREATE ) && nobusy) { + sdata =bpf_local_storage_update ( + msk, (struct bpf_local_storage_map *)map, value, +BPF_NOEXIST , gfp_flags); +return IS_ERR (sdata) ?NULL : sdata->data; + } + +return NULL ; +} + +/* *gfp_flags* is a hidden argument provided by the verifier */ +BPF_CALL_5(bpf_mptcp_storage_get, struct bpf_map *, map, struct mptcp_sock *, msk, + void *, value, u64, flags, gfp_t, gfp_flags) +{ +void *data; + +WARN_ON_ONCE (!bpf_rcu_lock_held ()); +if (flags & ~BPF_LOCAL_STORAGE_GET_F_CREATE || !msk) +return (unsigned long )NULL ; + +bpf_mptcp_storage_lock (); + data =__bpf_mptcp_storage_get (map, msk, value, flags, gfp_flags, true); +bpf_mptcp_storage_unlock (); +return (unsigned long )data; +} + +BPF_CALL_2(bpf_mptcp_storage_delete, struct bpf_map *, map, struct mptcp_sock *, msk) +{ +int ret; + +WARN_ON_ONCE (!bpf_rcu_lock_held ()); +if (!msk) +return -EINVAL ; + +if (!bpf_mptcp_storage_trylock ()) +return -EBUSY ; + + ret =mptcp_storage_delete (msk, map); +bpf_mptcp_storage_unlock (); +return ret; +} +static int notsupp_get_next_key (struct bpf_map *map,void *key,void *next_key) {return -ENOTSUPP ;@@ -334,6 +398,26 @@ const struct bpf_map_ops mptcp_storage_map_ops = { .map_btf_id = &bpf_local_storage_map_btf_id[0],
net/mptcp/bpf.c const struct bpf_map_ops mptcp_storage_map_ops = { · line 408
Quoted 10 lines
net/mptcp/bpf.c +8
-0.map_owner_storage_ptr = mptcp_storage_ptr, }; + +const struct bpf_func_proto bpf_mptcp_storage_get_proto = { + .func = bpf_mptcp_storage_get, + .gpl_only = false, + .ret_type =RET_PTR_TO_MAP_VALUE_OR_NULL , + .arg1_type =ARG_CONST_MAP_PTR , + .arg2_type =ARG_PTR_TO_BTF_ID_OR_NULL , + .arg2_btf_id = &bpf_mptcp_btf_id[0 ],
No need to add this bpf_mptcp_btf_id, we can use
&btf_sock_ids[BTF_SOCK_TYPE_MPTCP] instead.
net/mptcp/bpf.c const struct bpf_map_ops mptcp_storage_map_ops = { · line 419
Quoted 11 lines
net/mptcp/bpf.c +11
-0+ .arg3_type = ARG_PTR_TO_MAP_VALUE_OR_NULL, + .arg4_type = ARG_ANYTHING, +}; + +const struct bpf_func_proto bpf_mptcp_storage_delete_proto = { + .func = bpf_mptcp_storage_delete, + .gpl_only = false, + .ret_type =RET_INTEGER , + .arg1_type =ARG_CONST_MAP_PTR , + .arg2_type =ARG_PTR_TO_BTF_ID_OR_NULL , + .arg2_btf_id = &bpf_mptcp_btf_id[0 ],
Add here too.
-Geliang
Quoted 64 lines
net/mptcp/bpf.c +1
-0+};#endif /* CONFIG_BPF_JIT */ struct mptcp_sock *bpf_mptcp_sock_from_subflow (struct sock *sk)tools/include/uapi/linux/bpf.h +39
-0@@ -5570,6 +5570,43 @@ union bpf_attr {* 0 on success. * * **-ENOENT** if the bpf_local_storage cannot be found. +* +* void *bpf_mptcp_storage_get(struct bpf_map *map, struct mptcp_sock *msk, void *value, u64 flags) +* Description +* Get a bpf_local_storage from the *msk*. +* +* Logically, it could be thought of as getting the value from +* a *map* with *msk* as the **key**. From this +* perspective, the usage is not much different from +* **bpf_map_lookup_elem**\ (*map*, **&**\ *msk*) except this +* helper enforces the key must be a mptcp_sock and the map must also +* be a **BPF_MAP_TYPE_MPTCP_STORAGE**. +* +* Underneath, the value is stored locally at *msk* instead of +* the *map*. The *map* is used as the bpf-local-storage +* "type". The bpf-local-storage "type" (i.e. the *map*) is +* searched against all bpf_local_storage residing at *msk*. +* +* An optional *flags* (**BPF_LOCAL_STORAGE_GET_F_CREATE**) can be +* used such that a new bpf_local_storage will be +* created if one does not exist. *value* can be used +* together with **BPF_LOCAL_STORAGE_GET_F_CREATE** to specify +* the initial value of a bpf_local_storage. If *value* is +* **NULL**, the new bpf_local_storage will be zero initialized. +* Return +* A bpf_local_storage pointer is returned on success. +* +* **NULL** if not found or there was an error in adding +* a new bpf_local_storage. +* +* long bpf_mptcp_storage_delete(struct bpf_map *map, struct mptcp_sock *msk) +* Description +* Delete a bpf_local_storage from the *msk*. +* +* Return +* 0 on success. +* +* **-ENOENT** if the bpf_local_storage cannot be found. */ #define ___BPF_FUNC_MAPPER(FN, ctx...) \ FN(unspec, 0, ##ctx) \@@ -5784,6 +5821,8 @@ union bpf_attr { FN(user_ringbuf_drain, 209, ##ctx) \ FN(cgrp_storage_get, 210, ##ctx) \ FN(cgrp_storage_delete, 211, ##ctx) \ + FN(mptcp_storage_get, 212, ##ctx) \ + FN(mptcp_storage_delete, 213, ##ctx) \/* */ /* backwards-compatibility macros for users of __BPF_FUNC_MAPPER that don't Signature · 1 line
2.35.3
Re: Squash to "selftests/bpf: Add bpf_red scheduler": Tests Results
In reply to Geliang TangHi Geliang,
Thank you for your modifications, that's great!
Our CI did some validations and here is its report:
- KVM Validation: normal (except selftest_mptcp_join):
- Success! ✅:
- Task: https://cirrus-ci.com/task/5075409738399744
- Summary: https://api.cirrus-ci.com/v1/artifact/task/5075409738399744/summary/summary.txt
- KVM Validation: debug (except selftest_mptcp_join):
- Success! ✅:
- Task: https://cirrus-ci.com/task/4793934761689088
- Summary: https://api.cirrus-ci.com/v1/artifact/task/4793934761689088/summary/summary.txt
- KVM Validation: debug (only selftest_mptcp_join):
- Success! ✅:
- Task: https://cirrus-ci.com/task/5919834668531712
- Summary: https://api.cirrus-ci.com/v1/artifact/task/5919834668531712/summary/summary.txt
- KVM Validation: normal (only selftest_mptcp_join):
- Success! ✅:
- Task: https://cirrus-ci.com/task/6201309645242368
- Summary: https://api.cirrus-ci.com/v1/artifact/task/6201309645242368/summary/summary.txt
Initiator: Patchew Applier
Commits: https://github.com/multipath-tcp/mptcp_net-next/commits/be983431b4a0
If there are some issues, you can reproduce them using the same environment as the one used by the CI thanks to a docker image, e.g.:
$ cd [kernel source code]
$ docker run -v "${PWD}:${PWD}:rw" -w "${PWD}" --privileged --rm -it \
--pull always mptcp/mptcp-upstream-virtme-docker:latest \
auto-debugFor more details:
https://github.com/multipath-tcp/mptcp-upstream-virtme-docker
Please note that despite all the efforts that have been already done to have a stable tests suite when executed on a public CI like here, it is possible some reported issues are not due to your modifications. Still, do not hesitate to help us improve that ;-)
Cheers,
MPTCP GH Action bot
Bot operated by Matthieu Baerts (Tessares)