BPF packet scheduler updates part 3
Subsystems (5)
v3 cover BPF packet scheduler updates part 3
v3:
- define tcp_rtx_and_write_queues_empty() and sk_stream_memory_free() in
BPF context.
v2:
- use __always_inline instead of inline.
Some cleanups.
Geliang Tang (6):
Squash to "mptcp: add sched_data helpers" Squash to "bpf: Add bpf_mptcp_sched_kfunc_set" Squash to "selftests/bpf: add two mptcp netns helpers" Squash to "selftests/bpf: Add bpf_rr scheduler" Squash to "bpf: Export more bpf_burst related functions" Squash to "selftests/bpf: Add bpf_burst scheduler"
net/mptcp/bpf.c | 19 ++++---- net/mptcp/protocol.h | 6 --- net/mptcp/sched.c | 6 +++ tools/testing/selftests/bpf/bpf_tcp_helpers.h | 3 +- .../testing/selftests/bpf/prog_tests/mptcp.c | 1 - .../selftests/bpf/progs/mptcp_bpf_burst.c | 43 +++++++++++++++---- 6 files changed, 52 insertions(+), 26 deletions(-)
Signature · 1 line
2.35.3
PATCH v3 1/6 Squash to "mptcp: add sched_data helpers"
Add "ignore -Wmissing-prototypes" for mptcp_sched_data_set_contexts() and mptcp_subflow_ctx_by_pos(). Then the declarations of these functions in protocol.h can be dropped.
--- net/mptcp/protocol.h | 4 ---- net/mptcp/sched.c | 6 ++++++ 2 files changed, 6 insertions(+), 4 deletions(-)
net/mptcp/protocol.h +0 -4
@@ -671,10 +671,6 @@ int mptcp_init_sched(struct mptcp_sock *msk,void mptcp_release_sched (struct mptcp_sock *msk);void mptcp_subflow_set_scheduled (struct mptcp_subflow_context *subflow,bool scheduled); -void mptcp_sched_data_set_contexts (const struct mptcp_sock *msk, -struct mptcp_sched_data *data); -struct mptcp_subflow_context * -mptcp_subflow_ctx_by_pos (const struct mptcp_sched_data *data,unsigned int pos);struct sock *mptcp_subflow_get_send (struct mptcp_sock *msk);struct sock *mptcp_subflow_get_retrans (struct mptcp_sock *msk);int mptcp_sched_get_send (struct mptcp_sock *msk);
net/mptcp/sched.c +6 -0
@@ -127,6 +127,10 @@ void mptcp_subflow_set_scheduled(struct mptcp_subflow_context *subflow,WRITE_ONCE (subflow->scheduled, scheduled); } +__diag_push (); +__diag_ignore_all ("-Wmissing-prototypes" , +"kfuncs which will be used in BPF programs" ); +void mptcp_sched_data_set_contexts (const struct mptcp_sock *msk,struct mptcp_sched_data *data) {
@@ -155,6 +159,8 @@ mptcp_subflow_ctx_by_pos(const struct mptcp_sched_data *data, unsigned int pos)return data->contexts[pos]; } +__diag_pop (); +int mptcp_sched_get_send (struct mptcp_sock *msk) {struct mptcp_subflow_context *subflow;
Signature · 1 line
2.35.3
Re: [PATCH mptcp-next v3 1/6] Squash to "mptcp: add sched_data helpers"
Hi Geliang,
Quoted 3 lines · Geliang Tang
Add "ignore -Wmissing-prototypes" for
mptcp_sched_data_set_contexts()andmptcp_subflow_ctx_by_pos(). Then the declarations of these functions inprotocol.hcan be dropped.
I just had to revert this patch because it was causing issues reported by the CI when using 'sparce':
Quoted 3 lines
$ touch net/mptcp/sched.c && make C=1 net/mptcp/sched.o net/mptcp/sched.c:203:6: warning: symbol 'mptcp_sched_data_set_contexts' was not declared. Should it be static? net/mptcp/sched.c:223:30: warning: symbol 'mptcp_subflow_ctx_by_pos' was not declared. Should it be static?
Would it not be better to move these two functions (mptcp_sched_data_set_contexts - mptcp_subflow_ctx_by_pos) to net/mptcp/bpf.c where they are "used" with "BTF_ID_FLAGS(func, <name>)"?
Also, when I look at other kfunc in the kernel (e.g. bpf_skb_set_fou_encap()), I see that they are also always declared where they are being "used" (BTF_ID_FLAGS(func, <name>)) but also they are declared with "__bpf_kfunc" and the function name is prefixed with "bpf_" (which makes sense if they are BPF specific). Should we not do the same here with these 2 functions + bpf_mptcp_subflow_queues_empty() (only "__bpf_kfunc" is missing for this one, it already has the prefix and is declared where it is "used")?
About the revert:
New patches for t/upstream:
- 1701483e1326: Revert "Squash to "mptcp: add sched_data helpers""
- Results: ac006cd93f91..d30c287ad522 (export)
Tests are now in progress:
https://cirrus-ci.com/github/multipath-tcp/mptcp_net-next/export/20230816T103753
Cheers,
Matt
Signature · 2 lines
Tessares | Belgium | Hybrid Access Solutions www.tessares.net
PATCH v3 2/6 Squash to "bpf: Add bpf_mptcp_sched_kfunc_set"
In reply to Geliang TangRename bpf_mptcp_sched_kfunc_init to bpf_mptcp_kfunc_init. Since some kfuncs related "mptcpify" will init in this function too.
--- net/mptcp/bpf.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
net/mptcp/bpf.c +2 -2
@@ -172,12 +172,12 @@ static const struct btf_kfunc_id_set bpf_mptcp_sched_kfunc_set = { .set = &bpf_mptcp_sched_kfunc_ids, }; -static int __init bpf_mptcp_sched_kfunc_init (void ) +static int __init bpf_mptcp_kfunc_init (void ) {return register_btf_kfunc_id_set (BPF_PROG_TYPE_STRUCT_OPS , &bpf_mptcp_sched_kfunc_set); } -late_initcall (bpf_mptcp_sched_kfunc_init); +late_initcall (bpf_mptcp_kfunc_init);#endif /* CONFIG_BPF_JIT */ struct mptcp_sock *bpf_mptcp_sock_from_subflow (struct sock *sk)
Signature · 1 line
2.35.3
PATCH v3 3/6 Squash to "selftests/bpf: add two mptcp netns helpers"
In reply to Geliang TangJust drop a black line.
--- tools/testing/selftests/bpf/prog_tests/mptcp.c | 1 - 1 file changed, 1 deletion(-)
tools/testing/selftests/bpf/prog_tests/mptcp.c +0 -1
@@ -198,7 +198,6 @@ static void test_base(void) fail:cleanup_netns (nstoken); -close (cgroup_fd); }
Signature · 1 line
2.35.3
PATCH v3 4/6 Squash to "selftests/bpf: Add bpf_rr scheduler"
In reply to Geliang TangUse __always_inline instead of inline.
--- tools/testing/selftests/bpf/bpf_tcp_helpers.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
tools/testing/selftests/bpf/bpf_tcp_helpers.h +1 -1
@@ -274,7 +274,7 @@ extern void mptcp_sched_data_set_contexts(const struct mptcp_sock *msk, struct mptcp_sched_data *data) __ksym;extern struct mptcp_subflow_context *mptcp_subflow_ctx_by_pos (const struct mptcp_sched_data *data,unsigned int pos) __ksym; -static inline struct sock * +static __always_inline struct sock *mptcp_subflow_tcp_sock (const struct mptcp_subflow_context *subflow) {return subflow->tcp_sock;
Signature · 1 line
2.35.3
PATCH v3 5/6 Squash to "bpf: Export more bpf_burst related functions"
In reply to Geliang TangAdd "ignore -Wmissing-prototypes" for bpf_mptcp_subflow_queues_empty(), drop the declarations in protocol.h.
Drop bpf_mptcp_subflow_memory_free(), it will be defined in BPF context.
Update bpf_mptcp_subflow_queues_empty().
--- net/mptcp/bpf.c | 15 ++++++++------- net/mptcp/protocol.h | 2 -- 2 files changed, 8 insertions(+), 9 deletions(-)
net/mptcp/bpf.c +8 -7
@@ -145,16 +145,17 @@ struct bpf_struct_ops bpf_mptcp_sched_ops = { .name = "mptcp_sched_ops", }; -bool bpf_mptcp_subflow_memory_free (const struct sock *sk) -{ -return sk_stream_memory_free (sk); -} +__diag_push (); +__diag_ignore_all ("-Wmissing-prototypes" , +"kfuncs which will be used in BPF programs" ); -bool bpf_mptcp_subflow_queues_empty (const struct sock *sk) +bool bpf_mptcp_subflow_queues_empty (struct sock *sk) { -return tcp_rtx_and_write_queues_empty (sk); +return tcp_rtx_queue_empty (sk); } +__diag_pop (); + BTF_SET8_START(bpf_mptcp_sched_kfunc_ids) BTF_ID_FLAGS(func, mptcp_subflow_set_scheduled) BTF_ID_FLAGS(func, mptcp_sched_data_set_contexts)
@@ -162,7 +163,7 @@ BTF_ID_FLAGS(func, mptcp_subflow_ctx_by_pos) BTF_ID_FLAGS(func, mptcp_subflow_active) BTF_ID_FLAGS(func, mptcp_set_timeout) BTF_ID_FLAGS(func, mptcp_wnd_end) -BTF_ID_FLAGS(func, bpf_mptcp_subflow_memory_free) +BTF_ID_FLAGS(func, tcp_stream_memory_free) BTF_ID_FLAGS(func, bpf_mptcp_subflow_queues_empty) BTF_ID_FLAGS(func, mptcp_pm_subflow_chk_stale)BTF_SET8_END (bpf_mptcp_sched_kfunc_ids)
net/mptcp/protocol.h +0 -2
@@ -642,8 +642,6 @@ void mptcp_subflow_queue_clean(struct sock *sk, struct sock *ssk);void mptcp_sock_graft (struct sock *sk,struct socket *parent);u64 mptcp_wnd_end (const struct mptcp_sock *msk);void mptcp_set_timeout (struct sock *sk); -bool bpf_mptcp_subflow_memory_free (const struct sock *sk); -bool bpf_mptcp_subflow_queues_empty (const struct sock *sk);struct sock *__mptcp_nmpc_sk (struct mptcp_sock *msk);bool __mptcp_close (struct sock *sk,long timeout);void mptcp_cancel_work (struct sock *sk);
Signature · 1 line
2.35.3
Re: [PATCH mptcp-next v3 5/6] Squash to "bpf: Export more bpf_burst related functions"
Hi Geliang,
Quoted 6 lines · Geliang Tang
Add "ignore -Wmissing-prototypes" for
bpf_mptcp_subflow_queues_empty(), drop the declarations inprotocol.h.Drop
bpf_mptcp_subflow_memory_free(), it will be defined in BPF context.Update
bpf_mptcp_subflow_queues_empty().
I also had to partially revert this commit.
Maybe "__bpf_kfunc" would avoid such issues reported by Sparse?
#define __bpf_kfunc __used noinline
Do you mind looking at that please?
New patches for t/upstream:
- 11ace9718934: mptcp: partially revert "Squash to "bpf: Export more bpf_burst related functions""
- Results: d30c287ad522..dd7fb06cb618 (export)
Tests are now in progress:
https://cirrus-ci.com/github/multipath-tcp/mptcp_net-next/export/20230816T115317
Cheers,
Matt
Signature · 2 lines
Tessares | Belgium | Hybrid Access Solutions www.tessares.net
Re: [PATCH mptcp-next v3 5/6] Squash to "bpf: Export more bpf_burst related functions"
Quoted 10 lines · Matthieu Baerts
Hi Geliang,
Quoted 6 lines · Geliang Tang
Add "ignore -Wmissing-prototypes" for
bpf_mptcp_subflow_queues_empty(), drop the declarations inprotocol.h.Drop
bpf_mptcp_subflow_memory_free(), it will be defined in BPF context.Update
bpf_mptcp_subflow_queues_empty().
Quoted 7 lines
I also had to partially revert this commit.
Maybe "
__bpf_kfunc" would avoid such issues reported by Sparse?#define __bpf_kfunc __used noinlineDo you mind looking at that please?
Hi Matt,
"__bpf_kfunc" doesn't work. I have to ask BPF maintainers what to do for this and hope to receive their response.
Thanks,
-Geliang
Quoted 15 lines
New patches for t/upstream:
-11ace9718934: mptcp: partially revert "Squash to "bpf: Export morebpf_burstrelated functions""
- Results:d30c287ad522..dd7fb06cb618(export)Tests are now in progress:
https://cirrus-ci.com/github/multipath-tcp/mptcp_net-next/export/20230816T115317
Cheers,
MattSignature · 2 lines
Tessares | Belgium | Hybrid Access Solutions www.tessares.net
Re: [PATCH mptcp-next v3 5/6] Squash to "bpf: Export more bpf_burst related functions"
Hi Geliang,
Quoted 15 lines · Geliang Tang
Quoted 14 lines · Matthieu Baerts
Hi Geliang,
Quoted 6 lines · Geliang Tang
Add "ignore -Wmissing-prototypes" for
bpf_mptcp_subflow_queues_empty(), drop the declarations inprotocol.h.Drop
bpf_mptcp_subflow_memory_free(), it will be defined in BPF context.Update
bpf_mptcp_subflow_queues_empty().I also had to partially revert this commit.
Maybe "
__bpf_kfunc" would avoid such issues reported by Sparse?
Quoted 8 lines
Quoted 3 lines
#define __bpf_kfunc __used noinlineDo you mind looking at that please?
Hi Matt,
"
__bpf_kfunc" doesn't work. I have to ask BPF maintainers what to do for this and hope to receive their response.
Thank you for having asked!
Good idea to have asked on the previous patch, I didn't see that netdev's CI complained about that on this one:
We can see a similar warning for update_socket_protocol() in netdev/build_allmodconfig_warn and netdev/build_32bit results.
Cheers,
Matt
Signature · 2 lines
Tessares | Belgium | Hybrid Access Solutions www.tessares.net
PATCH v3 6/6 Squash to "selftests/bpf: Add bpf_burst scheduler"
In reply to Geliang TangUse __always_inline instead of inline.
Define tcp_rtx_and_write_queues_empty() and sk_stream_memory_free() in BPF context.
--- tools/testing/selftests/bpf/bpf_tcp_helpers.h | 1 + .../selftests/bpf/progs/mptcp_bpf_burst.c | 43 +++++++++++++++---- 2 files changed, 35 insertions(+), 9 deletions(-)
tools/testing/selftests/bpf/bpf_tcp_helpers.h +1 -0
@@ -36,6 +36,7 @@ enum sk_pacing {struct sock {struct sock_common __sk_common;#define sk_state __sk_common.skc_state +int sk_sndbuf;int sk_wmem_queued;unsigned long sk_pacing_rate;__u32 sk_pacing_status;/* see enum sk_pacing */
tools/testing/selftests/bpf/progs/mptcp_bpf_burst.c +34 -9
70 lines not on this page · raw message
Signature · 1 line
2.35.3
Re: Squash to "selftests/bpf: Add bpf_burst scheduler": Build Failure
Hi Geliang,
Thank you for your modifications, that's great!
But sadly, our CI spotted some issues with it when trying to build it.
You can find more details there:
https://patchwork.kernel.org/project/mptcp/patch/4f65c2b44b82e8682abcf5c5b91dceced0f07480.1691036765.git.geliang.tang@suse.com/
https://github.com/multipath-tcp/mptcp_net-next/actions/runs/5746700485
Status: failure
Initiator: MPTCPimporter
Commits: https://github.com/multipath-tcp/mptcp_net-next/commits/1f357c615e99
Feel free to reply to this email if you cannot access logs, if you need some support to fix the error, if this doesn't seem to be caused by your modifications or if the error is a false positive one.
Cheers,
MPTCP GH Action bot
Bot operated by Matthieu Baerts (Tessares)
Re: Squash to "selftests/bpf: Add bpf_burst scheduler": Tests Results
In reply to Geliang TangHi Geliang,
Thank you for your modifications, that's great!
Our CI did some validations and here is its report:
- KVM Validation: normal (except selftest_mptcp_join):
- Success! ✅:
- Task: https://cirrus-ci.com/task/5651357470818304
- Summary: https://api.cirrus-ci.com/v1/artifact/task/5651357470818304/summary/summary.txt
- KVM Validation: debug (except selftest_mptcp_join):
- Success! ✅:
- Task: https://cirrus-ci.com/task/6214307424239616
- Summary: https://api.cirrus-ci.com/v1/artifact/task/6214307424239616/summary/summary.txt
- KVM Validation: normal (only selftest_mptcp_join):
- Success! ✅:
- Task: https://cirrus-ci.com/task/5088407517396992
- Summary: https://api.cirrus-ci.com/v1/artifact/task/5088407517396992/summary/summary.txt
- KVM Validation: debug (only selftest_mptcp_join):
- Success! ✅:
- Task: https://cirrus-ci.com/task/4806932540686336
- Summary: https://api.cirrus-ci.com/v1/artifact/task/4806932540686336/summary/summary.txt
Initiator: Patchew Applier
Commits: https://github.com/multipath-tcp/mptcp_net-next/commits/1f357c615e99
If there are some issues, you can reproduce them using the same environment as the one used by the CI thanks to a docker image, e.g.:
$ cd [kernel source code]
$ docker run -v "${PWD}:${PWD}:rw" -w "${PWD}" --privileged --rm -it \
--pull always mptcp/mptcp-upstream-virtme-docker:latest \
auto-debugFor more details:
https://github.com/multipath-tcp/mptcp-upstream-virtme-docker
Please note that despite all the efforts that have been already done to have a stable tests suite when executed on a public CI like here, it is possible some reported issues are not due to your modifications. Still, do not hesitate to help us improve that ;-)
Cheers,
MPTCP GH Action bot
Bot operated by Matthieu Baerts (Tessares)
Mat Martineau
maintainerQuoted 3 lines · Geliang Tang
v3:
- definetcp_rtx_and_write_queues_empty()andsk_stream_memory_free()in BPF context.
v3 looks good to squash, thanks Geliang.
- Mat
Quoted 27 lines
v2:
- use__always_inlineinstead of inline.Some cleanups.
Geliang Tang (6):
Squash to "mptcp: addsched_datahelpers" Squash to "bpf: Addbpf_mptcp_sched_kfunc_set" Squash to "selftests/bpf: add two mptcp netns helpers" Squash to "selftests/bpf: Addbpf_rrscheduler" Squash to "bpf: Export morebpf_burstrelated functions" Squash to "selftests/bpf: Addbpf_burstscheduler"net/mptcp/bpf.c | 19 ++++---- net/mptcp/protocol.h | 6 --- net/mptcp/sched.c | 6 +++ tools/testing/selftests/bpf/bpf_tcp_helpers.h | 3 +- .../testing/selftests/bpf/prog_tests/mptcp.c | 1 - .../selftests/bpf/progs/mptcp_bpf_burst.c | 43 +++++++++++++++---- 6 files changed, 52 insertions(+), 26 deletions(-)Signature · 1 line
2.35.3
Hi Geliang, Mat,
Quoted 7 lines · Geliang Tang
Geliang Tang (6):
Squash to "mptcp: addsched_datahelpers" Squash to "bpf: Addbpf_mptcp_sched_kfunc_set" Squash to "selftests/bpf: add two mptcp netns helpers" Squash to "selftests/bpf: Addbpf_rrscheduler" Squash to "bpf: Export morebpf_burstrelated functions" Squash to "selftests/bpf: Addbpf_burstscheduler"
Thank you for the patches and the reviews!
Now in our tree:
New patches for t/upstream:
- a88f3f498b2f: "squashed" patch 1/6 (with conflicts) in "mptcp: add sched_data helpers"
- 5d8795774463: "squashed" patch 2/6 in "bpf: Add bpf_mptcp_sched_kfunc_set"
- be959ff059e4: "squashed" patch 3/6 in "selftests/bpf: add two mptcp netns helpers"
- 1736fcc031f4: "squashed" patch 4/6 in "selftests/bpf: Add bpf_rr scheduler"
- 574847a8b558: "squashed" patch 5/6 in "bpf: Export more bpf_burst related functions"
- 206993ff5b4f: "squashed" patch 6/6 in "selftests/bpf: Add bpf_burst scheduler"
- Results: 2d42b08369ef..03fd7c30320a (export)
Tests are now in progress:
https://cirrus-ci.com/github/multipath-tcp/mptcp_net-next/export/20230811T205838
Cheers,
Matt
Signature · 2 lines
Tessares | Belgium | Hybrid Access Solutions www.tessares.net