BPF round-robin scheduler
Subsystems (5)
v12 cover BPF round-robin scheduler
v12:
- init ssk from data->contexts[0], instead of msk->first.
- cycle through all the subflows, instead of the first two.
v11:
- rename array to contexts.
- drop number of subflows in mptcp_sched_data.
- set unused array elements to NULL.
- add MPTCP_SUBFLOWS_MAX check in mptcp_sched_data_init.
v10:
- init subflows array in mptcp_sched_data_init.
- for (int i = 0; i < data->subflows; i++) is not allowed in BPF, using this instead:
for (int i = 0; i < MPTCP_SUBFLOWS_MAX && i < data->subflows; i++)
- deponds on: "BPF packet scheduler" series v18.
v9:
- add subflows array in mptcp_sched_data
- deponds on: "BPF packet scheduler" series v17 + Squash to "mptcp: add struct mptcp_sched_ops v17".
v8:
- use struct mptcp_sched_data.
- deponds on: "BPF packet scheduler" series v14.
v7:
- rename retrans to reinject.
- drop last_snd setting.
- deponds on: "BPF packet scheduler" series v13.
v6:
- set call_me_again flag.
- deponds on: "BPF packet scheduler" series v12.
v5:
- update patch 2, use temporary storage instead.
- update patch 3, use new helpers.
- deponds on: "BPF packet scheduler" series v11.
v4:
- add retrans argment for get_subflow()
v3:
- add last_snd write access.
- keep msk->last_snd setting in get_subflow().
- deponds on: "BPF packet scheduler" series v10.
v2:
- merge the squash-to patch.
- implement bpf_mptcp_get_subflows helper, instead of bpf_mptcp_get_next_subflow.
- deponds on: "BPF packet scheduler v9".
This patchset implements round-robin scheduler using BPF. Address to some commends for the RFC version:
https://patchwork.kernel.org/project/mptcp/cover/[email protected]/
Geliang Tang (3):
mptcp: add subflows array in sched data
selftests: bpf: add bpf_rr scheduler
selftests: bpf: add bpf_rr test
include/net/mptcp.h | 2 + net/mptcp/sched.c | 14 ++++++ .../testing/selftests/bpf/bpf_mptcp_helpers.h | 8 ++++ .../testing/selftests/bpf/prog_tests/mptcp.c | 38 +++++++++++++++ .../selftests/bpf/progs/mptcp_bpf_rr.c | 47 +++++++++++++++++++ 5 files changed, 109 insertions(+) create mode 100644 tools/testing/selftests/bpf/progs/mptcp_bpf_rr.c
Signature · 1 line
2.34.1
PATCH v12 1/3 mptcp: add subflows array in sched data
This patch adds a subflow pointers array in struct mptcp_sched_data. Set the array before invoking get_subflow(), then get it in get_subflow() in the BPF contexts.
--- include/net/mptcp.h | 2 ++ net/mptcp/sched.c | 14 ++++++++++++++ tools/testing/selftests/bpf/bpf_mptcp_helpers.h | 2 ++ 3 files changed, 18 insertions(+)
include/net/mptcp.h +2 -0
@@ -96,10 +96,12 @@ struct mptcp_out_options { };#define MPTCP_SCHED_NAME_MAX 16 +#define MPTCP_SUBFLOWS_MAX 8struct mptcp_sched_data {struct sock *sock;bool call_again; +struct mptcp_subflow_context *contexts[MPTCP_SUBFLOWS_MAX ]; };struct mptcp_sched_ops {
net/mptcp/sched.c +14 -0
@@ -91,9 +91,23 @@ void mptcp_release_sched(struct mptcp_sock *msk)static int mptcp_sched_data_init (struct mptcp_sock *msk,struct mptcp_sched_data *data) { +struct mptcp_subflow_context *subflow; +int i =0 ; + data->sock =NULL ; data->call_again =0 ; +mptcp_for_each_subflow (msk, subflow) { +if (i ==MPTCP_SUBFLOWS_MAX ) { +pr_warn_once ("too many subflows" ); +break ; + } + data->contexts[i++] = subflow; + } + +for (; i <MPTCP_SUBFLOWS_MAX ; i++) + data->contexts[i++] =NULL ; +return 0 ; }
tools/testing/selftests/bpf/bpf_mptcp_helpers.h +2 -0
@@ -7,10 +7,12 @@#include "bpf_tcp_helpers.h" #define MPTCP_SCHED_NAME_MAX 16 +#define MPTCP_SUBFLOWS_MAX 8struct mptcp_sched_data {struct sock *sock;bool call_again; +struct mptcp_subflow_context *contexts[MPTCP_SUBFLOWS_MAX ]; };struct mptcp_sched_ops {
Signature · 1 line
2.34.1
PATCH v12 2/3 selftests: bpf: add bpf_rr scheduler
In reply to Geliang TangThis patch implements the round-robin BPF MPTCP scheduler, named bpf_rr, which always picks the next available subflow to send data. If no such next subflow available, picks the first one.
--- .../testing/selftests/bpf/bpf_mptcp_helpers.h | 6 +++ .../selftests/bpf/progs/mptcp_bpf_rr.c | 47 +++++++++++++++++++ 2 files changed, 53 insertions(+) create mode 100644 tools/testing/selftests/bpf/progs/mptcp_bpf_rr.c
tools/testing/selftests/bpf/bpf_mptcp_helpers.h +6 -0
@@ -29,10 +29,16 @@ struct mptcp_sched_ops {struct mptcp_sock {struct inet_connection_sock sk; +struct sock *last_snd;__u32 token;struct sock *first;struct mptcp_sched_ops *sched;char ca_name[TCP_CA_NAME_MAX ]; } __attribute__((preserve_access_index)); +struct mptcp_subflow_context { +__u32 token; +struct sock *tcp_sock;/* tcp sk backpointer */ +} __attribute__((preserve_access_index)); +#endif
tools/testing/selftests/bpf/progs/mptcp_bpf_rr.c new file +47 -0
47 lines not on this page · raw message
Signature · 1 line
2.34.1
PATCH v12 3/3 selftests: bpf: add bpf_rr test
In reply to Geliang TangThis patch adds the round-robin BPF MPTCP scheduler test. Use sysctl to set net.mptcp.scheduler to use this sched. Add a veth net device to simulate the multiple addresses case. Use 'ip mptcp endpoint' command to add this new endpoint to PM netlink.
--- .../testing/selftests/bpf/prog_tests/mptcp.c | 38 +++++++++++++++++++ 1 file changed, 38 insertions(+)
tools/testing/selftests/bpf/prog_tests/mptcp.c +38 -0
@@ -5,6 +5,7 @@#include "cgroup_helpers.h" #include "network_helpers.h" #include "mptcp_bpf_first.skel.h" +#include "mptcp_bpf_rr.skel.h" #ifndef TCP_CA_NAME_MAX #define TCP_CA_NAME_MAX 16
@@ -376,10 +377,47 @@ static void test_first(void)mptcp_bpf_first__destroy (first_skel); } +static void test_rr (void ) +{ +struct mptcp_bpf_rr *rr_skel; +int server_fd, client_fd; +struct bpf_link *link; + + rr_skel =mptcp_bpf_rr__open_and_load (); +if (CHECK (!rr_skel,"bpf_rr__open_and_load" ,"failed\n" )) +return ; + + link =bpf_map__attach_struct_ops (rr_skel->maps.rr); +if (!ASSERT_OK_PTR (link,"bpf_map__attach_struct_ops" )) { +mptcp_bpf_rr__destroy (rr_skel); +return ; + } + +system ("ip link add veth1 type veth" ); +system ("ip addr add 10.0.1.1/24 dev veth1" ); +system ("ip link set veth1 up" ); +system ("ip mptcp endpoint add 10.0.1.1 subflow" ); +system ("sysctl -qw net.mptcp.scheduler=bpf_rr" ); + server_fd =start_mptcp_server (AF_INET ,NULL ,0 ,0 ); + client_fd =connect_to_mptcp_fd (server_fd,0 ); + +send_data (server_fd, client_fd); + +close (client_fd); +close (server_fd); +system ("sysctl -qw net.mptcp.scheduler=default" ); +system ("ip mptcp endpoint flush" ); +system ("ip link del veth1" ); +bpf_link__destroy (link); +mptcp_bpf_rr__destroy (rr_skel); +} +void test_mptcp (void ) {if (test__start_subtest ("base" ))test_base ();if (test__start_subtest ("first" ))test_first (); +if (test__start_subtest ("rr" )) +test_rr (); }
Signature · 1 line
2.34.1
Re: selftests: bpf: add bpf_rr test: Tests Results
Hi Geliang,
Thank you for your modifications, that's great!
Our CI did some validations and here is its report:
- KVM Validation: normal:
- Success! ✅:
- Task: https://cirrus-ci.com/task/6384460542246912
- Summary: https://api.cirrus-ci.com/v1/artifact/task/6384460542246912/summary/summary.txt
- KVM Validation: debug:
- Unstable: 3 failed test(s): packetdrill_add_addr selftest_diag selftest_mptcp_join 🔴:
- Task: https://cirrus-ci.com/task/4977085658693632
- Summary: https://api.cirrus-ci.com/v1/artifact/task/4977085658693632/summary/summary.txt
Initiator: Patchew Applier
Commits: https://github.com/multipath-tcp/mptcp_net-next/commits/5dd02ab25990
If there are some issues, you can reproduce them using the same environment as the one used by the CI thanks to a docker image, e.g.:
$ cd [kernel source code]
$ docker run -v "${PWD}:${PWD}:rw" -w "${PWD}" --privileged --rm -it \
--pull always mptcp/mptcp-upstream-virtme-docker:latest \
auto-debugFor more details:
https://github.com/multipath-tcp/mptcp-upstream-virtme-docker
Please note that despite all the efforts that have been already done to have a stable tests suite when executed on a public CI like here, it is possible some reported issues are not due to your modifications. Still, do not hesitate to help us improve that ;-)
Cheers,
MPTCP GH Action bot
Bot operated by Matthieu Baerts (Tessares)
Quoted 4 lines · Geliang Tang
v12:
- init ssk from data->contexts[0], instead ofmsk->first.
- cycle through all the subflows, instead of the first two.
v12 looks ok for the export branch, but Andrii asked for us to use bpf_tcp_helpers.h instead of adding a new header file. Geliang, I think it would be easier to do a v13 with the header changes rather than apply v12 and squash header changes.
- Mat
Quoted 62 lines
v11:
- rename array to contexts.
- drop number of subflows inmptcp_sched_data.
- set unused array elements to NULL.
- addMPTCP_SUBFLOWS_MAXcheck inmptcp_sched_data_init.v10:
- init subflows array inmptcp_sched_data_init.
- for (int i = 0; i <data->subflows; i++) is not allowed in BPF, using this instead:for (int i = 0; i < MPTCP_SUBFLOWS_MAX && i < data->subflows; i++)- deponds on: "BPF packet scheduler" series v18.
v9:
- add subflows array inmptcp_sched_data
- deponds on: "BPF packet scheduler" series v17 + Squash to "mptcp: add structmptcp_sched_opsv17".v8:
- use structmptcp_sched_data.
- deponds on: "BPF packet scheduler" series v14.v7:
- rename retrans to reinject.
- droplast_sndsetting.
- deponds on: "BPF packet scheduler" series v13.v6:
- setcall_me_againflag.
- deponds on: "BPF packet scheduler" series v12.v5:
- update patch 2, use temporary storage instead.
- update patch 3, use new helpers.
- deponds on: "BPF packet scheduler" series v11.v4:
- add retrans argment forget_subflow()v3:
- addlast_sndwrite access.
- keepmsk->last_sndsetting inget_subflow().
- deponds on: "BPF packet scheduler" series v10.v2:
- merge the squash-to patch.
- implementbpf_mptcp_get_subflowshelper, instead ofbpf_mptcp_get_next_subflow.
- deponds on: "BPF packet scheduler v9".This patchset implements round-robin scheduler using BPF. Address to some commends for the RFC version:
https://patchwork.kernel.org/project/mptcp/cover/[email protected]/
Geliang Tang (3):
mptcp: add subflows array in sched data
selftests: bpf: addbpf_rrscheduler
selftests: bpf: addbpf_rrtest
Quoted 13 lines
include/net/mptcp.h | 2 + net/mptcp/sched.c | 14 ++++++ .../testing/selftests/bpf/bpf_mptcp_helpers.h | 8 ++++ .../testing/selftests/bpf/prog_tests/mptcp.c | 38 +++++++++++++++ .../selftests/bpf/progs/mptcp_bpf_rr.c | 47 +++++++++++++++++++ 5 files changed, 109 insertions(+)create mode 100644
tools/testing/selftests/bpf/progs/mptcp_bpf_rr.cSignature · 1 line
2.34.1
--
Mat Martineau
Intel
Hi Mat, Geliang,
7 May 2022 02:06:25 Mat Martineau <[email protected]>:
Quoted 8 lines · Geliang Tang
Quoted 5 lines
v12:
- init ssk from data->contexts[0], instead ofmsk->first.
- cycle through all the subflows, instead of the first two.v12 looks ok for the export branch, but Andrii asked for us to use
bpf_tcp_helpers.hinstead of adding a new header file. Geliang, I think it would be easier to do a v13 with the header changes rather than apply v12 and squash header changes.
I think Andrii is not opposed to have a dedicated MPTCP helper file because from what I understood, he only asked why we created a new file, no?
For me it makes sense to have this dedicated file. The main reason is related to the maintenance: to have us being cc'd for all patches modifying this file. Would it not be easier with a dedicated file?
Cheers,
Matt
--
Tessares | Belgium | Hybrid Access Solutions
www.tessares.net
Quoted 12 lines · Matthieu Baerts <[email protected]> 于2022年5月7日周六 13:34写道:
Hi Mat, Geliang,
7 May 2022 02:06:25 Mat Martineau <[email protected]>:
Quoted 7 lines · Geliang Tang
Quoted 5 lines
v12:
- init ssk from data->contexts[0], instead ofmsk->first.
- cycle through all the subflows, instead of the first two.
Quoted 5 lines
Quoted 1 line
v12 looks ok for the export branch, but Andrii asked for us to use
bpf_tcp_helpers.hinstead of adding a new header file. Geliang, I think it would be easier to do a v13 with the header changes rather than apply v12 and squash header changes.I think Andrii is not opposed to have a dedicated MPTCP helper file because from what I understood, he only asked why we created a new file, no?
For me it makes sense to have this dedicated file. The main reason is related to the maintenance: to have us being cc'd for all patches modifying this file. Would it not be easier with a dedicated file?
Hi Matt, Mat,
I just sent out 5 squash-to patches to address to Andrii's other comments except this bpf_tcp_helpers.h one.
Thanks,
-Geliang
Quoted 7 lines
Cheers,
Matt
--
Tessares | Belgium | Hybrid Access Solutions
www.tessares.net