mptcp: receive buffer auto-tuning
Subsystems (3)
Florian Westphal
cover mptcp: receive buffer auto-tuning
First patch adds switch to test script so it can be run with a fixed file size rather than a random one.
Second patch adds receive buffer autotuning. I rebased this on top of mptcp-next as per Christophs patch suggestion to make sure the initialisation function has been run for the fallback case too.
I suggest to apply this to mptcp-next so this can be pushed to net-next once that opens up again.
Florian Westphal
PATCH 1/2 selftests: mptcp: add option to specify size of file to transfer
The script generates two random files that are then sent via tcp and mptcp connections.
In order to compare throughput over consecutive runs add an option to provide the file size on the command line: "-f 128000".
Also add an option, -t, to enable tcp tests. This is useful to compare throughput of mptcp connections and tcp connections.
Example: run tests with a 4mb file size, 300ms delay 0.01% loss, default gso/tso/gro settings and with large write/blocking io:
mptcp_connect.sh -t -f $((4 * 1024 * 1024)) -d 300 -l 0.01% -r 0 -e "" -m mmap
--- .../selftests/net/mptcp/mptcp_connect.sh | 52 ++++++++++++++----- 1 file changed, 39 insertions(+), 13 deletions(-)
tools/testing/selftests/net/mptcp/mptcp_connect.sh +39 -13
@@ -3,7 +3,7 @@ time_start=$(date +%s) -optstring="S:R:d:e:l:r:h4cm:" +optstring="S:R:d:e:l:r:h4cm:f:t" ret=0 sin="" sout=""
@@ -21,6 +21,8 @@ testmode="" sndbuf=0 rcvbuf=0 options_log=true +do_tcp=0 +filesize=0 if [ $tc_loss -eq 100 ];then tc_loss=1%
@@ -40,9 +42,11 @@ usage() { echo -e "\t-e: ethtool features to disable, e.g.: \"-e tso -e gso\" (default: randomly disable any of tso/gso/gro)" echo -e "\t-4: IPv4 only: disable IPv6 tests (default: test both IPv4 and IPv6)" echo -e "\t-c: capture packets for each test using tcpdump (default: no capture)" + echo -e "\t-s: size of file to transfer in bytes (default random)" echo -e "\t-S: set sndbuf value (default: use kernel default)" echo -e "\t-R: set rcvbuf value (default: use kernel default)" echo -e "\t-m: test mode (poll, sendfile; default: poll)" + echo -e "\t-t: also run tests with TCP (use twice to non-fallback tcp)" } while getopts "$optstring" option;do
@@ -94,6 +98,12 @@ while getopts "$optstring" option;do "m") testmode="$OPTARG" ;; + "f") + filesize="$OPTARG" + ;; + "t") + do_tcp=$((do_tcp+1)) + ;; "?") usage $0 exit 1
@@ -449,20 +459,25 @@ make_file() { local name=$1 local who=$2 + local SIZE=$filesize + local ksize + local rem - local SIZE TSIZE - SIZE=$((RANDOM % (1024 * 8))) - TSIZE=$((SIZE * 1024)) + if [ $SIZE -eq 0 ]; then + local MAXSIZE=$((1024 * 1024 * 8)) + local MINSIZE=$((1024 * 256)) - dd if=/dev/urandom of="$name" bs=1024 count=$SIZE 2> /dev/null + SIZE=$(((RANDOM * RANDOM + MINSIZE) % MAXSIZE)) + fi - SIZE=$((RANDOM % 1024)) - SIZE=$((SIZE + 128)) - TSIZE=$((TSIZE + SIZE)) - dd if=/dev/urandom conv=notrunc of="$name" bs=1 count=$SIZE 2> /dev/null + ksize=$((SIZE / 1024)) + rem=$((SIZE - (ksize * 1024))) + + dd if=/dev/urandom of="$name" bs=1024 count=$ksize 2> /dev/null + dd if=/dev/urandom conv=notrunc of="$name" bs=1 count=$rem 2> /dev/null echo -e "\nMPTCP_TEST_FILE_END_MARKER" >> "$name" - echo "Created $name (size $TSIZE) containing data sent by $who" + echo "Created $name (size $(du -b "$name")) containing data sent by $who" } run_tests_lo()
@@ -497,9 +512,11 @@ run_tests_lo() return 1 fi - # don't bother testing fallback tcp except for loopback case. - if [ ${listener_ns} != ${connector_ns} ]; then - return 0 + if [ $do_tcp -eq 0 ]; then + # don't bother testing fallback tcp except for loopback case. + if [ ${listener_ns} != ${connector_ns} ]; then + return 0 + fi fi do_transfer ${listener_ns} ${connector_ns} MPTCP TCP ${connect_addr} ${local_addr}
@@ -516,6 +533,15 @@ run_tests_lo() return 1 fi + if [ $do_tcp -gt 1 ] ;then + do_transfer ${listener_ns} ${connector_ns} TCP TCP ${connect_addr} ${local_addr} + lret=$? + if [ $lret -ne 0 ]; then + ret=$lret + return 1 + fi + fi + return 0 }
Signature · 1 line
2.26.2
Matthieu Baerts
[MPTCP] Re: [PATCH mptcp-next 1/2] selftests: mptcp: add option to specify size of file to transfer
Hi Florian,
Quoted 6 lines · Florian Westphal
The script generates two random files that are then sent via tcp and mptcp connections.
In order to compare throughput over consecutive runs add an option to provide the file size on the command line: "-f 128000".
Quoted 7 lines
Also add an option, -t, to enable tcp tests. This is useful to compare throughput of mptcp connections and tcp connections.
Example: run tests with a 4mb file size, 300ms delay 0.01% loss, default gso/tso/gro settings and with large write/blocking io:
mptcp_connect.sh-t -f $((4 * 1024 * 1024)) -d 300 -l 0.01% -r 0 -e "" -m mmap
Thank you for the patch! Good idea!
Quoted 4 lines
tools/testing/selftests/net/mptcp/mptcp_connect.sh +0
-0
(...)
tools/testing/selftests/net/mptcp/mptcp_connect.sh usage() { · line 45
Quoted 5 lines
tools/testing/selftests/net/mptcp/mptcp_connect.sh +1
-0@@ -40,9 +42,11 @@ usage() { echo -e "\t-e: ethtool features to disable, e.g.: \"-e tso -e gso\" (default: randomly disable any of tso/gso/gro)" echo -e "\t-4: IPv4 only: disable IPv6 tests (default: test both IPv4 and IPv6)" echo -e "\t-c: capture packets for each test using tcpdump (default: no capture)" + echo -e "\t-s: size of file to transfer in bytes (default random)"
Small detail: it should be "-f" instead of "-s". I can fix this when applying the patch in mptcp-next.
Cheers,
Matt
--
Tessares | Belgium | Hybrid Access Solutions
www.tessares.net
Florian Westphal
[MPTCP] Re: [PATCH mptcp-next 1/2] selftests: mptcp: add option to specify size of file to transfer
Quoted 8 lines · Matthieu Baerts
tools/testing/selftests/net/mptcp/mptcp_connect.sh usage() { · line 45
Quoted 5 lines
tools/testing/selftests/net/mptcp/mptcp_connect.sh +1
-0@@ -40,9 +42,11 @@ usage() { echo -e "\t-e: ethtool features to disable, e.g.: \"-e tso -e gso\" (default: randomly disable any of tso/gso/gro)" echo -e "\t-4: IPv4 only: disable IPv6 tests (default: test both IPv4 and IPv6)" echo -e "\t-c: capture packets for each test using tcpdump (default: no capture)" + echo -e "\t-s: size of file to transfer in bytes (default random)"Small detail: it should be "-f" instead of "-s". I can fix this when applying the patch in mptcp-next.
Thanks, please do that.
Florian Westphal
PATCH 2/2 mptcp: add receive buffer auto-tuning
In reply to Florian WestphalWhen mptcp is used, userspace doesn't read from the tcp (subflow) socket but from the parent (mptcp) socket receive queue.
skbs are moved from the subflow socket to the mptcp rx queue either from 'data_ready' callback (if mptcp socket can be locked), a work queue, or the socket receive function.
This means tcp_rcv_space_adjust() is never called and thus no receive buffer size auto-tuning is done.
An earlier (not merged) patch added tcp_rcv_space_adjust() calls to the function that moves skbs from subflow to mptcp socket.
While this enabled autotuning, it also meant tuning was done even if userspace was reading the mptcp socket very slowly.
This adds mptcp_rcv_space_adjust() and calls it after userspace has read data from the mptcp socket rx queue.
Its very similar to tcp_rcv_space_adjust, with two differences:
1. The rtt estimate is the largest one observed on a subflow
2. The rcvbuf size and window clamp of all subflows is adjusted to the mptcp-level rcvbuf.
Otherwise, we get spurious drops at tcp (subflow) socket level if the skbs are not moved to the mptcp socket fast enough and reduced throughput..
Before:
time mptcp_connect.sh -t -f $((4*1024*1024)) -d 300 -l 0.01% -r 0 -e "" -m mmap [..]
ns4 MPTCP -> ns3 (10.0.3.2:10108 ) MPTCP (duration 40562ms) [ OK ] ns4 MPTCP -> ns3 (10.0.3.2:10109 ) TCP (duration 5415ms) [ OK ] ns4 TCP -> ns3 (10.0.3.2:10110 ) MPTCP (duration 5413ms) [ OK ] ns4 MPTCP -> ns3 (dead:beef:3::2:10111) MPTCP (duration 41331ms) [ OK ] ns4 MPTCP -> ns3 (dead:beef:3::2:10112) TCP (duration 5415ms) [ OK ] ns4 TCP -> ns3 (dead:beef:3::2:10113) MPTCP (duration 5714ms) [ OK ]
Time: 846 seconds
After:
ns4 MPTCP -> ns3 (10.0.3.2:10108 ) MPTCP (duration 5417ms) [ OK ] ns4 MPTCP -> ns3 (10.0.3.2:10109 ) TCP (duration 5429ms) [ OK ] ns4 TCP -> ns3 (10.0.3.2:10110 ) MPTCP (duration 5418ms) [ OK ] ns4 MPTCP -> ns3 (dead:beef:3::2:10111) MPTCP (duration 5423ms) [ OK ] ns4 MPTCP -> ns3 (dead:beef:3::2:10112) TCP (duration 5715ms) [ OK ] ns4 TCP -> ns3 (dead:beef:3::2:10113) MPTCP (duration 5415ms) [ OK ]
Time: 275 seconds
---
changes since last version:
- adjust for 'mptcp fallback' patches
- add 'const' qualifier to function argument
net/mptcp/protocol.c | 123 ++++++++++++++++++++++++++++++++++++++++--- net/mptcp/protocol.h | 7 +++ net/mptcp/subflow.c | 4 +- 3 files changed, 126 insertions(+), 8 deletions(-)
net/mptcp/protocol.c +116 -7
@@ -202,13 +202,6 @@ static bool __mptcp_move_skbs_from_subflow(struct mptcp_sock *msk,return false; } -if (!(sk->sk_userlocks &SOCK_RCVBUF_LOCK )) { -int rcvbuf =max (ssk->sk_rcvbuf, sk->sk_rcvbuf); - -if (rcvbuf > sk->sk_rcvbuf) - sk->sk_rcvbuf = rcvbuf; - } - tp =tcp_sk (ssk); do { u32 map_remaining, offset;
@@ -939,6 +932,100 @@ static int __mptcp_recvmsg_mskq(struct mptcp_sock *msk,return copied; } +/* receive buffer autotuning. See tcp_rcv_space_adjust for more information. +* +* Only difference: Use highest rtt estimate of the subflows in use. +*/ +static void mptcp_rcv_space_adjust (struct mptcp_sock *msk,int copied) +{ +struct mptcp_subflow_context *subflow; +struct sock *sk = (struct sock *)msk; +u32 time, advmss =1 ; +u64 rtt_us, mstamp; + +sock_owned_by_me (sk); + +if (copied <=0 ) +return ; + + msk->rcvq_space.copied += copied; + + mstamp =div_u64 (tcp_clock_ns (),NSEC_PER_USEC ); + time =tcp_stamp_us_delta (mstamp, msk->rcvq_space.time); + + rtt_us = msk->rcvq_space.rtt_us; +if (rtt_us && time < (rtt_us >>3 )) +return ; + + rtt_us =0 ; +mptcp_for_each_subflow (msk, subflow) { +const struct tcp_sock *tp; +u64 sf_rtt_us; +u32 sf_advmss; + + tp =tcp_sk (mptcp_subflow_tcp_sock (subflow)); + + sf_rtt_us =READ_ONCE (tp->rcv_rtt_est.rtt_us); + sf_advmss =READ_ONCE (tp->advmss); + + rtt_us =max (sf_rtt_us, rtt_us); + advmss =max (sf_advmss, advmss); + } + + msk->rcvq_space.rtt_us = rtt_us; +if (time < (rtt_us >>3 ) || rtt_us ==0 ) +return ; + +if (msk->rcvq_space.copied <= msk->rcvq_space.space) + goto new_measure; + +if (sock_net (sk)->ipv4.sysctl_tcp_moderate_rcvbuf && + !(sk->sk_userlocks &SOCK_RCVBUF_LOCK )) { +int rcvmem, rcvbuf; +u64 rcvwin, grow; + + rcvwin = ((u64 )msk->rcvq_space.copied <<1 ) +16 * advmss; + + grow = rcvwin *(msk->rcvq_space.copied - msk->rcvq_space.space); + +do_div (grow, msk->rcvq_space.space); + rcvwin += (grow <<1 ); + + rcvmem =SKB_TRUESIZE (advmss +MAX_TCP_HEADER ); +while (tcp_win_from_space (sk, rcvmem) < advmss) + rcvmem +=128 ; + +do_div (rcvwin, advmss); + rcvbuf =min_t (u64, rcvwin * rcvmem, +sock_net (sk)->ipv4.sysctl_tcp_rmem[2 ]); + +if (rcvbuf > sk->sk_rcvbuf) { +u32 window_clamp; + + window_clamp =tcp_win_from_space (sk, rcvbuf); +WRITE_ONCE (sk->sk_rcvbuf, rcvbuf); + +/* Make subflows follow along. If we do not do this, we +* get drops at subflow level if skbs can't be moved to +* the mptcp rx queue fast enough (announced rcv_win can +* exceed ssk->sk_rcvbuf). +*/ +mptcp_for_each_subflow (msk, subflow) { +struct sock *ssk; + + ssk =mptcp_subflow_tcp_sock (subflow); +WRITE_ONCE (ssk->sk_rcvbuf, rcvbuf); +tcp_sk (ssk)->window_clamp = window_clamp; + } + } + } + + msk->rcvq_space.space = msk->rcvq_space.copied; +new_measure: + msk->rcvq_space.copied =0 ; + msk->rcvq_space.time = mstamp; +} +static bool __mptcp_move_skbs (struct mptcp_sock *msk) {unsigned int moved =0 ;
@@ -1051,6 +1138,8 @@ static int mptcp_recvmsg(struct sock *sk, struct msghdr *msg, size_t len,set_bit (MPTCP_DATA_READY , &msk->flags); } out_err: +mptcp_rcv_space_adjust (msk, copied); +release_sock (sk);return copied; }
@@ -1263,6 +1352,7 @@ static int mptcp_init_sock(struct sock *sk)return ret;sk_sockets_allocated_inc (sk); + sk->sk_rcvbuf =sock_net (sk)->ipv4.sysctl_tcp_rmem[1 ]; sk->sk_sndbuf =sock_net (sk)->ipv4.sysctl_tcp_wmem[2 ];return 0 ;
@@ -1449,6 +1539,22 @@ struct sock *mptcp_sk_clone(const struct sock *sk,return nsk; } +void mptcp_rcv_space_init (struct mptcp_sock *msk,const struct sock *ssk) +{ +const struct tcp_sock *tp =tcp_sk (ssk); + + msk->rcvq_space.copied =0 ; + msk->rcvq_space.rtt_us =0 ; + + msk->rcvq_space.time = tp->tcp_mstamp; + +/* initial rcv_space offering made to peer */ + msk->rcvq_space.space =min_t (u32, tp->rcv_wnd, +TCP_INIT_CWND * tp->advmss); +if (msk->rcvq_space.space ==0 ) + msk->rcvq_space.space =TCP_INIT_CWND *TCP_MSS_DEFAULT ; +} +static struct sock *mptcp_accept (struct sock *sk,int flags,int *err,bool kern) {
@@ -1498,6 +1604,7 @@ static struct sock *mptcp_accept(struct sock *sk, int flags, int *err,list_add (&subflow->node, &msk->conn_list);inet_sk_state_store (newsk,TCP_ESTABLISHED ); +mptcp_rcv_space_init (msk, ssk);bh_unlock_sock (new_mptcp_sock);__MPTCP_INC_STATS (sock_net (sk),MPTCP_MIB_MPCAPABLEPASSIVEACK );
@@ -1660,6 +1767,8 @@ void mptcp_finish_connect(struct sock *ssk)atomic64_set (&msk->snd_una, msk->write_seq);mptcp_pm_new_connection (msk,0 ); + +mptcp_rcv_space_init (msk, ssk); }static void mptcp_sock_graft (struct sock *sk,struct socket *parent)
net/mptcp/protocol.h +7 -0
@@ -211,6 +211,12 @@ struct mptcp_sock {struct socket *subflow;/* outgoing connect/listener/!mp_capable */ struct sock *first;struct mptcp_pm_data pm; +struct { +u32 space;/* bytes copied in last measurement window */ +u32 copied;/* bytes copied in this measurement window */ +u64 time;/* start time of measurement window */ +u64 rtt_us;/* last maximum rtt of subflows */ + } rcvq_space; };#define mptcp_for_each_subflow (__msk, __subflow) \
@@ -378,6 +384,7 @@ void mptcp_get_options(const struct sk_buff *skb, struct mptcp_options_received *mp_opt);void mptcp_finish_connect (struct sock *sk); +void mptcp_rcv_space_init (struct mptcp_sock *msk,const struct sock *ssk);void mptcp_data_ready (struct sock *sk,struct sock *ssk);bool mptcp_finish_join (struct sock *sk);void mptcp_data_acked (struct sock *sk);
net/mptcp/subflow.c +3 -1
@@ -259,8 +259,10 @@ static void subflow_finish_connect(struct sock *sk, const struct sk_buff *skb) MPTCP_MIB_MPCAPABLEACTIVEFALLBACK); } -if (mptcp_check_fallback (sk)) +if (mptcp_check_fallback (sk)) { +mptcp_rcv_space_init (mptcp_sk (parent), sk);return ; + }if (subflow->mp_capable) { pr_debug("subflow=%p, remote_key=%llu", mptcp_subflow_ctx(sk),
Signature · 1 line
2.26.2
Christoph Paasch
[MPTCP] Re: [PATCH mptcp-next 2/2] mptcp: add receive buffer auto-tuning
Hello,
Quoted 46 lines · Florian Westphal
When mptcp is used, userspace doesn't read from the tcp (subflow) socket but from the parent (mptcp) socket receive queue.
skbs are moved from the subflow socket to the mptcp rx queue either from '
data_ready' callback (if mptcp socket can be locked), a work queue, or the socket receive function.This means
tcp_rcv_space_adjust()is never called and thus no receive buffer size auto-tuning is done.An earlier (not merged) patch added
tcp_rcv_space_adjust()calls to the function that moves skbs from subflow to mptcp socket.
While this enabled autotuning, it also meant tuning was done even if userspace was reading the mptcp socket very slowly.This adds
mptcp_rcv_space_adjust()and calls it after userspace has read data from the mptcp socket rx queue.Its very similar to
tcp_rcv_space_adjust, with two differences:1. The rtt estimate is the largest one observed on a subflow
2. The rcvbuf size and window clamp of all subflows is adjusted to the mptcp-level rcvbuf.Otherwise, we get spurious drops at tcp (subflow) socket level if the skbs are not moved to the mptcp socket fast enough and reduced throughput..
Before:
timemptcp_connect.sh-t -f $((4*1024*1024)) -d 300 -l 0.01% -r 0 -e "" -m mmap [..]ns4 MPTCP -> ns3 (10.0.3.2:10108 ) MPTCP (duration 40562ms) [ OK ] ns4 MPTCP -> ns3 (10.0.3.2:10109 ) TCP (duration 5415ms) [ OK ] ns4 TCP -> ns3 (10.0.3.2:10110 ) MPTCP (duration 5413ms) [ OK ] ns4 MPTCP -> ns3 (dead:beef:3::2:10111) MPTCP (duration 41331ms) [ OK ] ns4 MPTCP -> ns3 (dead:beef:3::2:10112) TCP (duration 5415ms) [ OK ] ns4 TCP -> ns3 (dead:beef:3::2:10113) MPTCP (duration 5714ms) [ OK ]Time: 846 seconds
After:
ns4 MPTCP -> ns3 (10.0.3.2:10108 ) MPTCP (duration 5417ms) [ OK ] ns4 MPTCP -> ns3 (10.0.3.2:10109 ) TCP (duration 5429ms) [ OK ] ns4 TCP -> ns3 (10.0.3.2:10110 ) MPTCP (duration 5418ms) [ OK ] ns4 MPTCP -> ns3 (dead:beef:3::2:10111) MPTCP (duration 5423ms) [ OK ] ns4 MPTCP -> ns3 (dead:beef:3::2:10112) TCP (duration 5715ms) [ OK ] ns4 TCP -> ns3 (dead:beef:3::2:10113) MPTCP (duration 5415ms) [ OK ]
Quoted 7 lines
Time: 275 seconds
Signed-off-by Florian Westphal <fw(a)strlen.de>---
changes since last version:
- adjust for 'mptcp fallback' patches
- add 'const' qualifier to function argument
syzkaller found another divide-by-zero. I don't yet have a real repro generated by syzkaller, but from the logs it looks like it is the simultaneous self-connect case.
In Davide's other patch I think we need to call mptcp_rcv_space_init in this hunk:
net/mptcp/subflow.c +10 -0
@@ -1114,6 +1114,16 @@ static void subflow_state_change(struct sock *sk)__subflow_state_change (sk); +if (subflow_simultaneous_connect (sk)) { +mptcp_do_fallback (sk); +pr_fallback (mptcp_sk (parent)); + subflow->conn_finished =1 ; +if (inet_sk_state_load (parent) ==TCP_SYN_SENT ) { +inet_sk_state_store (parent,TCP_ESTABLISHED ); + parent->sk_state_change (parent); + } + } +/* as recvmsg() does not acquire the subflow socket for ssk selection * a fin packet carrying a DSS can be unnoticed if we don't trigger * the data available machinery here.
Christoph
Matthieu Baerts
Hi Florian, Christoph,
Quoted 9 lines · Florian Westphal
First patch adds switch to test script so it can be run with a fixed file size rather than a random one.
Second patch adds receive buffer autotuning. I rebased this on top of mptcp-next as per Christophs patch suggestion to make sure the initialisation function has been run for the fallback case too.
I suggest to apply this to mptcp-next so this can be pushed to net-next once that opens up again.
Thank you for the patches and the reviews (of the previous version)!
I just added the two patches at the end of the "export" branch (before the two DO-NOT-MERGE commits).
- 5fb9128339d6..4f2bd7bdf714: result
As discussed in patch 1/2, I also added a tiny fix:
- ab8953b09d86: selftests: mptcp: fix typo in usage
- (of course, no new signed-off-by tags were added)
- 4f2bd7bdf714..76c6304b26cb: result
Just to be sure (and not to ask to work on this, it's not urgent): I guess these modifications don't improve the situation for:
https://github.com/multipath-tcp/mptcp_net-next/issues/6
The "export" branch has already been updated and tests are still OK.
Cheers,
Matt
Signature · 2 lines
Tessares | Belgium | Hybrid Access Solutions www.tessares.net