mptcp: fallback socket cleanup
Subsystems (2)
Paolo Abeni
cover mptcp: fallback socket cleanup
This is a quite early RFC to acquire as early feedback as possible. I did self-tests a few times on 3/5 and on patch 5/5 without failure, but likely some bugs are still present.
The first 3 patches in this series:
- clarify and cleanup the TCP fallback handling
- fixes setsockopt()/getsockopt() for connected/unbound sockets
The last 2 patches do more aggressive/invasive cleanup:
- drop the 'fallback' field so that we always use the conn_list
- drop the nested socket in mptcp_finish_connect()
- remove the lock/lookup/get/unlock dance in most case (due the the above)
After all the above, performance for connection fallen back to plain TCP should be almost the same of TCP connection on unpacked kernel.
Addittionally 'nc' and 'netperf' now run successfully when forced to MPTCP. The latter still report a non fatal error at connection closing time as it performs:
getsockopt(4, SOL_TCP, TCP_MAXSEG, //...);getsockopt(4, SOL_TCP, TCP_INFO, //...);
which is still unsupported after that the mp capable handshake is started.
For the records on the debug build I'm using netperf scores better result with MPTCP that with plain TCP ;) [agreed, completely irrelevant, still nice]
The series applies at selftests and is unsquashed - will require quite a bit of work there and more work to rebase/adjust the patches after kselftest. I'll start that after the pending patches are in.
_Any_ feedback _very_ welcome!
Paolo Abeni (5):
mptcp: clear 'is_tcp' socket flag when the MP_CAPABLE handshake fails
mptcp: pass subflow to mptcp_finish_connect()
mptcp: cleanup fallback handling
mptcp: remove 'fallback' field
mptcp: avoid acquiring the msk lock in mptcp_finisch_connect()
net/mptcp/options.c | 3 + net/mptcp/protocol.c | 342 +++++++++++++++++++++++-------------------- net/mptcp/protocol.h | 4 +- net/mptcp/subflow.c | 4 +- 4 files changed, 189 insertions(+), 164 deletions(-)
Signature · 1 line
2.21.0
Paolo Abeni
RFC PATCH 1/5 mptcp: clear 'is_tcp' socket flag when the MP_CAPABLE handshake fails
Currently we don't have a single flag for 'mp capable handshake failed', but we can clear the is_tcp flag to record such info.
This will make also processing for fallback socket almost as fast as native ones, and will help cleaning 'fallback' fields usage in later patches.
--- net/mptcp/options.c | 3 +++ net/mptcp/subflow.c | 2 ++ 2 files changed, 5 insertions(+)
net/mptcp/options.c +3 -0
@@ -301,6 +301,8 @@ void mptcp_rcv_synsent(struct sock *sk) subflow->mp_capable =1 ; subflow->can_ack =1 ; subflow->remote_key = tp->rx_opt.mptcp.sndr_key; + }else { +tcp_sk (sk)->is_mptcp =0 ; } }
@@ -505,6 +507,7 @@ static bool check_fourth_ack(struct mptcp_subflow_context *subflow,*/ if (!mp_opt->mp_capable) { subflow->mp_capable =0 ; +tcp_sk (mptcp_subflow_tcp_socket (subflow)->sk)->is_mptcp =0 ;return false; } subflow->remote_key = mp_opt->sndr_key;
net/mptcp/subflow.c +2 -0
@@ -699,6 +699,8 @@ static void subflow_ulp_clone(const struct request_sock *req, new_ctx->token = subflow_req->token; new_ctx->ssn_offset = subflow_req->ssn_offset; new_ctx->idsn = subflow_req->idsn; + }else { +tcp_sk (newsk)->is_mptcp =0 ; } }
Signature · 1 line
2.21.0
Paolo Abeni
RFC PATCH 2/5 mptcp: pass subflow to mptcp_finish_connect()
In reply to Paolo AbeniSo that we avoid going throw 'msk->fallback' to fetch such pointer.
--- net/mptcp/protocol.c | 8 +++----- net/mptcp/protocol.h | 3 ++- net/mptcp/subflow.c | 2 +- 3 files changed, 6 insertions(+), 7 deletions(-)
net/mptcp/protocol.c +3 -5
@@ -632,14 +632,12 @@ static int mptcp_get_port(struct sock *sk, unsigned short snum)return inet_csk_get_port (msk->subflow->sk, snum); } -void mptcp_finish_connect (struct sock *sk,int mp_capable) +void mptcp_finish_connect (struct sock *sk, +struct mptcp_subflow_context *subflow) { -struct mptcp_subflow_context *subflow;struct mptcp_sock *msk =mptcp_sk (sk); - subflow =mptcp_subflow_ctx (msk->subflow->sk); - -if (mp_capable) { +if (subflow->mp_capable) {u64 ack_seq;/* sk (new subflow socket) is already locked, but we need
net/mptcp/protocol.h +2 -1
@@ -185,7 +185,8 @@ int mptcp_read_actor(read_descriptor_t *desc, struct sk_buff *skb,void mptcp_get_options (const struct sk_buff *skb,struct tcp_options_received *opt_rx); -void mptcp_finish_connect (struct sock *sk,int mp_capable); +void mptcp_finish_connect (struct sock *sk, +struct mptcp_subflow_context *subflow);int mptcp_token_new_request (struct request_sock *req);void mptcp_token_destroy_request (u32 token);
net/mptcp/subflow.c +1 -1
@@ -119,7 +119,7 @@ static void subflow_finish_connect(struct sock *sk, const struct sk_buff *skb)if (subflow->conn && !subflow->conn_finished) {pr_debug ("subflow=%p, remote_key=%llu" ,mptcp_subflow_ctx (sk), subflow->remote_key); -mptcp_finish_connect (subflow->conn, subflow->mp_capable); +mptcp_finish_connect (subflow->conn, subflow); subflow->conn_finished =1 ;if (skb) {
Signature · 1 line
2.21.0
Paolo Abeni
RFC PATCH 3/5 mptcp: cleanup fallback handling
In reply to Paolo AbeniThe 'fallback' field has many different meanings:
tcp fallback socket, first subflow, listening socket. Enforcing the correct usage is not easy - currently we have several problems:
1) this will fail, but it should not:
fd = socket(AF_INET{,6}, SOCK_STREAM, MPTCP_PROTO); setsockopt(fd, SOL_IP{,V6}, //...) = -1 EOPNOTSUPP (Operation not supported)
2) this will report success, but it should not:
fd = socket(AF_INET{,6}, SOCK_STREAM, MPTCP_PROTO); connect(fd, //)bind(fd, //) = 0
3) this will report success, but it should not:
fd = socket(AF_INET{,6}, SOCK_STREAM, MPTCP_PROTO); listen(fd, // )
s = accept(fd, )bind(s, ...) = 0
This patch:
- explicitly handle the msk socket status in some missing places
- wrap msk->subflow access with helpers describing the expected semantic
- use the above to enforce correct syscall behavior
- create on demand the subflow socket for setsockopt(), if possible
--- net/mptcp/protocol.c | 192 +++++++++++++++++++++++++++---------------- 1 file changed, 123 insertions(+), 69 deletions(-)
net/mptcp/protocol.c +123 -69
@@ -22,26 +22,48 @@#include <net/mptcp.h> #include "protocol.h" -static struct socket *__mptcp_fallback_get_ref (const struct mptcp_sock *msk) +/* if msk has a single subflow socket, and the mp_capable handshake is not +* completed yet or has failed - that is, the socket is Not MP Capable, +* returns it. +* Otherwise returns NULL +*/ +static struct socket *__mptcp_nmpc_socket (const struct mptcp_sock *msk) {sock_owned_by_me ((const struct sock *)msk);if (!msk->subflow)return NULL ; -sock_hold (msk->subflow->sk);return msk->subflow; } -static struct socket *mptcp_fallback_get_ref (const struct mptcp_sock *msk) +/* get a reference to the first subflow socket and returns it */ +static struct socket *__mptcp_first_socket (const struct mptcp_sock *msk) { -struct socket *ssock; +struct socket *ssock =__mptcp_nmpc_socket (msk); +struct mptcp_subflow_context *subflow; -lock_sock ((struct sock *)msk); - ssock =__mptcp_fallback_get_ref (msk); -release_sock ((struct sock *)msk); +if (ssock) +return ssock; +if (list_empty (&msk->conn_list)) +return NULL ; -return ssock; + subflow =list_first_entry (&msk->conn_list, + struct mptcp_subflow_context, node); +return mptcp_subflow_tcp_socket (subflow); +} + +/* if msk has a single subflow, and the mp_capable handshake is failed, +* get a reference and return it. +* Otherwise returns NULL +*/ +static struct socket *__mptcp_tcp_fallback_get (const struct mptcp_sock *msk) +{ +if (!msk->subflow ||tcp_sk (msk->subflow->sk)->is_mptcp) +return NULL ; + +sock_hold (msk->subflow->sk); +return msk->subflow; }static struct sock *mptcp_subflow_get (const struct mptcp_sock *msk)
@@ -57,6 +79,51 @@ static struct sock *mptcp_subflow_get(const struct mptcp_sock *msk)return NULL ; } +static bool __mptcp_can_create_subflow (const struct mptcp_sock *msk) +{ +int state = ((struct sock *)msk)->sk_state; + +return state ==TCP_CLOSE || state ==TCP_LISTEN ; +} + +static struct socket *mptcp_socket_create_get (struct mptcp_sock *msk,int state) +{ +struct mptcp_subflow_context *subflow; +struct sock *sk = (struct sock *)msk; +struct socket *ssock; +int err; + +lock_sock (sk); + ssock =__mptcp_nmpc_socket (msk); +if (ssock) + goto set_state; + +if (!__mptcp_can_create_subflow (msk)) { + ssock =ERR_PTR (-EINVAL ); + goto release; + } + + err =mptcp_subflow_create_socket (sk, &ssock); +if (err) { + ssock =ERR_PTR (err); + goto release; + } + + msk->subflow = ssock; + subflow =mptcp_subflow_ctx (msk->subflow->sk); + subflow->request_mptcp =1 ;/* @@ if MPTCP enabled */ + subflow->request_version =1 ;/* only v1 supported */ + +set_state: +sock_hold (ssock->sk); +if (state !=TCP_MAX_STATES ) +inet_sk_state_store (sk, state); + +release: +release_sock (sk); +return ssock; +} +static bool mptcp_ext_cache_refill (struct mptcp_sock *msk) {if (!msk->cached_ext)
@@ -213,7 +280,7 @@ static int mptcp_sendmsg(struct sock *sk, struct msghdr *msg, size_t len)return -EOPNOTSUPP ;lock_sock (sk); - ssock =__mptcp_fallback_get_ref (msk); + ssock =__mptcp_tcp_fallback_get (msk);if (ssock) {release_sock (sk);pr_debug ("fallback passthrough" );
@@ -320,7 +387,7 @@ static int mptcp_recvmsg(struct sock *sk, struct msghdr *msg, size_t len,long timeo;lock_sock (sk); - ssock =__mptcp_fallback_get_ref (msk); + ssock =__mptcp_tcp_fallback_get (msk);if (ssock) {release_sock (sk); pr_debug("fallback-read subflow=%p",
@@ -493,7 +560,11 @@ static struct sock *mptcp_accept(struct sock *sk, int flags, int *err,struct socket *listener;struct sock *newsk; - listener = msk->subflow; + listener =__mptcp_first_socket (msk); +if (WARN_ON_ONCE (!listener)) { + *err = -EINVAL ; +return NULL ; + }pr_debug ("msk=%p, listener=%p" , msk,mptcp_subflow_ctx (listener->sk)); *err =kernel_accept (listener, &new_sock, flags);
@@ -580,8 +651,8 @@ static int mptcp_setsockopt(struct sock *sk, int level, int optname, optval = (char __kernel __force *)uoptval;pr_debug ("msk=%p" , msk); - ssock =mptcp_fallback_get_ref (msk); -if (ssock) { + ssock =mptcp_socket_create_get (msk,TCP_MAX_STATES ); +if (!IS_ERR (ssock)) {pr_debug ("subflow=%p" , ssock->sk); ret =kernel_setsockopt (ssock, level, optname, optval, optlen);sock_put (ssock->sk);
@@ -608,8 +679,8 @@ static int mptcp_getsockopt(struct sock *sk, int level, int optname, option = (int __kernel __force *)uoption;pr_debug ("msk=%p" , msk); - ssock =mptcp_fallback_get_ref (msk); -if (ssock) { + ssock =mptcp_socket_create_get (msk,TCP_MAX_STATES ); +if (!IS_ERR (ssock)) {pr_debug ("subflow=%p" , ssock->sk); ret =kernel_getsockopt (ssock, level, optname, optval, option);sock_put (ssock->sk);
@@ -625,11 +696,14 @@ static int mptcp_getsockopt(struct sock *sk, int level, int optname,static int mptcp_get_port (struct sock *sk,unsigned short snum) {struct mptcp_sock *msk =mptcp_sk (sk); +struct socket *ssock; -pr_debug ("msk=%p, subflow=%p" , msk, -mptcp_subflow_ctx (msk->subflow->sk)); + ssock =__mptcp_first_socket (msk); +pr_debug ("msk=%p, subflow=%p" , msk, ssock); +if (WARN_ON_ONCE (!ssock)) +return -EINVAL ; -return inet_csk_get_port (msk->subflow->sk, snum); +return inet_csk_get_port (ssock->sk, snum); } void mptcp_finish_connect(struct sock *sk,
@@ -704,43 +778,13 @@ static struct proto mptcp_prot = { .no_autobind = true, }; -static struct socket *mptcp_socket_create_get (struct mptcp_sock *msk) -{ -struct mptcp_subflow_context *subflow; -struct sock *sk = (struct sock *)msk; -struct socket *ssock; -int err; - -lock_sock (sk); - ssock =__mptcp_fallback_get_ref (msk); -if (ssock) - goto release; - - err =mptcp_subflow_create_socket (sk, &ssock); -if (err) { - ssock =ERR_PTR (err); - goto release; - } - - msk->subflow = ssock; - subflow =mptcp_subflow_ctx (msk->subflow->sk); - subflow->request_mptcp =1 ;/* @@ if MPTCP enabled */ - subflow->request_version =1 ;/* only v1 supported */ - -sock_hold (ssock->sk); - -release: -release_sock (sk); -return ssock; -} -static int mptcp_bind (struct socket *sock,struct sockaddr *uaddr,int addr_len) {struct mptcp_sock *msk =mptcp_sk (sock->sk);struct socket *ssock;int err; - ssock =mptcp_socket_create_get (msk); + ssock =mptcp_socket_create_get (msk,TCP_MAX_STATES );if (IS_ERR (ssock))return PTR_ERR (ssock);
@@ -756,7 +800,7 @@ static int mptcp_stream_connect(struct socket *sock, struct sockaddr *uaddr,struct socket *ssock;int err; - ssock =mptcp_socket_create_get (msk); + ssock =mptcp_socket_create_get (msk,TCP_SYN_SENT );if (IS_ERR (ssock))return PTR_ERR (ssock);
@@ -769,6 +813,10 @@ static int mptcp_stream_connect(struct socket *sock, struct sockaddr *uaddr,#endif err = ssock->ops->connect (ssock, uaddr, addr_len, flags); + +/* should we keep msk socket lock instead? and release it this +* latter state update */ +inet_sk_state_store (sock->sk,inet_sk_state_load (ssock->sk));sock_put (ssock->sk);return err; }
@@ -778,30 +826,20 @@ static int mptcp_getname(struct socket *sock, struct sockaddr *uaddr, {struct mptcp_sock *msk =mptcp_sk (sock->sk);struct socket *ssock; -struct sock *ssk;int ret;lock_sock (sock->sk); - ssock =__mptcp_fallback_get_ref (msk); -if (ssock) { -release_sock (sock->sk); -pr_debug ("subflow=%p" , ssock->sk); - ret = ssock->ops->getname (ssock, uaddr, peer); -sock_put (ssock->sk); -return ret; - } -/* @@ the meaning of getname() for the remote peer when the socket * is connected and there are multiple subflows is not defined. * For now just use the first subflow on the list. */ - ssk =mptcp_subflow_get (msk); -if (!ssk) { + ssock =__mptcp_first_socket (msk); +if (!ssock) {release_sock (sock->sk);return -ENOTCONN ; } - ret = ssk->sk_socket->ops->getname (ssk->sk_socket, uaddr, peer); + ret = ssock->ops->getname (ssock, uaddr, peer);release_sock (sock->sk);return ret; }
@@ -860,11 +898,15 @@ static int mptcp_listen(struct socket *sock, int backlog)pr_debug ("msk=%p" , msk); - ssock =mptcp_socket_create_get (msk); + ssock =mptcp_socket_create_get (msk,TCP_LISTEN );if (IS_ERR (ssock))return PTR_ERR (ssock); err = ssock->ops->listen (ssock, backlog); + +/* should we keep msk socket lock instead? and release it this +* latter state update */ +inet_sk_state_store (sock->sk,inet_sk_state_load (ssock->sk));sock_put (ssock->sk);return err; }
@@ -874,17 +916,28 @@ static int mptcp_stream_accept(struct socket *sock, struct socket *newsock, {struct mptcp_sock *msk =mptcp_sk (sock->sk);struct socket *ssock; -int err; +int err = -EINVAL ;pr_debug ("msk=%p" , msk); - ssock =mptcp_fallback_get_ref (msk); +lock_sock (sock->sk); +if (sock->sk->sk_state !=TCP_LISTEN ) + goto unlock_fail; + + ssock =__mptcp_first_socket (msk);if (!ssock) -return -EINVAL ; + goto unlock_fail; + +sock_hold (ssock->sk); +release_sock (sock->sk); err = ssock->ops->accept (sock, newsock, flags, kern);sock_put (ssock->sk);return err; + +unlock_fail: +release_sock (sock->sk); +return -EINVAL ; } static __poll_t mptcp_poll(struct file *file, struct socket *sock,
@@ -898,8 +951,9 @@ static __poll_t mptcp_poll(struct file *file, struct socket *sock, msk =mptcp_sk (sk);lock_sock (sk); - ssock =__mptcp_fallback_get_ref (msk); + ssock =__mptcp_nmpc_socket (msk);if (ssock) { +sock_hold (ssock->sk);release_sock (sk); ret = ssock->ops->poll (file, ssock, wait);sock_put (ssock->sk);
@@ -935,7 +989,7 @@ static int mptcp_shutdown(struct socket *sock, int how)if (how ==SHUT_WR || how ==SHUT_RDWR )inet_sk_state_store (sock->sk,TCP_FIN_WAIT1 ); - ssock =__mptcp_fallback_get_ref (msk); + ssock =__mptcp_tcp_fallback_get (msk);if (ssock) {release_sock (sock->sk);pr_debug ("subflow=%p" , ssock->sk);
Signature · 1 line
2.21.0
Paolo Abeni
RFC PATCH 4/5 mptcp: remove 'fallback' field
In reply to Paolo AbeniAfter the previous commit we can drop the fallback field, moving the subflow into conn_list at creation time. Just update the access helper to use conn_list only.
--- net/mptcp/protocol.c | 35 +++++++++++++++++------------------ net/mptcp/protocol.h | 1 - 2 files changed, 17 insertions(+), 19 deletions(-)
net/mptcp/protocol.c +17 -18
@@ -29,22 +29,23 @@*/ static struct socket *__mptcp_nmpc_socket (const struct mptcp_sock *msk) { +struct mptcp_subflow_context *subflow; +sock_owned_by_me ((const struct sock *)msk); -if (!msk->subflow) +if (msk->can_ack ||list_empty (&msk->conn_list))return NULL ; -return msk->subflow; + subflow =list_first_entry (&msk->conn_list, + struct mptcp_subflow_context, node); +return mptcp_subflow_tcp_socket (subflow); }/* get a reference to the first subflow socket and returns it */ static struct socket *__mptcp_first_socket (const struct mptcp_sock *msk) { -struct socket *ssock =__mptcp_nmpc_socket (msk);struct mptcp_subflow_context *subflow; -if (ssock) -return ssock;if (list_empty (&msk->conn_list))return NULL ;
@@ -59,11 +60,17 @@ static struct socket *__mptcp_first_socket(const struct mptcp_sock *msk)*/ static struct socket *__mptcp_tcp_fallback_get (const struct mptcp_sock *msk) { -if (!msk->subflow ||tcp_sk (msk->subflow->sk)->is_mptcp) +struct socket *ssock; + +if (msk->can_ack)return NULL ; -sock_hold (msk->subflow->sk); -return msk->subflow; + ssock =__mptcp_first_socket (msk); +if (tcp_sk (ssock->sk)->is_mptcp) +return NULL ; + +sock_hold (ssock->sk); +return ssock; }static struct sock *mptcp_subflow_get (const struct mptcp_sock *msk)
@@ -109,8 +116,8 @@ static struct socket *mptcp_socket_create_get(struct mptcp_sock *msk, int state) goto release; } - msk->subflow = ssock; - subflow =mptcp_subflow_ctx (msk->subflow->sk); + subflow =mptcp_subflow_ctx (ssock->sk); +list_add (&subflow->node, &msk->conn_list); subflow->request_mptcp =1 ;/* @@ if MPTCP enabled */ subflow->request_version =1 ;/* only v1 supported */
@@ -535,11 +542,6 @@ static void mptcp_close(struct sock *sk, long timeout)lock_sock (sk); -if (msk->subflow) { -sock_release (msk->subflow); - msk->subflow =NULL ; - } -list_for_each_entry_safe (subflow, tmp, &msk->conn_list, node) {pr_debug ("conn_list->subflow=%p" , subflow);sock_release (mptcp_subflow_tcp_socket (subflow));
@@ -598,7 +600,6 @@ static struct sock *mptcp_accept(struct sock *sk, int flags, int *err, msk->token = subflow->token;mptcp_token_update_accept (new_sock->sk, new_mptcp_sock); - msk->subflow =NULL ; msk->write_seq = subflow->idsn +1 ;if (subflow->can_ack) {
@@ -751,8 +752,6 @@ void mptcp_finish_connect(struct sock *sk, subflow->map_seq = ack_seq; subflow->map_subflow_seq =1 ; subflow->rel_write_seq =1 ; -list_add (&subflow->node, &msk->conn_list); - msk->subflow =NULL ;bh_unlock_sock (sk);local_bh_enable (); }
net/mptcp/protocol.h +0 -1
@@ -69,7 +69,6 @@ struct mptcp_sock {unsigned long flags;bool can_ack;struct list_head conn_list; -struct socket *subflow;/* outgoing connect/listener/!mp_capable */ };#define mptcp_for_each_subflow (__msk, __subflow) \
Signature · 1 line
2.21.0
Paolo Abeni
RFC PATCH 5/5 mptcp: avoid acquiring the msk lock in mptcp_finisch_connect()
In reply to Paolo AbeniNo longer required, just document the unlocked access with WRITE_ONCE() Since we now always acquire msk -> ssk lock in order, we can also avoid the msk lock/subflow lookup()/get()/ msk unlock()/ subflow op/ subflow put dance in most ops - simply keep the msk lock held the whole time.
--- net/mptcp/protocol.c | 181 ++++++++++++++++++------------------------- 1 file changed, 75 insertions(+), 106 deletions(-)
net/mptcp/protocol.c +75 -106
@@ -55,10 +55,10 @@ static struct socket *__mptcp_first_socket(const struct mptcp_sock *msk) }/* if msk has a single subflow, and the mp_capable handshake is failed, -* get a reference and return it. +* return it. * Otherwise returns NULL */ -static struct socket *__mptcp_tcp_fallback_get (const struct mptcp_sock *msk) +static struct socket *__mptcp_tcp_fallback (const struct mptcp_sock *msk) {struct socket *ssock;
@@ -69,7 +69,6 @@ static struct socket *__mptcp_tcp_fallback_get(const struct mptcp_sock *msk)if (tcp_sk (ssock->sk)->is_mptcp)return NULL ; -sock_hold (ssock->sk);return ssock; }
@@ -93,28 +92,23 @@ static bool __mptcp_can_create_subflow(const struct mptcp_sock *msk)return state ==TCP_CLOSE || state ==TCP_LISTEN ; } -static struct socket *mptcp_socket_create_get (struct mptcp_sock *msk,int state) +static struct socket *__mptcp_socket_create (struct mptcp_sock *msk,int state) {struct mptcp_subflow_context *subflow;struct sock *sk = (struct sock *)msk;struct socket *ssock;int err; -lock_sock (sk); ssock =__mptcp_nmpc_socket (msk);if (ssock) goto set_state; -if (!__mptcp_can_create_subflow (msk)) { - ssock =ERR_PTR (-EINVAL ); - goto release; - } +if (!__mptcp_can_create_subflow (msk)) +return ERR_PTR (-EINVAL ); err =mptcp_subflow_create_socket (sk, &ssock); -if (err) { - ssock =ERR_PTR (err); - goto release; - } +if (err) +return ERR_PTR (err); subflow =mptcp_subflow_ctx (ssock->sk);list_add (&subflow->node, &msk->conn_list);
@@ -122,12 +116,8 @@ static struct socket *mptcp_socket_create_get(struct mptcp_sock *msk, int state) subflow->request_version =1 ;/* only v1 supported */ set_state: -sock_hold (ssock->sk);if (state !=TCP_MAX_STATES )inet_sk_state_store (sk, state); - -release: -release_sock (sk);return ssock; }
@@ -287,12 +277,11 @@ static int mptcp_sendmsg(struct sock *sk, struct msghdr *msg, size_t len)return -EOPNOTSUPP ;lock_sock (sk); - ssock =__mptcp_tcp_fallback_get (msk); + ssock =__mptcp_tcp_fallback (msk);if (ssock) { -release_sock (sk);pr_debug ("fallback passthrough" ); ret =sock_sendmsg (ssock, msg); -sock_put (ssock->sk); +release_sock (sk);return ret; }
@@ -394,13 +383,12 @@ static int mptcp_recvmsg(struct sock *sk, struct msghdr *msg, size_t len,long timeo;lock_sock (sk); - ssock =__mptcp_tcp_fallback_get (msk); + ssock =__mptcp_tcp_fallback (msk);if (ssock) { -release_sock (sk);pr_debug ("fallback-read subflow=%p" ,mptcp_subflow_ctx (ssock->sk)); copied =sock_recvmsg (ssock, msg, flags); -sock_put (ssock->sk); +release_sock (sk);return copied; }
@@ -645,25 +633,26 @@ static int mptcp_setsockopt(struct sock *sk, int level, int optname, {struct mptcp_sock *msk =mptcp_sk (sk);char __kernel *optval; +int ret = -EOPNOTSUPP ;struct socket *ssock; -int ret;/* will be treated as __user in tcp_setsockopt */ optval = (char __kernel __force *)uoptval;pr_debug ("msk=%p" , msk); - ssock =mptcp_socket_create_get (msk,TCP_MAX_STATES ); + +/* @@ the meaning of setsockopt() when the socket is connected and +* there are multiple subflows is not defined. +*/ +lock_sock (sk); + ssock =__mptcp_socket_create (msk,TCP_MAX_STATES );if (!IS_ERR (ssock)) {pr_debug ("subflow=%p" , ssock->sk); ret =kernel_setsockopt (ssock, level, optname, optval, optlen); -sock_put (ssock->sk); -return ret; } +release_sock (sk); -/* @@ the meaning of setsockopt() when the socket is connected and -* there are multiple subflows is not defined. -*/ -return -EOPNOTSUPP ; +return ret; } static int mptcp_getsockopt(struct sock *sk, int level, int optname,
@@ -671,27 +660,28 @@ static int mptcp_getsockopt(struct sock *sk, int level, int optname, {struct mptcp_sock *msk =mptcp_sk (sk);char __kernel *optval; +int ret = -EOPNOTSUPP ;int __kernel *option;struct socket *ssock; -int ret;/* will be treated as __user in tcp_getsockopt */ optval = (char __kernel __force *)uoptval; option = (int __kernel __force *)uoption;pr_debug ("msk=%p" , msk); - ssock =mptcp_socket_create_get (msk,TCP_MAX_STATES ); +lock_sock (sk); + +/* @@ the meaning of getsockopt() when the socket is connected and +* there are multiple subflows is not defined. +*/ + ssock =__mptcp_socket_create (msk,TCP_MAX_STATES );if (!IS_ERR (ssock)) {pr_debug ("subflow=%p" , ssock->sk); ret =kernel_getsockopt (ssock, level, optname, optval, option); -sock_put (ssock->sk); -return ret; } +release_sock (sk); -/* @@ the meaning of getsockopt() when the socket is connected and -* there are multiple subflows is not defined. -*/ -return -EOPNOTSUPP ; +return ret; }static int mptcp_get_port (struct sock *sk,unsigned short snum)
@@ -711,51 +701,27 @@ void mptcp_finish_connect(struct sock *sk, struct mptcp_subflow_context *subflow) {struct mptcp_sock *msk =mptcp_sk (sk); +u64 ack_seq; -if (subflow->mp_capable) { -u64 ack_seq; +if (!subflow->mp_capable) +return ; -/* sk (new subflow socket) is already locked, but we need -* to lock the parent (mptcp) socket now to add the tcp socket -* to the subflow list. -* -* From lockdep point of view, this creates an ABBA type -* deadlock: Normally (sendmsg, recvmsg, ..), we lock the mptcp -* socket, then acquire a subflow lock. -* Here we do the reverse: "subflow lock, then mptcp lock". -* -* Its alright to do this here, because this subflow is not yet -* on the mptcp sockets subflow list. -* -* IOW, if another CPU has this mptcp socket locked, it cannot -* acquire this particular subflow, because subflow->sk isn't -* on msk->conn_list. -* -* This function can be called either from backlog processing -* (BH will be enabled) or from softirq, so we need to use BH -* locking scheme. -*/ -local_bh_disable (); -bh_lock_sock_nested (sk); +pr_debug ("msk=%p, token=%u" , msk, subflow->token); +mptcp_crypto_key_sha (subflow->remote_key,NULL , &ack_seq); + ack_seq++; + subflow->map_seq = ack_seq; + subflow->map_subflow_seq =1 ; + subflow->rel_write_seq =1 ; - msk->remote_key = subflow->remote_key; - msk->local_key = subflow->local_key; - msk->token = subflow->token; - -pr_debug ("msk=%p, token=%u" , msk, msk->token); - -mptcp_crypto_key_sha (msk->remote_key,NULL , &ack_seq); - msk->write_seq = subflow->idsn +1 ; - ack_seq++; - msk->ack_seq = ack_seq; - msk->can_ack =1 ; - subflow->map_seq = ack_seq; - subflow->map_subflow_seq =1 ; - subflow->rel_write_seq =1 ; -bh_unlock_sock (sk); -local_bh_enable (); - } -inet_sk_state_store (sk,TCP_ESTABLISHED ); +/* the socket is not connected yet, no msk/subflow ops can access/race +* accessing the field below +*/ +WRITE_ONCE (msk->remote_key, subflow->remote_key); +WRITE_ONCE (msk->local_key, subflow->local_key); +WRITE_ONCE (msk->token, subflow->token); +WRITE_ONCE (msk->write_seq, subflow->idsn +1 ); +WRITE_ONCE (msk->ack_seq, ack_seq); +WRITE_ONCE (msk->can_ack,1 ); }static struct proto mptcp_prot = {
@@ -783,12 +749,17 @@ static int mptcp_bind(struct socket *sock, struct sockaddr *uaddr, int addr_len)struct socket *ssock;int err; - ssock =mptcp_socket_create_get (msk,TCP_MAX_STATES ); -if (IS_ERR (ssock)) -return PTR_ERR (ssock); +lock_sock (sock->sk); + ssock =__mptcp_socket_create (msk,TCP_MAX_STATES ); +if (IS_ERR (ssock)) { + err =PTR_ERR (ssock); + goto unlock; + } err = ssock->ops->bind (ssock, uaddr, addr_len); -sock_put (ssock->sk); + +unlock: +release_sock (sock->sk);return err; }
@@ -799,9 +770,12 @@ static int mptcp_stream_connect(struct socket *sock, struct sockaddr *uaddr,struct socket *ssock;int err; - ssock =mptcp_socket_create_get (msk,TCP_SYN_SENT ); -if (IS_ERR (ssock)) -return PTR_ERR (ssock); +lock_sock (sock->sk); + ssock =__mptcp_socket_create (msk,TCP_SYN_SENT ); +if (IS_ERR (ssock)) { + err =PTR_ERR (ssock); + goto unlock; + }#ifdef CONFIG_TCP_MD5SIG /* no MPTCP if MD5SIG is enabled on this socket or we may run out of
@@ -816,7 +790,9 @@ static int mptcp_stream_connect(struct socket *sock, struct sockaddr *uaddr,/* should we keep msk socket lock instead? and release it this * latter state update */ inet_sk_state_store (sock->sk,inet_sk_state_load (ssock->sk)); -sock_put (ssock->sk); + +unlock: +release_sock (sock->sk);return err; }
@@ -897,16 +873,21 @@ static int mptcp_listen(struct socket *sock, int backlog)pr_debug ("msk=%p" , msk); - ssock =mptcp_socket_create_get (msk,TCP_LISTEN ); -if (IS_ERR (ssock)) -return PTR_ERR (ssock); +lock_sock (sock->sk); + ssock =__mptcp_socket_create (msk,TCP_LISTEN ); +if (IS_ERR (ssock)) { + err =PTR_ERR (ssock); + goto unlock; + } err = ssock->ops->listen (ssock, backlog);/* should we keep msk socket lock instead? and release it this * latter state update */ inet_sk_state_store (sock->sk,inet_sk_state_load (ssock->sk)); -sock_put (ssock->sk); + +unlock: +release_sock (sock->sk);return err; }
@@ -952,10 +933,8 @@ static __poll_t mptcp_poll(struct file *file, struct socket *sock,lock_sock (sk); ssock =__mptcp_nmpc_socket (msk);if (ssock) { -sock_hold (ssock->sk); -release_sock (sk); ret = ssock->ops->poll (file, ssock, wait); -sock_put (ssock->sk); +release_sock (sk);return ret; }
@@ -978,7 +957,6 @@ static int mptcp_shutdown(struct socket *sock, int how) {struct mptcp_sock *msk =mptcp_sk (sock->sk);struct mptcp_subflow_context *subflow; -struct socket *ssock;int ret =0 ;pr_debug ("sk=%p, how=%d" , msk, how);
@@ -988,15 +966,6 @@ static int mptcp_shutdown(struct socket *sock, int how)if (how ==SHUT_WR || how ==SHUT_RDWR )inet_sk_state_store (sock->sk,TCP_FIN_WAIT1 ); - ssock =__mptcp_tcp_fallback_get (msk); -if (ssock) { -release_sock (sock->sk); -pr_debug ("subflow=%p" , ssock->sk); - ret =kernel_sock_shutdown (ssock, how); -sock_put (ssock->sk); -return ret; - } -mptcp_for_each_subflow (msk, subflow) {struct socket *tcp_socket;
Signature · 1 line
2.21.0
Christoph Paasch
[MPTCP] Re: [RFC PATCH 5/5] mptcp: avoid acquiring the msk lock in mptcp_finisch_connect()
"[MPTCP] [RFC PATCH 5/5] mptcp: avoid acquiring the msk lock in mptcp_finisch_connect()"
I know there are two native German speakers here on this list, but let's not push it too far 😂
Christoph
Quoted 373 lines · Paolo Abeni
No longer required, just document the unlocked access with
WRITE_ONCE()Since we now always acquire msk -> ssk lock in order, we can also avoid the msk lock/subflow lookup()/get()/ msk unlock()/ subflow op/ subflow put dance in most ops - simply keep the msk lock held the whole time.Signed-off-by Paolo Abeni <pabeni(a)redhat.com>---
net/mptcp/protocol.c| 181 ++++++++++++++++++-------------------------1 file changed, 75 insertions(+), 106 deletions(-)net/mptcp/protocol.c +7
-13@@ -55,10 +55,10 @@ static struct socket *__mptcp_first_socket(const struct mptcp_sock *msk) }/* if msk has a single subflow, and the mp_capable handshake is failed, -* get a reference and return it. +* return it. * Otherwise returns NULL */ -static struct socket *__mptcp_tcp_fallback_get (const struct mptcp_sock *msk) +static struct socket *__mptcp_tcp_fallback (const struct mptcp_sock *msk) {struct socket *ssock;@@ -69,7 +69,6 @@ static struct socket *__mptcp_tcp_fallback_get(const struct mptcp_sock *msk)if (tcp_sk (ssock->sk)->is_mptcp)return NULL ; -sock_hold (ssock->sk);return ssock; }@@ -93,28 +92,23 @@ static bool __mptcp_can_create_subflow(const struct mptcp_sock *msk)return state ==TCP_CLOSE || state ==TCP_LISTEN ; } -static struct socket *mptcp_socket_create_get (struct mptcp_sock *msk,int state) +static struct socket *__mptcp_socket_create (struct mptcp_sock *msk,int state) {struct mptcp_subflow_context *subflow;struct sock *sk = (struct sock *)msk;struct socket *ssock;int err; -lock_sock (sk); ssock =__mptcp_nmpc_socket (msk);if (ssock) goto set_state; -if (!__mptcp_can_create_subflow (msk)) { - ssock =ERR_PTR (-EINVAL ); - goto release; - } +if (!__mptcp_can_create_subflow (msk)) +return ERR_PTR (-EINVAL ); err =mptcp_subflow_create_socket (sk, &ssock); -if (err) { - ssock =ERR_PTR (err); - goto release; - } +if (err) +return ERR_PTR (err); subflow =mptcp_subflow_ctx (ssock->sk);list_add (&subflow->node, &msk->conn_list);@@ -122,12 +116,8 @@ static struct socket *mptcp_socket_create_get(struct mptcp_sock *msk, int state) subflow->request_version =1 ;/* only v1 supported */
set_state:net/mptcp/protocol.c +0
-1-sock_hold (ssock->sk);if (state != TCP_MAX_STATES) inet_sk_state_store(sk, state);net/mptcp/protocol.c +0
-3- -release: -release_sock (sk);return ssock; }net/mptcp/protocol.c +68
-89@@ -287,12 +277,11 @@ static int mptcp_sendmsg(struct sock *sk, struct msghdr *msg, size_t len)return -EOPNOTSUPP ;lock_sock (sk); - ssock =__mptcp_tcp_fallback_get (msk); + ssock =__mptcp_tcp_fallback (msk);if (ssock) { -release_sock (sk);pr_debug ("fallback passthrough" ); ret =sock_sendmsg (ssock, msg); -sock_put (ssock->sk); +release_sock (sk);return ret; }@@ -394,13 +383,12 @@ static int mptcp_recvmsg(struct sock *sk, struct msghdr *msg, size_t len,long timeo;lock_sock (sk); - ssock =__mptcp_tcp_fallback_get (msk); + ssock =__mptcp_tcp_fallback (msk);if (ssock) { -release_sock (sk);pr_debug ("fallback-read subflow=%p" ,mptcp_subflow_ctx (ssock->sk)); copied =sock_recvmsg (ssock, msg, flags); -sock_put (ssock->sk); +release_sock (sk);return copied; }@@ -645,25 +633,26 @@ static int mptcp_setsockopt(struct sock *sk, int level, int optname, {struct mptcp_sock *msk =mptcp_sk (sk);char __kernel *optval; +int ret = -EOPNOTSUPP ;struct socket *ssock; -int ret;/* will be treated as __user in tcp_setsockopt */ optval = (char __kernel __force *)uoptval;pr_debug ("msk=%p" , msk); - ssock =mptcp_socket_create_get (msk,TCP_MAX_STATES ); + +/* @@ the meaning of setsockopt() when the socket is connected and +* there are multiple subflows is not defined. +*/ +lock_sock (sk); + ssock =__mptcp_socket_create (msk,TCP_MAX_STATES );if (!IS_ERR (ssock)) {pr_debug ("subflow=%p" , ssock->sk); ret =kernel_setsockopt (ssock, level, optname, optval, optlen); -sock_put (ssock->sk); -return ret; } +release_sock (sk); -/* @@ the meaning of setsockopt() when the socket is connected and -* there are multiple subflows is not defined. -*/ -return -EOPNOTSUPP ; +return ret; } static int mptcp_getsockopt(struct sock *sk, int level, int optname,@@ -671,27 +660,28 @@ static int mptcp_getsockopt(struct sock *sk, int level, int optname, {struct mptcp_sock *msk =mptcp_sk (sk);char __kernel *optval; +int ret = -EOPNOTSUPP ;int __kernel *option;struct socket *ssock; -int ret;/* will be treated as __user in tcp_getsockopt */ optval = (char __kernel __force *)uoptval; option = (int __kernel __force *)uoption;pr_debug ("msk=%p" , msk); - ssock =mptcp_socket_create_get (msk,TCP_MAX_STATES ); +lock_sock (sk); + +/* @@ the meaning of getsockopt() when the socket is connected and +* there are multiple subflows is not defined. +*/ + ssock =__mptcp_socket_create (msk,TCP_MAX_STATES );if (!IS_ERR (ssock)) {pr_debug ("subflow=%p" , ssock->sk); ret =kernel_getsockopt (ssock, level, optname, optval, option); -sock_put (ssock->sk); -return ret; } +release_sock (sk); -/* @@ the meaning of getsockopt() when the socket is connected and -* there are multiple subflows is not defined. -*/ -return -EOPNOTSUPP ; +return ret; }static int mptcp_get_port (struct sock *sk,unsigned short snum)@@ -711,51 +701,27 @@ void mptcp_finish_connect(struct sock *sk, struct mptcp_subflow_context *subflow) {struct mptcp_sock *msk =mptcp_sk (sk); +u64 ack_seq; -if (subflow->mp_capable) { -u64 ack_seq; +if (!subflow->mp_capable) +return ; -/* sk (new subflow socket) is already locked, but we need -* to lock the parent (mptcp) socket now to add the tcp socket -* to the subflow list. -* -* From lockdep point of view, this creates an ABBA type -* deadlock: Normally (sendmsg, recvmsg, ..), we lock the mptcp -* socket, then acquire a subflow lock. -* Here we do the reverse: "subflow lock, then mptcp lock". -* -* Its alright to do this here, because this subflow is not yet -* on the mptcp sockets subflow list. -* -* IOW, if another CPU has this mptcp socket locked, it cannot -* acquire this particular subflow, because subflow->sk isn't -* on msk->conn_list. -* -* This function can be called either from backlog processing -* (BH will be enabled) or from softirq, so we need to use BH -* locking scheme. -*/ -local_bh_disable (); -bh_lock_sock_nested (sk); +pr_debug ("msk=%p, token=%u" , msk, subflow->token); +mptcp_crypto_key_sha (subflow->remote_key,NULL , &ack_seq); + ack_seq++; + subflow->map_seq = ack_seq; + subflow->map_subflow_seq =1 ; + subflow->rel_write_seq =1 ; - msk->remote_key = subflow->remote_key; - msk->local_key = subflow->local_key; - msk->token = subflow->token; - -pr_debug ("msk=%p, token=%u" , msk, msk->token); - -mptcp_crypto_key_sha (msk->remote_key,NULL , &ack_seq); - msk->write_seq = subflow->idsn +1 ; - ack_seq++; - msk->ack_seq = ack_seq; - msk->can_ack =1 ; - subflow->map_seq = ack_seq; - subflow->map_subflow_seq =1 ; - subflow->rel_write_seq =1 ; -bh_unlock_sock (sk); -local_bh_enable (); - } -inet_sk_state_store (sk,TCP_ESTABLISHED ); +/* the socket is not connected yet, no msk/subflow ops can access/race +* accessing the field below +*/ +WRITE_ONCE (msk->remote_key, subflow->remote_key); +WRITE_ONCE (msk->local_key, subflow->local_key); +WRITE_ONCE (msk->token, subflow->token); +WRITE_ONCE (msk->write_seq, subflow->idsn +1 ); +WRITE_ONCE (msk->ack_seq, ack_seq); +WRITE_ONCE (msk->can_ack,1 ); }static struct proto mptcp_prot = {@@ -783,12 +749,17 @@ static int mptcp_bind(struct socket *sock, struct sockaddr *uaddr, int addr_len)struct socket *ssock;int err; - ssock =mptcp_socket_create_get (msk,TCP_MAX_STATES ); -if (IS_ERR (ssock)) -return PTR_ERR (ssock); +lock_sock (sock->sk); + ssock =__mptcp_socket_create (msk,TCP_MAX_STATES ); +if (IS_ERR (ssock)) { + err =PTR_ERR (ssock); + goto unlock; + } err = ssock->ops->bind (ssock, uaddr, addr_len); -sock_put (ssock->sk); + +unlock: +release_sock (sock->sk);return err; }@@ -799,9 +770,12 @@ static int mptcp_stream_connect(struct socket *sock, struct sockaddr *uaddr,struct socket *ssock;int err; - ssock =mptcp_socket_create_get (msk,TCP_SYN_SENT ); -if (IS_ERR (ssock)) -return PTR_ERR (ssock); +lock_sock (sock->sk); + ssock =__mptcp_socket_create (msk,TCP_SYN_SENT ); +if (IS_ERR (ssock)) { + err =PTR_ERR (ssock); + goto unlock; + }#ifdef CONFIG_TCP_MD5SIG /* no MPTCP if MD5SIG is enabled on this socket or we may run out of @@ -816,7 +790,9 @@ static int mptcp_stream_connect(struct socket *sock, struct sockaddr *uaddr,/* should we keep msk socket lock instead? and release it this * latter state update */ inet_sk_state_store (sock->sk,inet_sk_state_load (ssock->sk)); -sock_put (ssock->sk); + +unlock: +release_sock (sock->sk);return err; }@@ -897,16 +873,21 @@ static int mptcp_listen(struct socket *sock, int backlog)pr_debug ("msk=%p" , msk); - ssock =mptcp_socket_create_get (msk,TCP_LISTEN ); -if (IS_ERR (ssock)) -return PTR_ERR (ssock); +lock_sock (sock->sk); + ssock =__mptcp_socket_create (msk,TCP_LISTEN ); +if (IS_ERR (ssock)) { + err =PTR_ERR (ssock); + goto unlock; + } err = ssock->ops->listen (ssock, backlog);/* should we keep msk socket lock instead? and release it this * latter state update */ inet_sk_state_store (sock->sk,inet_sk_state_load (ssock->sk)); -sock_put (ssock->sk); + +unlock: +release_sock (sock->sk);return err; }@@ -952,10 +933,8 @@ static __poll_t mptcp_poll(struct file *file, struct socket *sock,lock_sock (sk); ssock =__mptcp_nmpc_socket (msk);if (ssock) { -sock_hold (ssock->sk); -release_sock (sk); ret = ssock->ops->poll (file, ssock, wait); -sock_put (ssock->sk); +release_sock (sk);return ret; }@@ -978,7 +957,6 @@ static int mptcp_shutdown(struct socket *sock, int how) {struct mptcp_sock *msk =mptcp_sk (sock->sk);struct mptcp_subflow_context *subflow; -struct socket *ssock;int ret =0 ;pr_debug ("sk=%p, how=%d" , msk, how);@@ -988,15 +966,6 @@ static int mptcp_shutdown(struct socket *sock, int how)if (how ==SHUT_WR || how ==SHUT_RDWR )inet_sk_state_store (sock->sk,TCP_FIN_WAIT1 ); - ssock =__mptcp_tcp_fallback_get (msk); -if (ssock) { -release_sock (sock->sk); -pr_debug ("subflow=%p" , ssock->sk); - ret =kernel_sock_shutdown (ssock, how); -sock_put (ssock->sk); -return ret; - } -mptcp_for_each_subflow (msk, subflow) {struct socket *tcp_socket;Signature · 4 lines
2.21.0 _______________________________________________ mptcp mailing list -- mptcp(a)lists.01.org To unsubscribe send an email to mptcp-leave(a)lists.01.org
Matthieu Baerts
[MPTCP] Re: [RFC PATCH 5/5] mptcp: avoid acquiring the msk lock in mptcp_finisch_connect()
Dec 6, 2019 16:16:32 Christoph Paasch :
Quoted 2 lines
"[MPTCP] [RFC PATCH 5/5] mptcp: avoid acquiring the msk lock in
mptcp_finisch_connect()" I know there are two native German speakers here on this list, but let's not push it too far ?
Good catch! This definitely needs to be polished! (FYI, it's "wyko?czenie" in Polish...)
Next challenge: discreetly insert "Donaudampfschifffahrtsgesellschaftskapit�n" in a commit message!