mptcp: fix tcp fallback crash
Subsystems (2)
Florian Westphal
PATCH 1/1 mptcp: fix tcp fallback crash
Christoph Paasch reports following crash:
general protection fault, probably for non-canonical address 0xfe85e717fe88a633: 0000 [#1] PREEMPT SMP CPU: 0 PID: 2874 Comm: syz-executor072 Not tainted 5.6.0-rc5 #62 RIP: 0010:__pv_queued_spin_lock_slowpath+0x1b3/0x2f0 kernel/locking/qspinlock.c:471 [..] queued_spin_lock_slowpath arch/x86/include/asm/qspinlock.h:50 [inline] do_raw_spin_lock include/linux/spinlock.h:181 [inline] spin_lock_bh include/linux/spinlock.h:343 [inline] __mptcp_flush_join_list+0x44/0xb0 net/mptcp/protocol.c:278 mptcp_shutdown+0xb3/0x230 net/mptcp/protocol.c:1882
[..]
Problem is that mptcp_shutdown() socket isn't an mptcp socket, its a plain tcp_sk. Thus, trying to access mptcp_sk specific members accesses garbage.
Root cause is that accept() returns a fallback (tcp) socket, not an mptcp one. There is code in getpeername to detect this and override the sockets stream_ops. But this will only run when accept() caller provided a sockaddr struct. "accept(fd, NULL, 0)" will therefore result in mptcp stream ops, with sock->sk being a tcp_sk. Update the existing fallback handling to detect this as well.
Moreover, mptcp_shutdown did not have fallback handling, and mptcp_poll did it too late so add that there as well.
--- net/mptcp/protocol.c | 50 ++++++++++++++++++++++++++++++++++++++++---- 1 file changed, 46 insertions(+), 4 deletions(-)
net/mptcp/protocol.c +46 -4
@@ -57,10 +57,43 @@ static bool __mptcp_needs_tcp_fallback(const struct mptcp_sock *msk)return msk->first && !sk_is_mptcp (msk->first); } +static struct socket *mptcp_is_tcpsk (struct sock *sk) +{ +struct socket *sock = sk->sk_socket; + +if (sock->sk != sk) +return NULL ; + +if (unlikely (sk->sk_prot == &tcp_prot)) { +/* we are being invoked after mptcp_accept() has +* accepted a non-mp-capable flow: sk is a tcp_sk, +* not an mptcp one. +* +* Hand the socket over to tcp so all further socket ops +* bypass mptcp. +*/ + sock->ops = &inet_stream_ops; +return sock; +#if IS_ENABLED (CONFIG_MPTCP_IPV6 ) + } else if (unlikely(sk->sk_prot == &tcpv6_prot)) { + sock->ops = &inet6_stream_ops; +return sock; +#endif + } + +return NULL ; +} +static struct socket *__mptcp_tcp_fallback (struct mptcp_sock *msk) { +struct socket *sock; +sock_owned_by_me ((const struct sock *)msk); + sock =mptcp_is_tcpsk ((struct sock *)msk); +if (unlikely (sock)) +return sock; +if (likely (!__mptcp_needs_tcp_fallback (msk)))return NULL ;
@@ -84,6 +117,10 @@ static struct socket *__mptcp_socket_create(struct mptcp_sock *msk, int state)struct socket *ssock;int err; + ssock =__mptcp_tcp_fallback (msk); +if (unlikely (ssock)) +return ssock; + ssock =__mptcp_nmpc_socket (msk);if (ssock) goto set_state;
@@ -1820,7 +1857,9 @@ static __poll_t mptcp_poll(struct file *file, struct socket *sock, msk =mptcp_sk (sk);lock_sock (sk); - ssock =__mptcp_nmpc_socket (msk); + ssock =__mptcp_tcp_fallback (msk); +if (!ssock) + ssock =__mptcp_nmpc_socket (msk);if (ssock) { mask = ssock->ops->poll (file, ssock, wait);release_sock (sk);
@@ -1830,9 +1869,6 @@ static __poll_t mptcp_poll(struct file *file, struct socket *sock,release_sock (sk);sock_poll_wait (file, sock, wait);lock_sock (sk); - ssock =__mptcp_tcp_fallback (msk); -if (unlikely (ssock)) -return ssock->ops->poll (file, ssock,NULL );if (test_bit (MPTCP_DATA_READY , &msk->flags)) mask =EPOLLIN |EPOLLRDNORM ;
@@ -1851,11 +1887,17 @@ static int mptcp_shutdown(struct socket *sock, int how) {struct mptcp_sock *msk =mptcp_sk (sock->sk);struct mptcp_subflow_context *subflow; +struct socket *ssock;int ret =0 ;pr_debug ("sk=%p, how=%d" , msk, how);lock_sock (sock->sk); + ssock =__mptcp_tcp_fallback (msk); +if (ssock) { +release_sock (sock->sk); +return inet_shutdown (ssock, how); + }if (how ==SHUT_WR || how ==SHUT_RDWR )inet_sk_state_store (sock->sk,TCP_FIN_WAIT1 );
Signature · 1 line
2.24.1
Davide Caratti
Quoted 24 lines · Florian Westphal
Christoph Paasch reports following crash:
general protection fault, probably for non-canonical address 0xfe85e717fe88a633: 0000 [#1] PREEMPT SMP CPU: 0 PID: 2874 Comm: syz-executor072 Not tainted 5.6.0-rc5 #62 RIP: 0010:__pv_queued_spin_lock_slowpath+0x1b3/0x2f0 kernel/locking/qspinlock.c:471 [..] queued_spin_lock_slowpath arch/x86/include/asm/qspinlock.h:50 [inline] do_raw_spin_lock include/linux/spinlock.h:181 [inline] spin_lock_bh include/linux/spinlock.h:343 [inline] __mptcp_flush_join_list+0x44/0xb0 net/mptcp/protocol.c:278 mptcp_shutdown+0xb3/0x230 net/mptcp/protocol.c:1882[..]
Problem is that
mptcp_shutdown()socket isn't an mptcp socket, its a plaintcp_sk. Thus, trying to accessmptcp_skspecific members accesses garbage.Root cause is that
accept()returns a fallback (tcp) socket, not an mptcp one. There is code in getpeername to detect this and override the socketsstream_ops. But this will only run whenaccept()caller provided a sockaddr struct. "accept(fd, NULL, 0)" will therefore result in mptcp stream ops, withsock->skbeing atcp_sk. Update the existing fallback handling to detect this as well.
Quoted 2 lines
Moreover,
mptcp_shutdowndid not have fallback handling, andmptcp_polldid it too late so add that there as well.
hi Florian,
maybe there is still a point where the NULL dereference can happen, in mptcp_sendmsg()
730 lock_sock(ssk);
731 while (msg_data_left(msg)) {
732 ret = mptcp_sendmsg_frag(sk, ssk, msg, NULL, &timeo, &mss_now,
733 &size_goal);
734 if (ret < 0)
735 break;
736 if (ret == 0 && unlikely(__mptcp_needs_tcp_fallback(msk))) {
737 release_sock(ssk); 738 ssock = __mptcp_tcp_fallback(msk); 739 goto fallback; <--- can ssock can be NULL here?
740 }
741
742 copied += ret;
743 }
wdyt?
Signature · 1 line
davide
Florian Westphal
Quoted 25 lines · Davide Caratti
Quoted 24 lines · Florian Westphal
Christoph Paasch reports following crash:
general protection fault, probably for non-canonical address 0xfe85e717fe88a633: 0000 [#1] PREEMPT SMP CPU: 0 PID: 2874 Comm: syz-executor072 Not tainted 5.6.0-rc5 #62 RIP: 0010:__pv_queued_spin_lock_slowpath+0x1b3/0x2f0 kernel/locking/qspinlock.c:471 [..] queued_spin_lock_slowpath arch/x86/include/asm/qspinlock.h:50 [inline] do_raw_spin_lock include/linux/spinlock.h:181 [inline] spin_lock_bh include/linux/spinlock.h:343 [inline] __mptcp_flush_join_list+0x44/0xb0 net/mptcp/protocol.c:278 mptcp_shutdown+0xb3/0x230 net/mptcp/protocol.c:1882[..]
Problem is that
mptcp_shutdown()socket isn't an mptcp socket, its a plaintcp_sk. Thus, trying to accessmptcp_skspecific members accesses garbage.Root cause is that
accept()returns a fallback (tcp) socket, not an mptcp one. There is code in getpeername to detect this and override the socketsstream_ops. But this will only run whenaccept()caller provided a sockaddr struct. "accept(fd, NULL, 0)" will therefore result in mptcp stream ops, withsock->skbeing atcp_sk. Update the existing fallback handling to detect this as well.
Quoted 7 lines
Quoted 2 lines
Moreover,
mptcp_shutdowndid not have fallback handling, andmptcp_polldid it too late so add that there as well.hi Florian,
maybe there is still a point where the NULL dereference can happen, in
mptcp_sendmsg()
Not because of this specific problem.
Quoted 11 lines
730
lock_sock(ssk);
731 while (msg_data_left(msg)) {
732 ret = mptcp_sendmsg_frag(sk, ssk, msg, NULL, &timeo,&mss_now,
733&size_goal);
734 if (ret < 0)
735 break;
736 if (ret == 0 && unlikely(__mptcp_needs_tcp_fallback(msk))) {737 release_sock(ssk); 738 ssock = __mptcp_tcp_fallback(msk); 739 goto fallback; <--- can ssock can be NULL here?740 }
It looks odd, yes. I propose this separate patch:
Subject: mptcp: consistently retry tcp fallback lookup
There is confusion about when we can assume that the fallback socket is non-null. Handle this consistently and do a full re-lookup.
net/mptcp/protocol.c +2 -3
@@ -734,9 +734,9 @@ static int mptcp_sendmsg(struct sock *sk, struct msghdr *msg, size_t len) goto out; } +fallback: ssock =__mptcp_tcp_fallback (msk);if (unlikely (ssock)) { -fallback:pr_debug ("fallback passthrough" ); ret =sock_sendmsg (ssock, msg);return ret >=0 ? ret + copied : (copied ? copied : ret);
@@ -779,7 +779,6 @@ static int mptcp_sendmsg(struct sock *sk, struct msghdr *msg, size_t len)break ;if (ret ==0 &&unlikely (__mptcp_needs_tcp_fallback (msk))) {release_sock (ssk); - ssock =__mptcp_tcp_fallback (msk); goto fallback; }
@@ -889,9 +888,9 @@ static int mptcp_recvmsg(struct sock *sk, struct msghdr *msg, size_t len,return -EOPNOTSUPP ;lock_sock (sk); +fallback: ssock =__mptcp_tcp_fallback (msk);if (unlikely (ssock)) { -fallback:pr_debug ("fallback-read subflow=%p" ,mptcp_subflow_ctx (ssock->sk));
Christoph Paasch
Tested-by on v1 1/1Quoted 24 lines · Florian Westphal
Christoph Paasch reports following crash:
general protection fault, probably for non-canonical address 0xfe85e717fe88a633: 0000 [#1] PREEMPT SMP CPU: 0 PID: 2874 Comm: syz-executor072 Not tainted 5.6.0-rc5 #62 RIP: 0010:__pv_queued_spin_lock_slowpath+0x1b3/0x2f0 kernel/locking/qspinlock.c:471 [..] queued_spin_lock_slowpath arch/x86/include/asm/qspinlock.h:50 [inline] do_raw_spin_lock include/linux/spinlock.h:181 [inline] spin_lock_bh include/linux/spinlock.h:343 [inline] __mptcp_flush_join_list+0x44/0xb0 net/mptcp/protocol.c:278 mptcp_shutdown+0xb3/0x230 net/mptcp/protocol.c:1882[..]
Problem is that
mptcp_shutdown()socket isn't an mptcp socket, its a plaintcp_sk. Thus, trying to accessmptcp_skspecific members accesses garbage.Root cause is that
accept()returns a fallback (tcp) socket, not an mptcp one. There is code in getpeername to detect this and override the socketsstream_ops. But this will only run whenaccept()caller provided a sockaddr struct. "accept(fd, NULL, 0)" will therefore result in mptcp stream ops, withsock->skbeing atcp_sk. Update the existing fallback handling to detect this as well.
Quoted 8 lines
Moreover,
mptcp_shutdowndid not have fallback handling, andmptcp_polldid it too late so add that there as well.Reported-by Christoph Paasch <cpaasch(a)apple.com>Signed-off-by Florian Westphal <fw(a)strlen.de>--- net/mptcp/protocol.c | 50 ++++++++++++++++++++++++++++++++++++++++---- 1 file changed, 46 insertions(+), 4 deletions(-)
Works for me!
I think it also needs the Fixes-tag:
Christoph
Mat Martineau
Reviewed-by on v1 1/1Quoted 24 lines · Florian Westphal
Christoph Paasch reports following crash:
general protection fault, probably for non-canonical address 0xfe85e717fe88a633: 0000 [#1] PREEMPT SMP CPU: 0 PID: 2874 Comm: syz-executor072 Not tainted 5.6.0-rc5 #62 RIP: 0010:__pv_queued_spin_lock_slowpath+0x1b3/0x2f0 kernel/locking/qspinlock.c:471 [..] queued_spin_lock_slowpath arch/x86/include/asm/qspinlock.h:50 [inline] do_raw_spin_lock include/linux/spinlock.h:181 [inline] spin_lock_bh include/linux/spinlock.h:343 [inline] __mptcp_flush_join_list+0x44/0xb0 net/mptcp/protocol.c:278 mptcp_shutdown+0xb3/0x230 net/mptcp/protocol.c:1882[..]
Problem is that
mptcp_shutdown()socket isn't an mptcp socket, its a plaintcp_sk. Thus, trying to accessmptcp_skspecific members accesses garbage.Root cause is that
accept()returns a fallback (tcp) socket, not an mptcp one. There is code in getpeername to detect this and override the socketsstream_ops. But this will only run whenaccept()caller provided a sockaddr struct. "accept(fd, NULL, 0)" will therefore result in mptcp stream ops, withsock->skbeing atcp_sk. Update the existing fallback handling to detect this as well.
Quoted 9 lines
Moreover,
mptcp_shutdowndid not have fallback handling, andmptcp_polldid it too late so add that there as well.Reported-by Christoph Paasch <cpaasch(a)apple.com>Signed-off-by Florian Westphal <fw(a)strlen.de>---
net/mptcp/protocol.c| 50 ++++++++++++++++++++++++++++++++++++++++----1 file changed, 46 insertions(+), 4 deletions(-)
Looks good to me, thanks Florian.
--
Mat Martineau
Intel
Matthieu Baerts
Hi Florian, Mat, Christoph,
Quoted 38 lines · Mat Martineau
Quoted 26 lines · Florian Westphal
Christoph Paasch reports following crash:
general protection fault, probably for non-canonical address 0xfe85e717fe88a633: 0000 [#1] PREEMPT SMP CPU: 0 PID: 2874 Comm: syz-executor072 Not tainted 5.6.0-rc5 #62 RIP: 0010:__pv_queued_spin_lock_slowpath+0x1b3/0x2f0 kernel/locking/qspinlock.c:471 [..] queued_spin_lock_slowpath arch/x86/include/asm/qspinlock.h:50 [inline] do_raw_spin_lock include/linux/spinlock.h:181 [inline] spin_lock_bh include/linux/spinlock.h:343 [inline] __mptcp_flush_join_list+0x44/0xb0 net/mptcp/protocol.c:278 mptcp_shutdown+0xb3/0x230 net/mptcp/protocol.c:1882[..]
Problem is that
mptcp_shutdown()socket isn't an mptcp socket, its a plain tcp_sk. Thus, trying to accessmptcp_skspecific members accesses garbage.Root cause is that
accept()returns a fallback (tcp) socket, not an mptcp one. There is code in getpeername to detect this and override the sockets stream_ops. But this will only run whenaccept()caller provided a sockaddr struct. "accept(fd, NULL, 0)" will therefore result in mptcp stream ops, withsock->skbeing a tcp_sk. Update the existing fallback handling to detect this as well.Quoted 9 lines
Moreover,
mptcp_shutdowndid not have fallback handling, andmptcp_polldid it too late so add that there as well.Reported-by Christoph Paasch <cpaasch(a)apple.com>Signed-off-by Florian Westphal <fw(a)strlen.de>---
net/mptcp/protocol.c| 50 ++++++++++++++++++++++++++++++++++++++++----1 file changed, 46 insertions(+), 4 deletions(-)
Quoted 3 lines
Looks good to me, thanks Florian.
Reviewed-by Mat Martineau <mathew.j.martineau(a)linux.intel.com>
Thank you for the patch, the review, the bug report and the test!
- 1367f91c2e4a: mptcp: fix tcp fallback crash
- 57580002e381: tg: remove accidental signed-off
I just added this patch at the end of the series (export branch)
Tests are all green on my side!
Cheers,
Matt
Signature · 5 lines
Matthieu Baerts | R&D Engineer matthieu.baerts(a)tessares.net Tessares SA | Hybrid Access Solutions www.tessares.net 1 Avenue Jean Monnet, 1348 Louvain-la-Neuve, Belgium