Skip to contentSkip to conversation

mptcp: fix syzkaller splats

PATCH Superseded Florian Westphal 5 participants 11 messages Jan 26, 2020 – Jan 28, 2020

Subsystems (2)

v1 of 2 Superseded Latest is v2 →

lore
Share & tools
lore

Florian Westphal

author
#0 lore raw

cover mptcp: fix syzkaller splats

This fixes the syzkaller splats with reproducers that Christoph forwarded to the mailing list.

First patch fixes a crash with set/getsockopt interface and resolves a lockdep splat due to calls to rtnl_lock while already holding the msk lock. I resolved both in the same patch because the fix touches the same spot.

Second patch fixes another lockdep splat, this time during mptcp_close but also related to rtnl_lock.

See patches for more detailed explanation.

 protocol.c |   51 +++++++++++++++++++++++++++++++++------------------
 1 file changed, 33 insertions(+), 18 deletions(-)
author
#1 lore raw

PATCH v3 1/2 mptcp: protocol: fix panic on user pointer access

Its not possible to call the kernel_(s|g)etsockopt functions here, the address points to user memory:

General protection fault in user access. Non-canonical address?

WARNING: CPU: 1 PID: 5352 at arch/x86/mm/extable.c:77 ex_handler_uaccess+0xba/0xe0 arch/x86/mm/extable.c:77
Kernel panic - not syncing: panic_on_warn set ...
[..]
Call Trace:
 fixup_exception+0x9d/0xcd arch/x86/mm/extable.c:178
 general_protection+0x2d/0x40 arch/x86/entry/entry_64.S:1202
 do_ip_getsockopt+0x1f6/0x1860 net/ipv4/ip_sockglue.c:1323
 ip_getsockopt+0x87/0x1c0 net/ipv4/ip_sockglue.c:1561
 tcp_getsockopt net/ipv4/tcp.c:3691 [inline]
 tcp_getsockopt+0x8c/0xd0 net/ipv4/tcp.c:3685
 kernel_getsockopt+0x121/0x1f0 net/socket.c:3736
 mptcp_getsockopt+0x69/0x90 net/mptcp/protocol.c:830
 __sys_getsockopt+0x13a/0x220 net/socket.c:2175

WARNING: possible circular locking dependency detected
5.5.0-rc6 #2 Not tainted
------------------------------------------------------
syz-executor.0/16334 is trying to acquire lock:
ffffffff84f7a080 (rtnl_mutex){+.+.}, at: do_ip_setsockopt.isra.0+0x277/0x3820 net/ipv4/ip_sockglue.c:644 but task is already holding lock:
ffff888116503b90 (sk_lock-AF_INET){+.+.}, at: lock_sock include/net/sock.h:1516 [inline] ffff888116503b90 (sk_lock-AF_INET){+.+.}, at: mptcp_setsockopt+0x28/0x90 net/mptcp/protocol.c:1284

which lock already depends on the new lock. the existing dependency chain (in reverse order) is:

-> #1 (sk_lock-AF_INET){+.+.}:

        lock_sock_nested+0xca/0x120 net/core/sock.c:2944
        lock_sock include/net/sock.h:1516 [inline]
        do_ip_setsockopt.isra.0+0x281/0x3820 net/ipv4/ip_sockglue.c:645
        ip_setsockopt+0x44/0xf0 net/ipv4/ip_sockglue.c:1248
        udp_setsockopt+0x5d/0xa0 net/ipv4/udp.c:2639
        __sys_setsockopt+0x152/0x240 net/socket.c:2130
        __do_sys_setsockopt net/socket.c:2146 [inline]
        __se_sys_setsockopt net/socket.c:2143 [inline]
        __x64_sys_setsockopt+0xba/0x150 net/socket.c:2143
        do_syscall_64+0xbd/0x5b0 arch/x86/entry/common.c:294
        entry_SYSCALL_64_after_hwframe+0x49/0xbe

-> #0 (rtnl_mutex){+.+.}:

        check_prev_add kernel/locking/lockdep.c:2475 [inline]
        check_prevs_add kernel/locking/lockdep.c:2580 [inline]
        validate_chain kernel/locking/lockdep.c:2970 [inline]
        __lock_acquire+0x1fb2/0x4680 kernel/locking/lockdep.c:3954
        lock_acquire+0x127/0x330 kernel/locking/lockdep.c:4484
        __mutex_lock_common kernel/locking/mutex.c:956 [inline]
        __mutex_lock+0x158/0x1340 kernel/locking/mutex.c:1103
        do_ip_setsockopt.isra.0+0x277/0x3820 net/ipv4/ip_sockglue.c:644
        ip_setsockopt+0x44/0xf0 net/ipv4/ip_sockglue.c:1248
        tcp_setsockopt net/ipv4/tcp.c:3159 [inline]
        tcp_setsockopt+0x8c/0xd0 net/ipv4/tcp.c:3153
        kernel_setsockopt+0x121/0x1f0 net/socket.c:3767
        mptcp_setsockopt+0x69/0x90 net/mptcp/protocol.c:1288
        __sys_setsockopt+0x152/0x240 net/socket.c:2130
        __do_sys_setsockopt net/socket.c:2146 [inline]
        __se_sys_setsockopt net/socket.c:2143 [inline]
        __x64_sys_setsockopt+0xba/0x150 net/socket.c:2143
        do_syscall_64+0xbd/0x5b0 arch/x86/entry/common.c:294
        entry_SYSCALL_64_after_hwframe+0x49/0xbe

other info that might help us debug this:

  Possible unsafe locking scenario:
        CPU0                    CPU1
        ----                    ----
   lock(sk_lock-AF_INET);
                                lock(rtnl_mutex);
                                lock(sk_lock-AF_INET);

lock(rtnl_mutex);

Fixes 717e79c867ca5 ("mptcp: Add setsockopt()/getsockopt() socket operations")
Reported-by Christoph Paasch <cpaasch(a)apple.com>
Signed-off-by Florian Westphal <fw(a)strlen.de>
---
 net/mptcp/protocol.c | 42 ++++++++++++++++++++++++++----------------
 1 file changed, 26 insertions(+), 16 deletions(-)

v2: also fix a circular locking dependency reported by Christoph:
release the mptcp-level lock before calling set/getsockopt on the underlying tcp subflow socket, else this yields:

v3: fix function name: we call the subflow sockets set/getsockopt
functions.

net/mptcp/protocol.c +26 -16
index 39fdca79ce90..f8232e9f08e7 100644
@@ -778,17 +778,23 @@ static void mptcp_destroy(struct sock *sk)
 {
 }
 
+static int subflow_setsockopt(struct socket *sock, int level, int optname,
+			       char __user *optval, unsigned int optlen)
+{
+	if (level == SOL_SOCKET)
+		return sock_setsockopt(sock, level, optname, optval, optlen);
+
+	return sock->ops->setsockopt(sock, level, optname, optval,
+				     optlen);
+}
+
 static int mptcp_setsockopt(struct sock *sk, int level, int optname,
-			    char __user *uoptval, unsigned int optlen)
+			    char __user *optval, unsigned int optlen)
 {
 	struct mptcp_sock *msk = mptcp_sk(sk);
-	char __kernel *optval;
 	int ret = -EOPNOTSUPP;
 	struct socket *ssock;
 
-	/* will be treated as __user in tcp_setsockopt */
-	optval = (char __kernel __force *)uoptval;
-
 	pr_debug("msk=%p", msk);
 
 	/* @@ the meaning of setsockopt() when the socket is connected and
@@ -796,28 +802,32 @@ static int mptcp_setsockopt(struct sock *sk, int level, int optname,
 	 */
 	lock_sock(sk);
 	ssock = __mptcp_socket_create(msk, MPTCP_SAME_STATE);
+	release_sock(sk);
 	if (!IS_ERR(ssock)) {
 		pr_debug("subflow=%p", ssock->sk);
-		ret = kernel_setsockopt(ssock, level, optname, optval, optlen);
+		ret = subflow_setsockopt(ssock, level, optname, optval, optlen);
 	}
-	release_sock(sk);
 
 	return ret;
 }
 
+static int subflow_getsockopt(struct socket *sock, int level, int optname,
+			       char __user *optval, int __user *optlen)
+{
+	if (level == SOL_SOCKET)
+		return sock_getsockopt(sock, level, optname, optval, optlen);
+
+	return sock->ops->getsockopt(sock, level, optname, optval,
+				     optlen);
+}
+
 static int mptcp_getsockopt(struct sock *sk, int level, int optname,
-			    char __user *uoptval, int __user *uoption)
+			    char __user *optval, int __user *option)
 {
 	struct mptcp_sock *msk = mptcp_sk(sk);
-	char __kernel *optval;
 	int ret = -EOPNOTSUPP;
-	int __kernel *option;
 	struct socket *ssock;
 
-	/* will be treated as __user in tcp_getsockopt */
-	optval = (char __kernel __force *)uoptval;
-	option = (int __kernel __force *)uoption;
-
 	pr_debug("msk=%p", msk);
 
 	/* @@ the meaning of getsockopt() when the socket is connected and
@@ -825,11 +835,11 @@ static int mptcp_getsockopt(struct sock *sk, int level, int optname,
 	 */
 	lock_sock(sk);
 	ssock = __mptcp_socket_create(msk, MPTCP_SAME_STATE);
+	release_sock(sk);
 	if (!IS_ERR(ssock)) {
 		pr_debug("subflow=%p", ssock->sk);
-		ret = kernel_getsockopt(ssock, level, optname, optval, option);
+		ret = subflow_getsockopt(ssock, level, optname, optval, option);
 	}
-	release_sock(sk);
 
 	return ret;
 }
Signature · 1 line
2.24.1
#2 lore raw

[MPTCP] Re: [PATCH v3 net-next 1/2] mptcp: protocol: fix panic on user pointer access

Hi Florian,

Quoted 2 lines · Florian Westphal

Its not possible to call the kernel_(s|g)etsockopt functions here, the address points to user memory:

I am sorry for this maybe stupid question but why can we not use these functions directly? Is it because of the "(g|s)et_fs()"?

[...]

net/mptcp/protocol.c static void mptcp_destroy(struct sock *sk) · line 784

Quoted 12 lines
net/mptcp/protocol.c +4 -0
index 39fdca79ce90..f8232e9f08e7 100644
@@ -778,17 +778,23 @@ static void mptcp_destroy(struct sock *sk)
  {
  }
 
+static int subflow_setsockopt(struct socket *sock, int level, int optname,
+			       char __user *optval, unsigned int optlen)
+{
+	if (level == SOL_SOCKET)

Can we arrive in this case with MPTCP? I guess not form the userspace' syscall but maybe from somewhere else in the kernel code?

Cheers,
Matt

Signature · 5 lines
Matthieu Baerts | R&D Engineer
matthieu.baerts(a)tessares.net
Tessares SA | Hybrid Access Solutions
www.tessares.net
1 Avenue Jean Monnet, 1348 Louvain-la-Neuve, Belgium
#3 lore raw

[MPTCP] Re: [PATCH v3 net-next 1/2] mptcp: protocol: fix panic on user pointer access

Quoted 8 lines · Matthieu Baerts

Hi Florian,

Quoted 2 lines · Florian Westphal

Its not possible to call the kernel_(s|g)etsockopt functions here, the address points to user memory:

I am sorry for this maybe stupid question but why can we not use these functions directly? Is it because of the "(g|s)et_fs()"?

Yes, the set_fs(KERNEL_DS) bypasses some checks for userspace memory ranges.

Quoted 17 lines

[...]

net/mptcp/protocol.c static void mptcp_destroy(struct sock *sk) · line 784

Quoted 12 lines
net/mptcp/protocol.c +3 -0
index 39fdca79ce90..f8232e9f08e7 100644
@@ -778,17 +778,23 @@ static void mptcp_destroy(struct sock *sk)
  {
  }
  +static int subflow_setsockopt(struct socket *sock, int level, int 
 optname,
+			       char __user *optval, unsigned int optlen)
+{
+	if (level == SOL_SOCKET)

Can we arrive in this case with MPTCP? I guess not form the userspace' syscall but maybe from somewhere else in the kernel code?

Right now there isn't any code in mptcp_setsockopt() that checks which levels or options are allowed through to the subflows. We discussed whitelisting the socket options allowed to pass through to the subflows - but the patch to whitelist TCP_CONGESTION was mostly dropped and allowed everything instead of allowing nothing. I'm not sure that's what was intended, I will add it to this week's agenda.

For this patch, though, the SOL_SOCKET check is needed to match kernel_setsockopt behavior.

I think this patch looks good, and we can handle the whitelisting in a separate patch depending on what we figure out in the meeting.

--
Mat Martineau
Intel

#4 lore raw

[MPTCP] Re: [PATCH v3 net-next 1/2] mptcp: protocol: fix panic on user pointer access

Quoted 38 lines · Mat Martineau
Quoted 8 lines · Matthieu Baerts

Hi Florian,

Quoted 2 lines · Florian Westphal

Its not possible to call the kernel_(s|g)etsockopt functions here, the address points to user memory:

I am sorry for this maybe stupid question but why can we not use these functions directly? Is it because of the "(g|s)et_fs()"?

Yes, the set_fs(KERNEL_DS) bypasses some checks for userspace memory ranges.

Quoted 16 lines

[...]

net/mptcp/protocol.c static void mptcp_destroy(struct sock *sk) · line 784

Quoted 12 lines
net/mptcp/protocol.c +3 -0
index 39fdca79ce90..f8232e9f08e7 100644
@@ -778,17 +778,23 @@ static void mptcp_destroy(struct sock *sk)
  {
  }
  +static int subflow_setsockopt(struct socket *sock, int level, int 
 optname,
+			       char __user *optval, unsigned int optlen)
+{
+	if (level == SOL_SOCKET)

Can we arrive in this case with MPTCP? I guess not form the userspace' syscall but maybe from somewhere else in the kernel code?

Right now there isn't any code in mptcp_setsockopt() that checks which levels or options are allowed through to the subflows. We discussed whitelisting the socket options allowed to pass through to the subflows - but the patch to whitelist TCP_CONGESTION was mostly dropped and allowed everything instead of allowing nothing. I'm not sure that's what was intended, I will add it to this week's agenda.

Quoted 2 lines

For this patch, though, the SOL_SOCKET check is needed to match kernel_setsockopt behavior.

Uhm... I think handling of SOL_SOCKET is not needed ?!? we reach mptcp_setsockopt via __sys_setsockopt(), which already took care of SOL_SOCKET case. The check inside kernel_{get,set}sockopt was rendundant. I think calling directly ssock->ops->setsockopt suffice (and also keep the code smaller)

Thanks,

Paolo

#5 lore raw

[MPTCP] Re: [PATCH v3 net-next 1/2] mptcp: protocol: fix panic on user pointer access

Hi Paolo,

Quoted 39 lines · Paolo Abeni
Quoted 38 lines · Mat Martineau
Quoted 8 lines · Matthieu Baerts

Hi Florian,

Quoted 2 lines · Florian Westphal

Its not possible to call the kernel_(s|g)etsockopt functions here, the address points to user memory:

I am sorry for this maybe stupid question but why can we not use these functions directly? Is it because of the "(g|s)et_fs()"?

Yes, the set_fs(KERNEL_DS) bypasses some checks for userspace memory ranges.

Quoted 16 lines

[...]

net/mptcp/protocol.c static void mptcp_destroy(struct sock *sk) · line 784

Quoted 12 lines
net/mptcp/protocol.c +3 -0
index 39fdca79ce90..f8232e9f08e7 100644
@@ -778,17 +778,23 @@ static void mptcp_destroy(struct sock *sk)
   {
   }
   +static int subflow_setsockopt(struct socket *sock, int level, int
 optname,
+			       char __user *optval, unsigned int optlen)
+{
+	if (level == SOL_SOCKET)

Can we arrive in this case with MPTCP? I guess not form the userspace' syscall but maybe from somewhere else in the kernel code?

Right now there isn't any code in mptcp_setsockopt() that checks which levels or options are allowed through to the subflows. We discussed whitelisting the socket options allowed to pass through to the subflows - but the patch to whitelist TCP_CONGESTION was mostly dropped and allowed everything instead of allowing nothing. I'm not sure that's what was intended, I will add it to this week's agenda.

Quoted 8 lines
Quoted 2 lines

For this patch, though, the SOL_SOCKET check is needed to match kernel_setsockopt behavior.

Uhm... I think handling of SOL_SOCKET is not needed ?!? we reach mptcp_setsockopt via __sys_setsockopt(), which already took care of SOL_SOCKET case. The check inside kernel_{get,set}sockopt was rendundant. I think calling directly ssock->ops->setsockopt suffice (and also keep the code smaller)

Do we need to protect against other parts of the kernel calling:

sock->ops->(g|s)etsockopt()

directly, possibly with level == SOL_SOCKET?

Cheers,
Matt

Signature · 5 lines
Matthieu Baerts | R&D Engineer
matthieu.baerts(a)tessares.net
Tessares SA | Hybrid Access Solutions
www.tessares.net
1 Avenue Jean Monnet, 1348 Louvain-la-Neuve, Belgium
#6 lore raw

[MPTCP] Re: [PATCH v3 net-next 1/2] mptcp: protocol: fix panic on user pointer access

Quoted 51 lines · Matthieu Baerts

Hi Paolo,

Quoted 39 lines · Paolo Abeni
Quoted 38 lines · Mat Martineau
Quoted 8 lines · Matthieu Baerts

Hi Florian,

Quoted 2 lines · Florian Westphal

Its not possible to call the kernel_(s|g)etsockopt functions here, the address points to user memory:

I am sorry for this maybe stupid question but why can we not use these functions directly? Is it because of the "(g|s)et_fs()"?

Yes, the set_fs(KERNEL_DS) bypasses some checks for userspace memory ranges.

Quoted 16 lines

[...]

net/mptcp/protocol.c static void mptcp_destroy(struct sock *sk) · line 784

Quoted 12 lines
net/mptcp/protocol.c +3 -0
index 39fdca79ce90..f8232e9f08e7 100644
@@ -778,17 +778,23 @@ static void mptcp_destroy(struct sock *sk)
   {
   }
   +static int subflow_setsockopt(struct socket *sock, int level, int
 optname,
+			       char __user *optval, unsigned int optlen)
+{
+	if (level == SOL_SOCKET)

Can we arrive in this case with MPTCP? I guess not form the userspace' syscall but maybe from somewhere else in the kernel code?

Right now there isn't any code in mptcp_setsockopt() that checks which levels or options are allowed through to the subflows. We discussed whitelisting the socket options allowed to pass through to the subflows - but the patch to whitelist TCP_CONGESTION was mostly dropped and allowed everything instead of allowing nothing. I'm not sure that's what was intended, I will add it to this week's agenda.

Quoted 8 lines
Quoted 2 lines

For this patch, though, the SOL_SOCKET check is needed to match kernel_setsockopt behavior.

Uhm... I think handling of SOL_SOCKET is not needed ?!? we reach mptcp_setsockopt via __sys_setsockopt(), which already took care of SOL_SOCKET case. The check inside kernel_{get,set}sockopt was rendundant. I think calling directly ssock->ops->setsockopt suffice (and also keep the code smaller)

Quoted 5 lines

Do we need to protect against other parts of the kernel calling:

   sock->ops->(g|s)etsockopt()

directly, possibly with level == SOL_SOCKET?

I think we don't, as we don't for tcp_{g,s}etsockopt(). Why need that additionallty here?!?

Paolo

#7 lore raw

[MPTCP] Re: [PATCH v3 net-next 1/2] mptcp: protocol: fix panic on user pointer access

Quoted 52 lines · Paolo Abeni
Quoted 51 lines · Matthieu Baerts

Hi Paolo,

Quoted 47 lines · Paolo Abeni
Quoted 38 lines · Mat Martineau
Quoted 8 lines · Matthieu Baerts

Hi Florian,

Quoted 2 lines · Florian Westphal

Its not possible to call the kernel_(s|g)etsockopt functions here, the address points to user memory:

I am sorry for this maybe stupid question but why can we not use these functions directly? Is it because of the "(g|s)et_fs()"?

Yes, the set_fs(KERNEL_DS) bypasses some checks for userspace memory ranges.

Quoted 16 lines

[...]

net/mptcp/protocol.c static void mptcp_destroy(struct sock *sk) · line 784

Quoted 12 lines
net/mptcp/protocol.c +3 -0
index 39fdca79ce90..f8232e9f08e7 100644
@@ -778,17 +778,23 @@ static void mptcp_destroy(struct sock *sk)
   {
   }
   +static int subflow_setsockopt(struct socket *sock, int level, int
 optname,
+			       char __user *optval, unsigned int optlen)
+{
+	if (level == SOL_SOCKET)

Can we arrive in this case with MPTCP? I guess not form the userspace' syscall but maybe from somewhere else in the kernel code?

Right now there isn't any code in mptcp_setsockopt() that checks which levels or options are allowed through to the subflows. We discussed whitelisting the socket options allowed to pass through to the subflows - but the patch to whitelist TCP_CONGESTION was mostly dropped and allowed everything instead of allowing nothing. I'm not sure that's what was intended, I will add it to this week's agenda.

Quoted 2 lines

For this patch, though, the SOL_SOCKET check is needed to match kernel_setsockopt behavior.

Uhm... I think handling of SOL_SOCKET is not needed ?!? we reach mptcp_setsockopt via __sys_setsockopt(), which already took care of SOL_SOCKET case. The check inside kernel_{get,set}sockopt was rendundant. I think calling directly ssock->ops->setsockopt suffice (and also keep the code smaller)

Quoted 8 lines
Quoted 5 lines

Do we need to protect against other parts of the kernel calling:

   sock->ops->(g|s)etsockopt()

directly, possibly with level == SOL_SOCKET?

I think we don't, as we don't for tcp_{g,s}etsockopt(). Why need that additionallty here?!?

I think Paolo's right (I forgot to look at __sys_setsockopt). Any direct calls in the kernel would be for the MPTCP-level socket and would use kernel_{get,set}sockopt that handles SOL_SOCKET.

--
Mat Martineau
Intel

author
#8 lore raw

PATCH 2/2 mptcp: mptcp_close: avoid a lockdep splat when mcast group was joined

In reply to Florian Westphal

syzbot triggered following lockdep splat:

ffffffff82d2cd40 (rtnl_mutex){+.+.}, at: ip_mc_drop_socket+0x52/0x180 but task is already holding lock:
ffff8881187a2310 (sk_lock-AF_INET){+.+.}, at: mptcp_close+0x18/0x30

which lock already depends on the new lock.

the existing dependency chain (in reverse order) is:
-> #1 (sk_lock-AF_INET){+.+.}:

       lock_acquire+0xee/0x230
       lock_sock_nested+0x89/0xc0
       do_ip_setsockopt.isra.0+0x335/0x22f0
       ip_setsockopt+0x35/0x60
       tcp_setsockopt+0x5d/0x90
       __sys_setsockopt+0xf3/0x190
       __x64_sys_setsockopt+0x61/0x70
       do_syscall_64+0x72/0x300
       entry_SYSCALL_64_after_hwframe+0x49/0xbe
-> #0 (rtnl_mutex){+.+.}:
       check_prevs_add+0x2b7/0x1210
       __lock_acquire+0x10b6/0x1400
       lock_acquire+0xee/0x230
       __mutex_lock+0x120/0xc70
       ip_mc_drop_socket+0x52/0x180
       inet_release+0x36/0xe0
       __sock_release+0xfd/0x130
       __mptcp_close+0xa8/0x1f0
       inet_release+0x7f/0xe0
       __sock_release+0x69/0x130
       sock_close+0x18/0x20
       __fput+0x179/0x400
       task_work_run+0xd5/0x110
       do_exit+0x685/0x1510
       do_group_exit+0x7e/0x170
       __x64_sys_exit_group+0x28/0x30
       do_syscall_64+0x72/0x300
       entry_SYSCALL_64_after_hwframe+0x49/0xbe

The trigger is:
socket(AF_INET, SOCK_STREAM, 0x106 /* IPPROTO_MPTCP */) = 4
setsockopt(4, SOL_IP, MCAST_JOIN_GROUP, {gr_interface=7, gr_group={sa_family=AF_INET, sin_port=htons(20003), sin_addr=inet_addr("224.0.0.2")}}, 136) = 0 exit(0)

Which results in a call to rtnl_lock while we are holding
the parent mptcp socket lock via
mptcp_close -> lock_sock(msk) -> inet_release -> ip_mc_drop_socket -> rtnl_lock().

From lockdep point of view we thus have both 'rtnl_lock; lock_sock' and 'lock_sock; rtnl_lock'.

Fix this by stealing the msk conn_list and doing the subflow close without holding the msk lock.

Fixes cec37a6e41aae7bf ("mptcp: Handle MP_CAPABLE options for outgoing connections")
Reported-by Christoph Paasch <cpaasch(a)apple.com>
Signed-off-by Florian Westphal <fw(a)strlen.de>
---
 net/mptcp/protocol.c | 9 +++++++--
 1 file changed, 7 insertions(+), 2 deletions(-)
net/mptcp/protocol.c +7 -2
index f8232e9f08e7..120b3305a594 100644
@@ -644,11 +644,16 @@ static void __mptcp_close(struct sock *sk, long timeout)
 {
 	struct mptcp_subflow_context *subflow, *tmp;
 	struct mptcp_sock *msk = mptcp_sk(sk);
+	LIST_HEAD(conn_list);
 
 	mptcp_token_destroy(msk->token);
 	inet_sk_state_store(sk, TCP_CLOSE);
 
-	list_for_each_entry_safe(subflow, tmp, &msk->conn_list, node) {
+	list_splice_init(&msk->conn_list, &conn_list);
+
+	release_sock(sk);
+
+	list_for_each_entry_safe(subflow, tmp, &conn_list, node) {
 		struct sock *ssk = mptcp_subflow_tcp_sock(subflow);
 
 		__mptcp_close_ssk(sk, ssk, subflow, timeout);
@@ -656,7 +661,7 @@ static void __mptcp_close(struct sock *sk, long timeout)
 
 	if (msk->cached_ext)
 		__skb_ext_put(msk->cached_ext);
-	release_sock(sk);
+
 	sk_common_release(sk);
 }
 
Signature · 1 line
2.24.1
#9 lore raw

[MPTCP] Re: [PATCH net-next 2/2] mptcp: mptcp_close: avoid a lockdep splat when mcast group was joined

Quoted 68 lines · Florian Westphal

syzbot triggered following lockdep splat:

ffffffff82d2cd40 (rtnl_mutex){+.+.}, at: ip_mc_drop_socket+0x52/0x180 but task is already holding lock:
ffff8881187a2310 (sk_lock-AF_INET){+.+.}, at: mptcp_close+0x18/0x30

which lock already depends on the new lock.

the existing dependency chain (in reverse order) is:
-> #1 (sk_lock-AF_INET){+.+.}:

      lock_acquire+0xee/0x230
      lock_sock_nested+0x89/0xc0
      do_ip_setsockopt.isra.0+0x335/0x22f0
      ip_setsockopt+0x35/0x60
      tcp_setsockopt+0x5d/0x90
      __sys_setsockopt+0xf3/0x190
      __x64_sys_setsockopt+0x61/0x70
      do_syscall_64+0x72/0x300
      entry_SYSCALL_64_after_hwframe+0x49/0xbe
-> #0 (rtnl_mutex){+.+.}:
      check_prevs_add+0x2b7/0x1210
      __lock_acquire+0x10b6/0x1400
      lock_acquire+0xee/0x230
      __mutex_lock+0x120/0xc70
      ip_mc_drop_socket+0x52/0x180
      inet_release+0x36/0xe0
      __sock_release+0xfd/0x130
      __mptcp_close+0xa8/0x1f0
      inet_release+0x7f/0xe0
      __sock_release+0x69/0x130
      sock_close+0x18/0x20
      __fput+0x179/0x400
      task_work_run+0xd5/0x110
      do_exit+0x685/0x1510
      do_group_exit+0x7e/0x170
      __x64_sys_exit_group+0x28/0x30
      do_syscall_64+0x72/0x300
      entry_SYSCALL_64_after_hwframe+0x49/0xbe

The trigger is:
socket(AF_INET, SOCK_STREAM, 0x106 /* IPPROTO_MPTCP */) = 4
setsockopt(4, SOL_IP, MCAST_JOIN_GROUP, {gr_interface=7, gr_group={sa_family=AF_INET, sin_port=htons(20003), sin_addr=inet_addr("224.0.0.2")}}, 136) = 0 exit(0)

Which results in a call to rtnl_lock while we are holding
the parent mptcp socket lock via
mptcp_close -> lock_sock(msk) -> inet_release -> ip_mc_drop_socket -> rtnl_lock().

From lockdep point of view we thus have both 'rtnl_lock; lock_sock' and 'lock_sock; rtnl_lock'.

Fix this by stealing the msk conn_list and doing the subflow close without holding the msk lock.

Fixes cec37a6e41aae7bf ("mptcp: Handle MP_CAPABLE options for outgoing connections")
Reported-by Christoph Paasch <cpaasch(a)apple.com>
Signed-off-by Florian Westphal <fw(a)strlen.de>

---
net/mptcp/protocol.c | 9 +++++++--

1 file changed, 7 insertions(+), 2 deletions(-)
net/mptcp/protocol.c +0 -0
index f8232e9f08e7..120b3305a594 100644
@@ -644,11 +644,16 @@ static void __mptcp_close(struct sock *sk, long timeout)
 {
 	struct mptcp_subflow_context *subflow, *tmp;

net/mptcp/protocol.c static void __mptcp_close(struct sock *sk, long timeout) · line 654

Quoted 10 lines
net/mptcp/protocol.c +4 -1
 	struct mptcp_sock *msk = mptcp_sk(sk);
+	LIST_HEAD(conn_list);
 
 	mptcp_token_destroy(msk->token);
 	inet_sk_state_store(sk, TCP_CLOSE);
 
-	list_for_each_entry_safe(subflow, tmp, &msk->conn_list, node) {
+	list_splice_init(&msk->conn_list, &conn_list);
+
+	release_sock(sk);

Moving the release_sock() here also means the lock is not held for the cached_ext and __mptcp_clear_xmit cleanup. Should those be moved around to happen under the lock or use some other temp variables to safely clean up?

Mat

Quoted 20 lines
net/mptcp/protocol.c +3 -1
+
+	list_for_each_entry_safe(subflow, tmp, &conn_list, node) {
 		struct sock *ssk = mptcp_subflow_tcp_sock(subflow);
 
 		__mptcp_close_ssk(sk, ssk, subflow, timeout);
@@ -656,7 +661,7 @@ static void __mptcp_close(struct sock *sk, long timeout)
 
 	if (msk->cached_ext)
 		__skb_ext_put(msk->cached_ext);
-	release_sock(sk);
+
 	sk_common_release(sk);
 }
 
Signature · 4 lines
2.24.1
_______________________________________________
mptcp mailing list -- mptcp(a)lists.01.org
To unsubscribe send an email to mptcp-leave(a)lists.01.org

--
Mat Martineau
Intel

Tested-by on the v1 series
#10 lore raw
In reply to Florian Westphal
Quoted 7 lines · Florian Westphal

This fixes the syzkaller splats with reproducers that Christoph forwarded to the mailing list.

First patch fixes a crash with set/getsockopt interface and resolves a lockdep splat due to calls to rtnl_lock while already holding the msk lock. I resolved both in the same patch because the fix touches the same spot.

Quoted 7 lines

Second patch fixes another lockdep splat, this time during mptcp_close but also related to rtnl_lock.

See patches for more detailed explanation.

 protocol.c |   51 +++++++++++++++++++++++++++++++++------------------
 1 file changed, 33 insertions(+), 18 deletions(-)

syzkaller has been running for 300 hours of fuzzing and none of the previous reported issues triggered. All good! Also, no new issues for now.

Tested-by Christoph Paasch <cpaasch(a)apple.com>

Cheers,
Christoph