Skip to contentSkip to conversation

mptcp: msk diag support

PATCH Superseded Paolo Abeni 3 participants 10 messages Jun 26, 2020 – Jul 2, 2020

Subsystems (4)

v2 of 3 Superseded Latest is v3 →

lore
Share & tools
lore

Paolo Abeni

author
#0 lore raw

v2 cover mptcp: msk diag support

This introduces basic mptcp sockets diag support.
As IPPROTO_MPTCP excedes 8 bits, we need some changes at the inet_diag level:
a new attribute is introduced to allow user-space providing u32 protocol values.

Patch 2 introduces new token APIs to allow traversing the existing msks, while patch 3 bring in the actual diag implementation.

Patch 4 includes some basic functional tests

v1 -> v2
- fixed dump issue on large dump
- use flags for fallback, etc
- patch 4

Paolo Abeni (4):
inet_diag: support for wider protocol numbers
mptcp: add msk interations helpers
mptcp: add MPTCP socket diag interface
selftests/mptcp: add diag interface tests

 include/uapi/linux/inet_diag.h                |   1 +
 include/uapi/linux/mptcp.h                    |  17 ++
 net/core/sock.c                               |   1 +
 net/ipv4/inet_diag.c                          |  63 +++++--
 net/mptcp/Kconfig                             |   4 +
 net/mptcp/Makefile                            |   2 +
 net/mptcp/mptcp_diag.c                        | 167 ++++++++++++++++++
 net/mptcp/protocol.h                          |   3 +
 net/mptcp/token.c                             |  83 +++++++++
 tools/testing/selftests/net/mptcp/Makefile    |   2 +-
 tools/testing/selftests/net/mptcp/diag.sh     | 122 +++++++++++++
 .../selftests/net/mptcp/mptcp_connect.c       |  14 +-
 12 files changed, 457 insertions(+), 22 deletions(-)
 create mode 100644 net/mptcp/mptcp_diag.c
 create mode 100755 tools/testing/selftests/net/mptcp/diag.sh
Signature · 1 line
2.26.2
author
#1 lore raw

PATCH v2 1/4 inet_diag: support for wider protocol numbers

After commit bf9765145b85 ("sock: Make sk_protocol a 16-bit value") the current size of 'sdiag_protocol' is not sufficient to represent the possible protocol values.

This change introduces a new inet diag request attribute to let user space specify the relevant protocol number using u32 values.

The attribute is parsed by inet diag core on get/dump command and the extended protocol value, if available, is preferred to 'sdiag_protocol' to lookup the diag handler.

The parse attributed are exposed to all the diag handlers via the cb->data.

Note that inet_diag_dump_one_icsk() is left unmodified, as it will not be used by protocol using the extended attribute.

Suggested-by David S. Miller <davem(a)davemloft.net>
Signed-off-by Paolo Abeni <pabeni(a)redhat.com>
---
 include/uapi/linux/inet_diag.h |  1 +
 net/core/sock.c                |  1 +
 net/ipv4/inet_diag.c           | 63 +++++++++++++++++++++++++---------
 3 files changed, 48 insertions(+), 17 deletions(-)
include/uapi/linux/inet_diag.h +1 -0
index e6f183ee8417..5ba122c1949a 100644
@@ -65,6 +65,7 @@ enum {
 	INET_DIAG_REQ_NONE,
 	INET_DIAG_REQ_BYTECODE,
 	INET_DIAG_REQ_SK_BPF_STORAGES,
+	INET_DIAG_REQ_PROTOCOL,
 	__INET_DIAG_REQ_MAX,
 };
 
net/core/sock.c +1 -0
index 6c4acf1f0220..b359e29bb737 100644
@@ -3573,6 +3573,7 @@ int sock_load_diag_module(int family, int protocol)
 #ifdef CONFIG_INET
 	if (family == AF_INET &&
 	    protocol != IPPROTO_RAW &&
+	    protocol < MAX_INET_PROTOS &&
 	    !rcu_access_pointer(inet_protos[protocol]))
 		return -ENOENT;
 #endif
net/ipv4/inet_diag.c +46 -17

131 lines not on this page · raw message

Signature · 1 line
2.26.2
author
#3 lore raw

PATCH v2 2/4 mptcp: add msk interations helpers

In reply to Paolo Abeni

mptcp_token_iter_start(), mptcp_token_iter_next() and mptcp_token_iter_stop() allow traversing all the MPTCP sockets inside the token container with a quite standard iterator semantic.

Will be used by the next patch.

Signed-off-by Paolo Abeni <pabeni(a)redhat.com>
---
 net/mptcp/protocol.h |  3 ++
 net/mptcp/token.c    | 83 ++++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 86 insertions(+)
net/mptcp/protocol.h +3 -0
index dfe67e8a86ed..757a0f5f57bb 100644
@@ -391,6 +391,9 @@ int mptcp_token_new_connect(struct sock *sk);
 void mptcp_token_accept(struct mptcp_subflow_request_sock *r,
 			struct mptcp_sock *msk);
 struct mptcp_sock *mptcp_token_get_sock(u32 token);
+struct mptcp_sock *mptcp_token_iter_start(long *s_slot, long *s_num);
+struct mptcp_sock *mptcp_token_iter_next(long *s_slot, long *s_num);
+void mptcp_token_iter_stop(long slot);
 void mptcp_token_destroy(struct mptcp_sock *msk);
 
 void mptcp_crypto_key_sha(u64 key, u32 *token, u64 *idsn);
net/mptcp/token.c +83 -0

89 lines not on this page · raw message

Signature · 1 line
2.26.2
author
#4 lore raw

PATCH v2 3/4 mptcp: add MPTCP socket diag interface

In reply to Paolo Abeni

exposes basic inet socket attribute, plus some MPTCP socket fields comprising PM status and MPTCP-level sequence numbers.

Signed-off-by Paolo Abeni <pabeni(a)redhat.com>

---
v1 -> v2:
- fix dump on skb full event
- use 'flags' for fallback, can_ack

---
 include/uapi/linux/mptcp.h |  17 ++++
 net/mptcp/Kconfig          |   4 +
 net/mptcp/Makefile         |   2 +
 net/mptcp/mptcp_diag.c     | 167 +++++++++++++++++++++++++++++++++++++
 4 files changed, 190 insertions(+)
 create mode 100644 net/mptcp/mptcp_diag.c
include/uapi/linux/mptcp.h +17 -0
index 5f2c77082d9e..9762660df741 100644
@@ -86,4 +86,21 @@ enum {
 	__MPTCP_PM_CMD_AFTER_LAST
 };
 
+#define MPTCP_INFO_FLAG_FALLBACK		_BITUL(0)
+#define MPTCP_INFO_FLAG_REMOTE_KEY_RECEIVED	_BITUL(1)
+
+struct mptcp_info {
+	__u8	mptcpi_subflows;
+	__u8	mptcpi_add_addr_signal;
+	__u8	mptcpi_add_addr_accepted;
+	__u8	mptcpi_subflows_max;
+	__u8	mptcpi_add_addr_signal_max;
+	__u8	mptcpi_add_addr_accepted_max;
+	__u32	mptcpi_flags;
+	__u32	mptcpi_token;
+	__u64	mptcpi_write_seq;
+	__u64	mptcpi_snd_una;
+	__u64	mptcpi_rcv_nxt;
+};
+
 #endif /* _UAPI_MPTCP_H */
net/mptcp/Kconfig +4 -0
index 0ec787d8b9a2..97c29caa65f3 100644
@@ -14,6 +14,10 @@ config MPTCP
 
 if MPTCP
 
+config INET_MPTCP_DIAG
+	depends on INET_DIAG
+	def_tristate INET_DIAG
+
 config MPTCP_IPV6
 	bool "MPTCP: IPv6 support for Multipath TCP"
 	select IPV6
net/mptcp/Makefile +2 -0
index c53f9b845523..2360cbd27d59 100644
@@ -4,6 +4,8 @@ obj-$(CONFIG_MPTCP) += mptcp.o
 mptcp-y := protocol.o subflow.o options.o token.o crypto.o ctrl.o pm.o diag.o \
 	   mib.o pm_netlink.o
 
+obj-$(CONFIG_INET_MPTCP_DIAG) += mptcp_diag.o
+
 mptcp_crypto_test-objs := crypto_test.o
 mptcp_token_test-objs := token_test.o
 obj-$(CONFIG_MPTCP_KUNIT_TESTS) += mptcp_crypto_test.o mptcp_token_test.o
net/mptcp/mptcp_diag.c new file +167 -0

167 lines not on this page · raw message

Signature · 1 line
2.26.2
author
#9 lore raw

PATCH v2 4/4 selftests/mptcp: add diag interface tests

In reply to Paolo Abeni

basic functional test, triggering the msk diag interface code. Require appropriate iproute2 support, skip elsewhere.

Signed-off-by Paolo Abeni <pabeni(a)redhat.com>

---
v1 -> v2:
- include diag.sh in the makefile

---
 tools/testing/selftests/net/mptcp/Makefile    |   2 +-
 tools/testing/selftests/net/mptcp/diag.sh     | 122 ++++++++++++++++++
 .../selftests/net/mptcp/mptcp_connect.c       |  14 +-
 3 files changed, 133 insertions(+), 5 deletions(-)
 create mode 100755 tools/testing/selftests/net/mptcp/diag.sh
tools/testing/selftests/net/mptcp/Makefile +1 -1
index f50976ee7d44..aa254aefc2c3 100644
@@ -5,7 +5,7 @@ KSFT_KHDR_INSTALL := 1
 
 CFLAGS =  -Wall -Wl,--no-as-needed -O2 -g  -I$(top_srcdir)/usr/include
 
-TEST_PROGS := mptcp_connect.sh pm_netlink.sh mptcp_join.sh
+TEST_PROGS := mptcp_connect.sh pm_netlink.sh mptcp_join.sh diag.sh
 
 TEST_GEN_FILES = mptcp_connect pm_nl_ctl
 
tools/testing/selftests/net/mptcp/diag.sh new file +122 -0

123 lines not on this page · raw message

tools/testing/selftests/net/mptcp/mptcp_connect.c +10 -4
index cedee5b952ba..76899d6ef8e2 100644
@@ -52,11 +52,12 @@ static int pf = AF_INET;
 static int cfg_sndbuf;
 static int cfg_rcvbuf;
 static bool cfg_join;
+static int cfg_wait;
 
 static void die_usage(void)
 {
 	fprintf(stderr, "Usage: mptcp_connect [-6] [-u] [-s MPTCP|TCP] [-p port] [-m mode]"
-		"[-l] connect_address\n");
+		"[-l] [-w sec] connect_address\n");
 	fprintf(stderr, "\t-6 use ipv6\n");
 	fprintf(stderr, "\t-t num -- set poll timeout to num\n");
 	fprintf(stderr, "\t-S num -- set SO_SNDBUF to num\n");
@@ -65,6 +66,7 @@ static void die_usage(void)
 	fprintf(stderr, "\t-m [MPTCP|TCP] -- use tcp or mptcp sockets\n");
 	fprintf(stderr, "\t-s [mmap|poll] -- use poll (default) or mmap\n");
 	fprintf(stderr, "\t-u -- check mptcp ulp\n");
+	fprintf(stderr, "\t-w num -- wait num sec before closing the socket\n");
 	exit(1);
 }
 
@@ -418,8 +420,8 @@ static int copyfd_io_poll(int infd, int peerfd, int outfd)
 	}
 
 	/* leave some time for late join/announce */
-	if (cfg_join)
-		usleep(400000);
+	if (cfg_wait)
+		usleep(cfg_wait);
 
 	close(peerfd);
 	return 0;
@@ -812,11 +814,12 @@ static void parse_opts(int argc, char **argv)
 {
 	int c;
 
-	while ((c = getopt(argc, argv, "6jlp:s:hut:m:S:R:")) != -1) {
+	while ((c = getopt(argc, argv, "6jlp:s:hut:m:S:R:w:")) != -1) {
 		switch (c) {
 		case 'j':
 			cfg_join = true;
 			cfg_mode = CFG_MODE_POLL;
+			cfg_wait = 400000;
 			break;
 		case 'l':
 			listen_mode = true;
@@ -850,6 +853,9 @@ static void parse_opts(int argc, char **argv)
 		case 'R':
 			cfg_rcvbuf = parse_int(optarg);
 			break;
+		case 'w':
+			cfg_wait = atoi(optarg)*1000000;
+			break;
 		}
 	}
 
Signature · 1 line
2.26.2
#5 lore raw

[MPTCP] Re: [PATCH v2 3/4] mptcp: add MPTCP socket diag interface

In reply to Paolo Abeni
Quoted 150 lines · Paolo Abeni

exposes basic inet socket attribute, plus some MPTCP socket fields comprising PM status and MPTCP-level sequence numbers.

Signed-off-by Paolo Abeni <pabeni(a)redhat.com>

---
v1 -> v2:
- fix dump on skb full event
- use 'flags' for fallback, can_ack
---

include/uapi/linux/mptcp.h |  17 ++++
net/mptcp/Kconfig          |   4 +
net/mptcp/Makefile         |   2 +
net/mptcp/mptcp_diag.c     | 167 +++++++++++++++++++++++++++++++++++++
4 files changed, 190 insertions(+)

create mode 100644 net/mptcp/mptcp_diag.c

include/uapi/linux/mptcp.h +17 -0
index 5f2c77082d9e..9762660df741 100644
@@ -86,4 +86,21 @@ enum {
 	__MPTCP_PM_CMD_AFTER_LAST
 };
 
+#define MPTCP_INFO_FLAG_FALLBACK		_BITUL(0)
+#define MPTCP_INFO_FLAG_REMOTE_KEY_RECEIVED	_BITUL(1)
+
+struct mptcp_info {
+	__u8	mptcpi_subflows;
+	__u8	mptcpi_add_addr_signal;
+	__u8	mptcpi_add_addr_accepted;
+	__u8	mptcpi_subflows_max;
+	__u8	mptcpi_add_addr_signal_max;
+	__u8	mptcpi_add_addr_accepted_max;
+	__u32	mptcpi_flags;
+	__u32	mptcpi_token;
+	__u64	mptcpi_write_seq;
+	__u64	mptcpi_snd_una;
+	__u64	mptcpi_rcv_nxt;
+};
+
 #endif /* _UAPI_MPTCP_H */
net/mptcp/Kconfig +0 -0
index 0ec787d8b9a2..97c29caa65f3 100644
@@ -14,6 +14,10 @@ config MPTCP
 

if MPTCP

net/mptcp/Kconfig +4 -0
+config INET_MPTCP_DIAG
+	depends on INET_DIAG
+	def_tristate INET_DIAG
+

config MPTCP_IPV6
bool "MPTCP: IPv6 support for Multipath TCP" select IPV6

net/mptcp/Makefile +2 -0
index c53f9b845523..2360cbd27d59 100644
@@ -4,6 +4,8 @@ obj-$(CONFIG_MPTCP) += mptcp.o
 mptcp-y := protocol.o subflow.o options.o token.o crypto.o ctrl.o pm.o diag.o \
 	   mib.o pm_netlink.o
 
+obj-$(CONFIG_INET_MPTCP_DIAG) += mptcp_diag.o
+

mptcp_crypto_test-objs := crypto_test.o
mptcp_token_test-objs := token_test.o obj-$(CONFIG_MPTCP_KUNIT_TESTS) += mptcp_crypto_test.o mptcp_token_test.o

net/mptcp/mptcp_diag.c new file +74 -0
new file mode 100644
index 000000000000..30aa768e23bd
@@ -0,0 +1,167 @@
+// SPDX-License-Identifier: GPL-2.0
+/* MPTCP socket monitoring support
+ *
+ * Copyright (c) 2020 Red Hat
+ *
+ * Author: Paolo Abeni <pabeni(a)redhat.com>
+ */
+
+#include <linux/kernel.h>
+#include <linux/net.h>
+#include <linux/inet_diag.h>
+#include <net/netlink.h>
+#include <uapi/linux/mptcp.h>
+#include "protocol.h"
+
+static int sk_diag_dump(struct sock *sk, struct sk_buff *skb,
+			struct netlink_callback *cb,
+			const struct inet_diag_req_v2 *req,
+			struct nlattr *bc, bool net_admin)
+{
+	if (!inet_diag_bc_sk(bc, sk))
+		return 0;
+
+	return inet_sk_diag_fill(sk, inet_csk(sk), skb, cb, req, NLM_F_MULTI,
+				 net_admin);
+}
+
+static int mptcp_diag_dump_one(struct netlink_callback *cb,
+			       const struct inet_diag_req_v2 *req)
+{
+	struct sk_buff *in_skb = cb->skb;
+	struct mptcp_sock *msk = NULL;
+	struct sk_buff *rep;
+	int err = -ENOENT;
+	struct net *net;
+	struct sock *sk;
+
+	net = sock_net(in_skb->sk);
+	msk = mptcp_token_get_sock(req->id.idiag_cookie[0]);
+	if (!msk)
+		goto out_nosk;
+
+	err = -ENOMEM;
+	sk = (struct sock *)msk;
+	rep = nlmsg_new(nla_total_size(sizeof(struct inet_diag_msg)) +
+			inet_diag_msg_attrs_size() +
+			nla_total_size(sizeof(struct mptcp_info)) +
+			nla_total_size(sizeof(struct inet_diag_meminfo)) + 64,
+			GFP_KERNEL);
+	if (!rep)
+		goto out;
+
+	err = inet_sk_diag_fill(sk, inet_csk(sk), rep, cb, req, 0,
+				netlink_net_capable(in_skb, CAP_NET_ADMIN));
+	if (err < 0) {
+		WARN_ON(err == -EMSGSIZE);
+		kfree_skb(rep);
+		goto out;
+	}
+	err = netlink_unicast(net->diag_nlsk, rep, NETLINK_CB(in_skb).portid,
+			      MSG_DONTWAIT);
+	if (err > 0)
+		err = 0;
+out:
+	sock_put(sk);
+
+out_nosk:
+	return err;
+}
+
+static void mptcp_diag_dump(struct sk_buff *skb, struct netlink_callback *cb,
+			    const struct inet_diag_req_v2 *r)
+{
+	bool net_admin = netlink_net_capable(cb->skb, CAP_NET_ADMIN);

net/mptcp/mptcp_diag.c line 84

Quoted 10 lines
net/mptcp/mptcp_diag.c new file +10 -0
+	struct net *net = sock_net(skb->sk);
+	struct inet_diag_dump_data *cb_data;
+	struct mptcp_sock *msk;
+	struct nlattr *bc;
+
+	cb_data = cb->data;
+	bc = cb_data->inet_diag_nla_bc;
+
+	for (msk = mptcp_token_iter_start(&cb->args[0], &cb->args[1]);
+	     msk; msk = mptcp_token_iter_next(&cb->args[0], &cb->args[1])) {

Should there be a rcu_read_lock/rcu_read_unlock around this loop with mptcp_token_iter_next() using sk_nulls_for_each_rcu()? Or is that already handled somewhere up the call stack?

Quoted 49 lines
net/mptcp/mptcp_diag.c new file +49 -0
+		struct inet_sock *inet;
+		struct sock *sk;
+
+		inet = (struct inet_sock *)msk;
+		sk = (struct sock *)msk;
+		if (!net_eq(sock_net(sk), net))
+			continue;
+		if (!(r->idiag_states & (1 << sk->sk_state)))
+			continue;
+		if (r->sdiag_family != AF_UNSPEC &&
+		    sk->sk_family != r->sdiag_family)
+			continue;
+		if (r->id.idiag_sport != inet->inet_sport &&
+		    r->id.idiag_sport)
+			continue;
+		if (r->id.idiag_dport != inet->inet_dport &&
+		    r->id.idiag_dport)
+			continue;
+
+		if (sk_diag_dump(sk, skb, cb, r, bc, net_admin) < 0) {
+			mptcp_token_iter_stop(cb->args[0]);
+
+			/* will retry on the same position */
+			cb->args[1]--;
+			break;
+		}
+	}
+}
+
+static void mptcp_diag_get_info(struct sock *sk, struct inet_diag_msg *r,
+				void *_info)
+{
+	struct mptcp_sock *msk = mptcp_sk(sk);
+	struct mptcp_info *info = _info;
+	u32 flags = 0;
+	u8 val;
+
+	r->idiag_rqueue = sk_rmem_alloc_get(sk);
+	r->idiag_wqueue = sk_wmem_alloc_get(sk);
+	if (!info)
+		return;
+
+	info->mptcpi_subflows = READ_ONCE(msk->pm.subflows);
+	info->mptcpi_add_addr_signal = READ_ONCE(msk->pm.add_addr_signaled);
+	info->mptcpi_add_addr_accepted = READ_ONCE(msk->pm.add_addr_accepted);
+	info->mptcpi_subflows_max = READ_ONCE(msk->pm.subflows_max);
+	val = READ_ONCE(msk->pm.add_addr_signal_max);
+	info->mptcpi_add_addr_signal_max = val;
+	val = READ_ONCE(msk->pm.add_addr_accept_max);

net/mptcp/mptcp_diag.c line 143

Quoted 10 lines
net/mptcp/mptcp_diag.c new file +10 -0
+	info->mptcpi_add_addr_accepted_max = val;
+	if (test_bit(MPTCP_FALLBACK_DONE, &msk->flags))
+		flags |= MPTCP_INFO_FLAG_FALLBACK;
+	if (READ_ONCE(msk->can_ack))
+		flags |= MPTCP_INFO_FLAG_REMOTE_KEY_RECEIVED;
+	info->mptcpi_flags = flags;
+	info->mptcpi_token = READ_ONCE(msk->token);
+	info->mptcpi_write_seq = READ_ONCE(msk->write_seq);
+	info->mptcpi_snd_una = atomic64_read(&msk->snd_una);
+	info->mptcpi_rcv_nxt = READ_ONCE(msk->ack_seq);

I noticed that tcp_get_info() uses lock_sock_fast() when it populates the tcp_info struct, is it worth using that here? It seems like the sequence numbers in particular could easily be inconsistent without locking.

Mat

Quoted 26 lines
net/mptcp/mptcp_diag.c new file +24 -0
+}
+
+static const struct inet_diag_handler mptcp_diag_handler = {
+	.dump		 = mptcp_diag_dump,
+	.dump_one	 = mptcp_diag_dump_one,
+	.idiag_get_info  = mptcp_diag_get_info,
+	.idiag_type	 = IPPROTO_MPTCP,
+	.idiag_info_size = sizeof(struct mptcp_info),
+};
+
+int __init mptcp_diag_init(void)
+{
+	return inet_diag_register(&mptcp_diag_handler);
+}
+
+static void __exit mptcp_diag_exit(void)
+{
+	inet_diag_unregister(&mptcp_diag_handler);
+}
+
+module_init(mptcp_diag_init);
+module_exit(mptcp_diag_exit);
+MODULE_LICENSE("GPL");
+MODULE_ALIAS_NET_PF_PROTO_TYPE(PF_NETLINK, NETLINK_SOCK_DIAG, 2-262 /* AF_INET - IPPROTO_MPTCP */);
Signature · 1 line
2.26.2

--
Mat Martineau
Intel

author
#6 lore raw

[MPTCP] Re: [PATCH v2 3/4] mptcp: add MPTCP socket diag interface

Quoted 158 lines · Mat Martineau
Quoted 156 lines · Paolo Abeni

exposes basic inet socket attribute, plus some MPTCP socket fields comprising PM status and MPTCP-level sequence numbers.

Signed-off-by Paolo Abeni <pabeni(a)redhat.com>

---
v1 -> v2:
- fix dump on skb full event
- use 'flags' for fallback, can_ack
---

include/uapi/linux/mptcp.h |  17 ++++
net/mptcp/Kconfig          |   4 +
net/mptcp/Makefile         |   2 +
net/mptcp/mptcp_diag.c     | 167 +++++++++++++++++++++++++++++++++++++
4 files changed, 190 insertions(+)

create mode 100644 net/mptcp/mptcp_diag.c

include/uapi/linux/mptcp.h +17 -0
index 5f2c77082d9e..9762660df741 100644
@@ -86,4 +86,21 @@ enum {
 	__MPTCP_PM_CMD_AFTER_LAST
 };
 
+#define MPTCP_INFO_FLAG_FALLBACK		_BITUL(0)
+#define MPTCP_INFO_FLAG_REMOTE_KEY_RECEIVED	_BITUL(1)
+
+struct mptcp_info {
+	__u8	mptcpi_subflows;
+	__u8	mptcpi_add_addr_signal;
+	__u8	mptcpi_add_addr_accepted;
+	__u8	mptcpi_subflows_max;
+	__u8	mptcpi_add_addr_signal_max;
+	__u8	mptcpi_add_addr_accepted_max;
+	__u32	mptcpi_flags;
+	__u32	mptcpi_token;
+	__u64	mptcpi_write_seq;
+	__u64	mptcpi_snd_una;
+	__u64	mptcpi_rcv_nxt;
+};
+
 #endif /* _UAPI_MPTCP_H */
net/mptcp/Kconfig +0 -0
index 0ec787d8b9a2..97c29caa65f3 100644
@@ -14,6 +14,10 @@ config MPTCP
 

if MPTCP

net/mptcp/Kconfig +4 -0
+config INET_MPTCP_DIAG
+	depends on INET_DIAG
+	def_tristate INET_DIAG
+

config MPTCP_IPV6
bool "MPTCP: IPv6 support for Multipath TCP" select IPV6

net/mptcp/Makefile +2 -0
index c53f9b845523..2360cbd27d59 100644
@@ -4,6 +4,8 @@ obj-$(CONFIG_MPTCP) += mptcp.o
 mptcp-y := protocol.o subflow.o options.o token.o crypto.o ctrl.o pm.o diag.o \
 	   mib.o pm_netlink.o
 
+obj-$(CONFIG_INET_MPTCP_DIAG) += mptcp_diag.o
+

mptcp_crypto_test-objs := crypto_test.o
mptcp_token_test-objs := token_test.o obj-$(CONFIG_MPTCP_KUNIT_TESTS) += mptcp_crypto_test.o mptcp_token_test.o

net/mptcp/mptcp_diag.c new file +80 -0
new file mode 100644
index 000000000000..30aa768e23bd
@@ -0,0 +1,167 @@
+// SPDX-License-Identifier: GPL-2.0
+/* MPTCP socket monitoring support
+ *
+ * Copyright (c) 2020 Red Hat
+ *
+ * Author: Paolo Abeni <pabeni(a)redhat.com>
+ */
+
+#include <linux/kernel.h>
+#include <linux/net.h>
+#include <linux/inet_diag.h>
+#include <net/netlink.h>
+#include <uapi/linux/mptcp.h>
+#include "protocol.h"
+
+static int sk_diag_dump(struct sock *sk, struct sk_buff *skb,
+			struct netlink_callback *cb,
+			const struct inet_diag_req_v2 *req,
+			struct nlattr *bc, bool net_admin)
+{
+	if (!inet_diag_bc_sk(bc, sk))
+		return 0;
+
+	return inet_sk_diag_fill(sk, inet_csk(sk), skb, cb, req, NLM_F_MULTI,
+				 net_admin);
+}
+
+static int mptcp_diag_dump_one(struct netlink_callback *cb,
+			       const struct inet_diag_req_v2 *req)
+{
+	struct sk_buff *in_skb = cb->skb;
+	struct mptcp_sock *msk = NULL;
+	struct sk_buff *rep;
+	int err = -ENOENT;
+	struct net *net;
+	struct sock *sk;
+
+	net = sock_net(in_skb->sk);
+	msk = mptcp_token_get_sock(req->id.idiag_cookie[0]);
+	if (!msk)
+		goto out_nosk;
+
+	err = -ENOMEM;
+	sk = (struct sock *)msk;
+	rep = nlmsg_new(nla_total_size(sizeof(struct inet_diag_msg)) +
+			inet_diag_msg_attrs_size() +
+			nla_total_size(sizeof(struct mptcp_info)) +
+			nla_total_size(sizeof(struct inet_diag_meminfo)) + 64,
+			GFP_KERNEL);
+	if (!rep)
+		goto out;
+
+	err = inet_sk_diag_fill(sk, inet_csk(sk), rep, cb, req, 0,
+				netlink_net_capable(in_skb, CAP_NET_ADMIN));
+	if (err < 0) {
+		WARN_ON(err == -EMSGSIZE);
+		kfree_skb(rep);
+		goto out;
+	}
+	err = netlink_unicast(net->diag_nlsk, rep, NETLINK_CB(in_skb).portid,
+			      MSG_DONTWAIT);
+	if (err > 0)
+		err = 0;
+out:
+	sock_put(sk);
+
+out_nosk:
+	return err;
+}
+
+static void mptcp_diag_dump(struct sk_buff *skb, struct netlink_callback *cb,
+			    const struct inet_diag_req_v2 *r)
+{
+	bool net_admin = netlink_net_capable(cb->skb, CAP_NET_ADMIN);
+	struct net *net = sock_net(skb->sk);
+	struct inet_diag_dump_data *cb_data;
+	struct mptcp_sock *msk;
+	struct nlattr *bc;
+
+	cb_data = cb->data;
Quoted 8 lines

net/mptcp/mptcp_diag.c line 84

Quoted 4 lines
net/mptcp/mptcp_diag.c new file +4 -0
+	bc = cb_data->inet_diag_nla_bc;
+
+	for (msk = mptcp_token_iter_start(&cb->args[0], &cb->args[1]);
+	     msk; msk = mptcp_token_iter_next(&cb->args[0], &cb->args[1])) {

Should there be a rcu_read_lock/rcu_read_unlock around this loop with mptcp_token_iter_next() using sk_nulls_for_each_rcu()? Or is that already handled somewhere up the call stack?

This code tried to follow the same schema used for inet_hashtable(): it acquires explicit per bucket spinlock and keep it held while dumping the socket. Could be converted to RCU, but then we will get sock_hold()/sock_put() for each entry and I was under the impression that would be more overhead on busy servers with many mptcp sockets, but...

Quoted 55 lines
Quoted 55 lines
net/mptcp/mptcp_diag.c new file +55 -0
+		struct inet_sock *inet;
+		struct sock *sk;
+
+		inet = (struct inet_sock *)msk;
+		sk = (struct sock *)msk;
+		if (!net_eq(sock_net(sk), net))
+			continue;
+		if (!(r->idiag_states & (1 << sk->sk_state)))
+			continue;
+		if (r->sdiag_family != AF_UNSPEC &&
+		    sk->sk_family != r->sdiag_family)
+			continue;
+		if (r->id.idiag_sport != inet->inet_sport &&
+		    r->id.idiag_sport)
+			continue;
+		if (r->id.idiag_dport != inet->inet_dport &&
+		    r->id.idiag_dport)
+			continue;
+
+		if (sk_diag_dump(sk, skb, cb, r, bc, net_admin) < 0) {
+			mptcp_token_iter_stop(cb->args[0]);
+
+			/* will retry on the same position */
+			cb->args[1]--;
+			break;
+		}
+	}
+}
+
+static void mptcp_diag_get_info(struct sock *sk, struct inet_diag_msg *r,
+				void *_info)
+{
+	struct mptcp_sock *msk = mptcp_sk(sk);
+	struct mptcp_info *info = _info;
+	u32 flags = 0;
+	u8 val;
+
+	r->idiag_rqueue = sk_rmem_alloc_get(sk);
+	r->idiag_wqueue = sk_wmem_alloc_get(sk);
+	if (!info)
+		return;
+
+	info->mptcpi_subflows = READ_ONCE(msk->pm.subflows);
+	info->mptcpi_add_addr_signal = READ_ONCE(msk->pm.add_addr_signaled);
+	info->mptcpi_add_addr_accepted = READ_ONCE(msk->pm.add_addr_accepted);
+	info->mptcpi_subflows_max = READ_ONCE(msk->pm.subflows_max);
+	val = READ_ONCE(msk->pm.add_addr_signal_max);
+	info->mptcpi_add_addr_signal_max = val;
+	val = READ_ONCE(msk->pm.add_addr_accept_max);
+	info->mptcpi_add_addr_accepted_max = val;
+	if (test_bit(MPTCP_FALLBACK_DONE, &msk->flags))
+		flags |= MPTCP_INFO_FLAG_FALLBACK;
+	if (READ_ONCE(msk->can_ack))
+		flags |= MPTCP_INFO_FLAG_REMOTE_KEY_RECEIVED;
+	info->mptcpi_flags = flags;
Quoted 8 lines

net/mptcp/mptcp_diag.c line 143

Quoted 4 lines
net/mptcp/mptcp_diag.c new file +4 -0
+	info->mptcpi_token = READ_ONCE(msk->token);
+	info->mptcpi_write_seq = READ_ONCE(msk->write_seq);
+	info->mptcpi_snd_una = atomic64_read(&msk->snd_una);
+	info->mptcpi_rcv_nxt = READ_ONCE(msk->ack_seq);

I noticed that tcp_get_info() uses lock_sock_fast() when it populates the tcp_info struct, is it worth using that here? It seems like the sequence numbers in particular could easily be inconsistent without locking.

... thanks for the pointer! I misread some of the inet_hashtable diag related code (specifically, I mixed-up LISTEN and ESTABLISHED sockets traversing). Having a better look, TCP does per established socket sock_hold()/sock_lock_fast()/sock_unlock_fast()/sock_put(), so I guess we can do the same and move to RCUs for traversing. I'll try to cook v3 soon.

Thanks!

Paolo

author
#7 lore raw

[MPTCP] Re: [PATCH v2 3/4] mptcp: add MPTCP socket diag interface

Quoted 239 lines · Paolo Abeni
Quoted 158 lines · Mat Martineau
Quoted 156 lines · Paolo Abeni

exposes basic inet socket attribute, plus some MPTCP socket fields comprising PM status and MPTCP-level sequence numbers.

Signed-off-by Paolo Abeni <pabeni(a)redhat.com>

---
v1 -> v2:
- fix dump on skb full event
- use 'flags' for fallback, can_ack
---

include/uapi/linux/mptcp.h |  17 ++++
net/mptcp/Kconfig          |   4 +
net/mptcp/Makefile         |   2 +
net/mptcp/mptcp_diag.c     | 167 +++++++++++++++++++++++++++++++++++++
4 files changed, 190 insertions(+)

create mode 100644 net/mptcp/mptcp_diag.c

include/uapi/linux/mptcp.h +17 -0
index 5f2c77082d9e..9762660df741 100644
@@ -86,4 +86,21 @@ enum {
 	__MPTCP_PM_CMD_AFTER_LAST
 };
 
+#define MPTCP_INFO_FLAG_FALLBACK		_BITUL(0)
+#define MPTCP_INFO_FLAG_REMOTE_KEY_RECEIVED	_BITUL(1)
+
+struct mptcp_info {
+	__u8	mptcpi_subflows;
+	__u8	mptcpi_add_addr_signal;
+	__u8	mptcpi_add_addr_accepted;
+	__u8	mptcpi_subflows_max;
+	__u8	mptcpi_add_addr_signal_max;
+	__u8	mptcpi_add_addr_accepted_max;
+	__u32	mptcpi_flags;
+	__u32	mptcpi_token;
+	__u64	mptcpi_write_seq;
+	__u64	mptcpi_snd_una;
+	__u64	mptcpi_rcv_nxt;
+};
+
 #endif /* _UAPI_MPTCP_H */
net/mptcp/Kconfig +0 -0
index 0ec787d8b9a2..97c29caa65f3 100644
@@ -14,6 +14,10 @@ config MPTCP
 

if MPTCP

net/mptcp/Kconfig +4 -0
+config INET_MPTCP_DIAG
+	depends on INET_DIAG
+	def_tristate INET_DIAG
+

config MPTCP_IPV6
bool "MPTCP: IPv6 support for Multipath TCP" select IPV6

net/mptcp/Makefile +2 -0
index c53f9b845523..2360cbd27d59 100644
@@ -4,6 +4,8 @@ obj-$(CONFIG_MPTCP) += mptcp.o
 mptcp-y := protocol.o subflow.o options.o token.o crypto.o ctrl.o pm.o diag.o \
 	   mib.o pm_netlink.o
 
+obj-$(CONFIG_INET_MPTCP_DIAG) += mptcp_diag.o
+

mptcp_crypto_test-objs := crypto_test.o
mptcp_token_test-objs := token_test.o obj-$(CONFIG_MPTCP_KUNIT_TESTS) += mptcp_crypto_test.o mptcp_token_test.o

net/mptcp/mptcp_diag.c new file +80 -0
new file mode 100644
index 000000000000..30aa768e23bd
@@ -0,0 +1,167 @@
+// SPDX-License-Identifier: GPL-2.0
+/* MPTCP socket monitoring support
+ *
+ * Copyright (c) 2020 Red Hat
+ *
+ * Author: Paolo Abeni <pabeni(a)redhat.com>
+ */
+
+#include <linux/kernel.h>
+#include <linux/net.h>
+#include <linux/inet_diag.h>
+#include <net/netlink.h>
+#include <uapi/linux/mptcp.h>
+#include "protocol.h"
+
+static int sk_diag_dump(struct sock *sk, struct sk_buff *skb,
+			struct netlink_callback *cb,
+			const struct inet_diag_req_v2 *req,
+			struct nlattr *bc, bool net_admin)
+{
+	if (!inet_diag_bc_sk(bc, sk))
+		return 0;
+
+	return inet_sk_diag_fill(sk, inet_csk(sk), skb, cb, req, NLM_F_MULTI,
+				 net_admin);
+}
+
+static int mptcp_diag_dump_one(struct netlink_callback *cb,
+			       const struct inet_diag_req_v2 *req)
+{
+	struct sk_buff *in_skb = cb->skb;
+	struct mptcp_sock *msk = NULL;
+	struct sk_buff *rep;
+	int err = -ENOENT;
+	struct net *net;
+	struct sock *sk;
+
+	net = sock_net(in_skb->sk);
+	msk = mptcp_token_get_sock(req->id.idiag_cookie[0]);
+	if (!msk)
+		goto out_nosk;
+
+	err = -ENOMEM;
+	sk = (struct sock *)msk;
+	rep = nlmsg_new(nla_total_size(sizeof(struct inet_diag_msg)) +
+			inet_diag_msg_attrs_size() +
+			nla_total_size(sizeof(struct mptcp_info)) +
+			nla_total_size(sizeof(struct inet_diag_meminfo)) + 64,
+			GFP_KERNEL);
+	if (!rep)
+		goto out;
+
+	err = inet_sk_diag_fill(sk, inet_csk(sk), rep, cb, req, 0,
+				netlink_net_capable(in_skb, CAP_NET_ADMIN));
+	if (err < 0) {
+		WARN_ON(err == -EMSGSIZE);
+		kfree_skb(rep);
+		goto out;
+	}
+	err = netlink_unicast(net->diag_nlsk, rep, NETLINK_CB(in_skb).portid,
+			      MSG_DONTWAIT);
+	if (err > 0)
+		err = 0;
+out:
+	sock_put(sk);
+
+out_nosk:
+	return err;
+}
+
+static void mptcp_diag_dump(struct sk_buff *skb, struct netlink_callback *cb,
+			    const struct inet_diag_req_v2 *r)
+{
+	bool net_admin = netlink_net_capable(cb->skb, CAP_NET_ADMIN);
+	struct net *net = sock_net(skb->sk);
+	struct inet_diag_dump_data *cb_data;
+	struct mptcp_sock *msk;
+	struct nlattr *bc;
+
+	cb_data = cb->data;
Quoted 8 lines

net/mptcp/mptcp_diag.c line 84

Quoted 4 lines
net/mptcp/mptcp_diag.c new file +4 -0
+	bc = cb_data->inet_diag_nla_bc;
+
+	for (msk = mptcp_token_iter_start(&cb->args[0], &cb->args[1]);
+	     msk; msk = mptcp_token_iter_next(&cb->args[0], &cb->args[1])) {

Should there be a rcu_read_lock/rcu_read_unlock around this loop with mptcp_token_iter_next() using sk_nulls_for_each_rcu()? Or is that already handled somewhere up the call stack?

This code tried to follow the same schema used for inet_hashtable(): it acquires explicit per bucket spinlock and keep it held while dumping the socket. Could be converted to RCU, but then we will get sock_hold()/sock_put() for each entry and I was under the impression that would be more overhead on busy servers with many mptcp sockets, but...

Quoted 55 lines
Quoted 55 lines
net/mptcp/mptcp_diag.c new file +55 -0
+		struct inet_sock *inet;
+		struct sock *sk;
+
+		inet = (struct inet_sock *)msk;
+		sk = (struct sock *)msk;
+		if (!net_eq(sock_net(sk), net))
+			continue;
+		if (!(r->idiag_states & (1 << sk->sk_state)))
+			continue;
+		if (r->sdiag_family != AF_UNSPEC &&
+		    sk->sk_family != r->sdiag_family)
+			continue;
+		if (r->id.idiag_sport != inet->inet_sport &&
+		    r->id.idiag_sport)
+			continue;
+		if (r->id.idiag_dport != inet->inet_dport &&
+		    r->id.idiag_dport)
+			continue;
+
+		if (sk_diag_dump(sk, skb, cb, r, bc, net_admin) < 0) {
+			mptcp_token_iter_stop(cb->args[0]);
+
+			/* will retry on the same position */
+			cb->args[1]--;
+			break;
+		}
+	}
+}
+
+static void mptcp_diag_get_info(struct sock *sk, struct inet_diag_msg *r,
+				void *_info)
+{
+	struct mptcp_sock *msk = mptcp_sk(sk);
+	struct mptcp_info *info = _info;
+	u32 flags = 0;
+	u8 val;
+
+	r->idiag_rqueue = sk_rmem_alloc_get(sk);
+	r->idiag_wqueue = sk_wmem_alloc_get(sk);
+	if (!info)
+		return;
+
+	info->mptcpi_subflows = READ_ONCE(msk->pm.subflows);
+	info->mptcpi_add_addr_signal = READ_ONCE(msk->pm.add_addr_signaled);
+	info->mptcpi_add_addr_accepted = READ_ONCE(msk->pm.add_addr_accepted);
+	info->mptcpi_subflows_max = READ_ONCE(msk->pm.subflows_max);
+	val = READ_ONCE(msk->pm.add_addr_signal_max);
+	info->mptcpi_add_addr_signal_max = val;
+	val = READ_ONCE(msk->pm.add_addr_accept_max);
+	info->mptcpi_add_addr_accepted_max = val;
+	if (test_bit(MPTCP_FALLBACK_DONE, &msk->flags))
+		flags |= MPTCP_INFO_FLAG_FALLBACK;
+	if (READ_ONCE(msk->can_ack))
+		flags |= MPTCP_INFO_FLAG_REMOTE_KEY_RECEIVED;
+	info->mptcpi_flags = flags;
Quoted 8 lines

net/mptcp/mptcp_diag.c line 143

Quoted 4 lines
net/mptcp/mptcp_diag.c new file +4 -0
+	info->mptcpi_token = READ_ONCE(msk->token);
+	info->mptcpi_write_seq = READ_ONCE(msk->write_seq);
+	info->mptcpi_snd_una = atomic64_read(&msk->snd_una);
+	info->mptcpi_rcv_nxt = READ_ONCE(msk->ack_seq);

I noticed that tcp_get_info() uses lock_sock_fast() when it populates the tcp_info struct, is it worth using that here? It seems like the sequence numbers in particular could easily be inconsistent without locking.

Quoted 6 lines

... thanks for the pointer! I misread some of the inet_hashtable diag related code (specifically, I mixed-up LISTEN and ESTABLISHED sockets traversing). Having a better look, TCP does per established socket sock_hold()/sock_lock_fast()/sock_unlock_fast()/sock_put(), so I guess we can do the same and move to RCUs for traversing. I'll try to cook v3 soon.

Reporting here things already shared on IRC for larger audience e better trackability

An RCU based msk diag implementation does not fit well with lock_sock[_fast]() as we have to release the RCU lock on each socket.

We will have the same constraint even with bucket lock - as in the current implementation - to avoid possible issue due to lock order.

inet_diag_dump_icsk() tries to amortize the cost of releasing/acquiring the lock for each collecting up to 16 sockets in a local array and the filling the info individually.

That add a bit of complexity and also do not fit well with iterator: the iterator helper will have to implement the filtering as required by the diag interface (which looks like a layering violation).

The simpler solution I can think of is avoiding the lock_sock() in mptcp_diag_get_info(). that means possible inconsistence between write_seq and snd_una. e.g. snd_una could be greater/later then write_seq do we really care?

/P

#8 lore raw

[MPTCP] Re: [PATCH v2 3/4] mptcp: add MPTCP socket diag interface

Quoted 260 lines · Paolo Abeni
Quoted 239 lines · Paolo Abeni
Quoted 158 lines · Mat Martineau
Quoted 156 lines · Paolo Abeni

exposes basic inet socket attribute, plus some MPTCP socket fields comprising PM status and MPTCP-level sequence numbers.

Signed-off-by Paolo Abeni <pabeni(a)redhat.com>

---
v1 -> v2:
- fix dump on skb full event
- use 'flags' for fallback, can_ack
---

include/uapi/linux/mptcp.h |  17 ++++
net/mptcp/Kconfig          |   4 +
net/mptcp/Makefile         |   2 +
net/mptcp/mptcp_diag.c     | 167 +++++++++++++++++++++++++++++++++++++
4 files changed, 190 insertions(+)

create mode 100644 net/mptcp/mptcp_diag.c

include/uapi/linux/mptcp.h +17 -0
index 5f2c77082d9e..9762660df741 100644
@@ -86,4 +86,21 @@ enum {
 	__MPTCP_PM_CMD_AFTER_LAST
 };
 
+#define MPTCP_INFO_FLAG_FALLBACK		_BITUL(0)
+#define MPTCP_INFO_FLAG_REMOTE_KEY_RECEIVED	_BITUL(1)
+
+struct mptcp_info {
+	__u8	mptcpi_subflows;
+	__u8	mptcpi_add_addr_signal;
+	__u8	mptcpi_add_addr_accepted;
+	__u8	mptcpi_subflows_max;
+	__u8	mptcpi_add_addr_signal_max;
+	__u8	mptcpi_add_addr_accepted_max;
+	__u32	mptcpi_flags;
+	__u32	mptcpi_token;
+	__u64	mptcpi_write_seq;
+	__u64	mptcpi_snd_una;
+	__u64	mptcpi_rcv_nxt;
+};
+
 #endif /* _UAPI_MPTCP_H */
net/mptcp/Kconfig +0 -0
index 0ec787d8b9a2..97c29caa65f3 100644
@@ -14,6 +14,10 @@ config MPTCP
 

if MPTCP

net/mptcp/Kconfig +4 -0
+config INET_MPTCP_DIAG
+	depends on INET_DIAG
+	def_tristate INET_DIAG
+

config MPTCP_IPV6
bool "MPTCP: IPv6 support for Multipath TCP" select IPV6

net/mptcp/Makefile +2 -0
index c53f9b845523..2360cbd27d59 100644
@@ -4,6 +4,8 @@ obj-$(CONFIG_MPTCP) += mptcp.o
 mptcp-y := protocol.o subflow.o options.o token.o crypto.o ctrl.o pm.o diag.o \
 	   mib.o pm_netlink.o
 
+obj-$(CONFIG_INET_MPTCP_DIAG) += mptcp_diag.o
+

mptcp_crypto_test-objs := crypto_test.o
mptcp_token_test-objs := token_test.o obj-$(CONFIG_MPTCP_KUNIT_TESTS) += mptcp_crypto_test.o mptcp_token_test.o

net/mptcp/mptcp_diag.c new file +80 -0
new file mode 100644
index 000000000000..30aa768e23bd
@@ -0,0 +1,167 @@
+// SPDX-License-Identifier: GPL-2.0
+/* MPTCP socket monitoring support
+ *
+ * Copyright (c) 2020 Red Hat
+ *
+ * Author: Paolo Abeni <pabeni(a)redhat.com>
+ */
+
+#include <linux/kernel.h>
+#include <linux/net.h>
+#include <linux/inet_diag.h>
+#include <net/netlink.h>
+#include <uapi/linux/mptcp.h>
+#include "protocol.h"
+
+static int sk_diag_dump(struct sock *sk, struct sk_buff *skb,
+			struct netlink_callback *cb,
+			const struct inet_diag_req_v2 *req,
+			struct nlattr *bc, bool net_admin)
+{
+	if (!inet_diag_bc_sk(bc, sk))
+		return 0;
+
+	return inet_sk_diag_fill(sk, inet_csk(sk), skb, cb, req, NLM_F_MULTI,
+				 net_admin);
+}
+
+static int mptcp_diag_dump_one(struct netlink_callback *cb,
+			       const struct inet_diag_req_v2 *req)
+{
+	struct sk_buff *in_skb = cb->skb;
+	struct mptcp_sock *msk = NULL;
+	struct sk_buff *rep;
+	int err = -ENOENT;
+	struct net *net;
+	struct sock *sk;
+
+	net = sock_net(in_skb->sk);
+	msk = mptcp_token_get_sock(req->id.idiag_cookie[0]);
+	if (!msk)
+		goto out_nosk;
+
+	err = -ENOMEM;
+	sk = (struct sock *)msk;
+	rep = nlmsg_new(nla_total_size(sizeof(struct inet_diag_msg)) +
+			inet_diag_msg_attrs_size() +
+			nla_total_size(sizeof(struct mptcp_info)) +
+			nla_total_size(sizeof(struct inet_diag_meminfo)) + 64,
+			GFP_KERNEL);
+	if (!rep)
+		goto out;
+
+	err = inet_sk_diag_fill(sk, inet_csk(sk), rep, cb, req, 0,
+				netlink_net_capable(in_skb, CAP_NET_ADMIN));
+	if (err < 0) {
+		WARN_ON(err == -EMSGSIZE);
+		kfree_skb(rep);
+		goto out;
+	}
+	err = netlink_unicast(net->diag_nlsk, rep, NETLINK_CB(in_skb).portid,
+			      MSG_DONTWAIT);
+	if (err > 0)
+		err = 0;
+out:
+	sock_put(sk);
+
+out_nosk:
+	return err;
+}
+
+static void mptcp_diag_dump(struct sk_buff *skb, struct netlink_callback *cb,
+			    const struct inet_diag_req_v2 *r)
+{
+	bool net_admin = netlink_net_capable(cb->skb, CAP_NET_ADMIN);
+	struct net *net = sock_net(skb->sk);
+	struct inet_diag_dump_data *cb_data;
+	struct mptcp_sock *msk;
+	struct nlattr *bc;
+
+	cb_data = cb->data;
Quoted 8 lines

net/mptcp/mptcp_diag.c line 84

Quoted 4 lines
net/mptcp/mptcp_diag.c new file +4 -0
+	bc = cb_data->inet_diag_nla_bc;
+
+	for (msk = mptcp_token_iter_start(&cb->args[0], &cb->args[1]);
+	     msk; msk = mptcp_token_iter_next(&cb->args[0], &cb->args[1])) {

Should there be a rcu_read_lock/rcu_read_unlock around this loop with mptcp_token_iter_next() using sk_nulls_for_each_rcu()? Or is that already handled somewhere up the call stack?

This code tried to follow the same schema used for inet_hashtable(): it acquires explicit per bucket spinlock and keep it held while dumping the socket. Could be converted to RCU, but then we will get sock_hold()/sock_put() for each entry and I was under the impression that would be more overhead on busy servers with many mptcp sockets, but...

Quoted 55 lines
Quoted 55 lines
net/mptcp/mptcp_diag.c new file +55 -0
+		struct inet_sock *inet;
+		struct sock *sk;
+
+		inet = (struct inet_sock *)msk;
+		sk = (struct sock *)msk;
+		if (!net_eq(sock_net(sk), net))
+			continue;
+		if (!(r->idiag_states & (1 << sk->sk_state)))
+			continue;
+		if (r->sdiag_family != AF_UNSPEC &&
+		    sk->sk_family != r->sdiag_family)
+			continue;
+		if (r->id.idiag_sport != inet->inet_sport &&
+		    r->id.idiag_sport)
+			continue;
+		if (r->id.idiag_dport != inet->inet_dport &&
+		    r->id.idiag_dport)
+			continue;
+
+		if (sk_diag_dump(sk, skb, cb, r, bc, net_admin) < 0) {
+			mptcp_token_iter_stop(cb->args[0]);
+
+			/* will retry on the same position */
+			cb->args[1]--;
+			break;
+		}
+	}
+}
+
+static void mptcp_diag_get_info(struct sock *sk, struct inet_diag_msg *r,
+				void *_info)
+{
+	struct mptcp_sock *msk = mptcp_sk(sk);
+	struct mptcp_info *info = _info;
+	u32 flags = 0;
+	u8 val;
+
+	r->idiag_rqueue = sk_rmem_alloc_get(sk);
+	r->idiag_wqueue = sk_wmem_alloc_get(sk);
+	if (!info)
+		return;
+
+	info->mptcpi_subflows = READ_ONCE(msk->pm.subflows);
+	info->mptcpi_add_addr_signal = READ_ONCE(msk->pm.add_addr_signaled);
+	info->mptcpi_add_addr_accepted = READ_ONCE(msk->pm.add_addr_accepted);
+	info->mptcpi_subflows_max = READ_ONCE(msk->pm.subflows_max);
+	val = READ_ONCE(msk->pm.add_addr_signal_max);
+	info->mptcpi_add_addr_signal_max = val;
+	val = READ_ONCE(msk->pm.add_addr_accept_max);
+	info->mptcpi_add_addr_accepted_max = val;
+	if (test_bit(MPTCP_FALLBACK_DONE, &msk->flags))
+		flags |= MPTCP_INFO_FLAG_FALLBACK;
+	if (READ_ONCE(msk->can_ack))
+		flags |= MPTCP_INFO_FLAG_REMOTE_KEY_RECEIVED;
+	info->mptcpi_flags = flags;
Quoted 8 lines

net/mptcp/mptcp_diag.c line 143

Quoted 4 lines
net/mptcp/mptcp_diag.c new file +4 -0
+	info->mptcpi_token = READ_ONCE(msk->token);
+	info->mptcpi_write_seq = READ_ONCE(msk->write_seq);
+	info->mptcpi_snd_una = atomic64_read(&msk->snd_una);
+	info->mptcpi_rcv_nxt = READ_ONCE(msk->ack_seq);

I noticed that tcp_get_info() uses lock_sock_fast() when it populates the tcp_info struct, is it worth using that here? It seems like the sequence numbers in particular could easily be inconsistent without locking.

Quoted 6 lines

... thanks for the pointer! I misread some of the inet_hashtable diag related code (specifically, I mixed-up LISTEN and ESTABLISHED sockets traversing). Having a better look, TCP does per established socket sock_hold()/sock_lock_fast()/sock_unlock_fast()/sock_put(), so I guess we can do the same and move to RCUs for traversing. I'll try to cook v3 soon.

Reporting here things already shared on IRC for larger audience e better trackability

An RCU based msk diag implementation does not fit well with lock_sock[_fast]() as we have to release the RCU lock on each socket.

We will have the same constraint even with bucket lock - as in the current implementation - to avoid possible issue due to lock order.

inet_diag_dump_icsk() tries to amortize the cost of releasing/acquiring the lock for each collecting up to 16 sockets in a local array and the filling the info individually.

Quoted 8 lines

That add a bit of complexity and also do not fit well with iterator: the iterator helper will have to implement the filtering as required by the diag interface (which looks like a layering violation).

The simpler solution I can think of is avoiding the lock_sock() in mptcp_diag_get_info(). that means possible inconsistence between write_seq and snd_una. e.g. snd_una could be greater/later then write_seq do we really care?

And for further IRC capture:

Regarding the lock_sock question, we can probably minimize the inconsistencies by ordering the reads. For example, if we read snd_una before write_seq, then snd_una wouldn't show as later. As someone working on the protocol it's helpful to have a coherent snapshot of the values but I don't want to add too much complexity to guarantee that.

So, I'm ok with reordering some of the reads and avoiding the additional locking. Still interested if others have comments!

--
Mat Martineau
Intel

#2 lore raw

[MPTCP] Re: [PATCH v2 1/4] inet_diag: support for wider protocol numbers

In reply to Paolo Abeni
Quoted 54 lines · Paolo Abeni

After commit bf9765145b85 ("sock: Make sk_protocol a 16-bit value") the current size of 'sdiag_protocol' is not sufficient to represent the possible protocol values.

This change introduces a new inet diag request attribute to let user space specify the relevant protocol number using u32 values.

The attribute is parsed by inet diag core on get/dump command and the extended protocol value, if available, is preferred to 'sdiag_protocol' to lookup the diag handler.

The parse attributed are exposed to all the diag handlers via the cb->data.

Note that inet_diag_dump_one_icsk() is left unmodified, as it will not be used by protocol using the extended attribute.

Suggested-by David S. Miller <davem(a)davemloft.net>
Signed-off-by Paolo Abeni <pabeni(a)redhat.com>
---
 include/uapi/linux/inet_diag.h |  1 +
 net/core/sock.c                |  1 +
 net/ipv4/inet_diag.c           | 63 +++++++++++++++++++++++++---------
 3 files changed, 48 insertions(+), 17 deletions(-)
include/uapi/linux/inet_diag.h +1 -0
index e6f183ee8417..5ba122c1949a 100644
@@ -65,6 +65,7 @@ enum {
 	INET_DIAG_REQ_NONE,
 	INET_DIAG_REQ_BYTECODE,
 	INET_DIAG_REQ_SK_BPF_STORAGES,
+	INET_DIAG_REQ_PROTOCOL,
 	__INET_DIAG_REQ_MAX,
 };
 
net/core/sock.c +1 -0
index 6c4acf1f0220..b359e29bb737 100644
@@ -3573,6 +3573,7 @@ int sock_load_diag_module(int family, int protocol)
 #ifdef CONFIG_INET
 	if (family == AF_INET &&
 	    protocol != IPPROTO_RAW &&
+	    protocol < MAX_INET_PROTOS &&
 	    !rcu_access_pointer(inet_protos[protocol]))
 		return -ENOENT;
 #endif
net/ipv4/inet_diag.c +0 -0
index 125f4f8a36b4..a3a7ef3fada6 100644

net/ipv4/inet_diag.c static DEFINE_MUTEX(inet_diag_table_mutex); · line 56

Quoted 5 lines
net/ipv4/inet_diag.c +2 -0
@@ -52,6 +52,9 @@ static DEFINE_MUTEX(inet_diag_table_mutex);
 
 static const struct inet_diag_handler *inet_diag_lock_handler(int proto)
 {
+	if (proto >= IPPROTO_MAX)
+		return ERR_PTR(-ENOENT);

Just here by email so that it's not lost in IRC,... and so that it is picked up in patchwork:

Cfr. https://github.com/multipath-tcp/mptcp_net-next/issues/44

This needs to be if (proto < 0 || proto >= IPPROTO_MAX).

I tried that out and my syzkaller reproducers are no more panicking the system.

Christoph