Skip to contentSkip to conversation

mptcp: add MPTCP_INFO getsockopt

RFC In review Florian Westphal 3 participants 5 messages Jul 22, 2021 – Jul 23, 2021

Subsystems (3)
lore
Share & tools
lore
#0 lore raw

RFC 1/1 mptcp: add MPTCP_INFO getsockopt

Its not compatible with mptcp.org kernel one:
1. mptcp.org defines 'struct mptcp_info', but we already have this for inet_diag.

2. 'struct mptcp_info', as defined by mptcp.org has different layout for 32/64 bit arches.
This is a problem for 32bit binaries on 64bit kernels.

For those reasons a new 'struct mptcp_info_opt' is added which contains aligned_u64 fields to store the userspace buffer addresses.

'struct mptcp_sub_info' is added as per mptcp.org kernel, but I don't like this structure. I think 'src' and 'dst' are confusing terms.

AFAICS 'src' really is 'local address' (getsockname) and 'dst' is peer (getpeername).

Given mptcp-next has IPPROTO_MPTCP, this adds SOL_MPTCP. This also gives ample space to add mptcp specific sockopts.

In light of this, I wonder if it would make more sense to split the functionality.

Examples:

getsockopt(fd, SOL_MPTCP, SUBFLOW_ID, subflow_ids, &count);

sa.sa_family = subflow_ids[0];
getsockopt(fd, SOL_MPTCP, SUBFLOW_GETPEERNAME, &sa, &len); ..
sa.sa_family = subflow_ids[0];
getsockopt(fd, SOL_MPTCP, SUBFLOW_GETSOCKNAME, &sa, &len);

tcp_info.tcpi_state = subflow_ids[0]; getsockopt(fd, SOL_MPTCP, SUBFLOW_TCP_INFO, &tcp_info, &len);

...

and so on.

Alternatively one could overload e.g. the upper 8 bit:

getsockopt(fd, SOL_MPTCP, subflow_ids[0] << 24 | SUBFLOW_FOO, ...);

In any case, here is a tentative patch to add a mptcp.org-alike MPTCP_INFO getsockopt.

It depends on a tiny extra patch to move part of inet_diag to the core.

---
 include/linux/socket.h     |   1 +
 include/uapi/linux/mptcp.h |  30 ++++++
 net/mptcp/sockopt.c        | 197 +++++++++++++++++++++++++++++++++++++
 3 files changed, 228 insertions(+)
include/linux/socket.h +1 -0
index 0d8e3dcb7f88..af853eadadcb 100644
@@ -360,6 +360,7 @@ struct ucred {
 #define SOL_KCM		281
 #define SOL_TLS		282
 #define SOL_XDP		283
+#define SOL_MPTCP	284
 
 /* IPX options */
 #define IPX_TYPE	1
include/uapi/linux/mptcp.h +30 -0
index 7b05f7102321..35d90f08b376 100644
@@ -4,6 +4,7 @@
 
 #include <linux/const.h>
 #include <linux/types.h>
+#include <linux/in.h>
 
 #define MPTCP_SUBFLOW_FLAG_MCAP_REM		_BITUL(0)
 #define MPTCP_SUBFLOW_FLAG_MCAP_LOC		_BITUL(1)
@@ -192,4 +193,33 @@ enum mptcp_event_attr {
 #define MPTCP_RST_EBADPERF	5
 #define MPTCP_RST_EMIDDLEBOX	6
 
+struct mptcp_info_opt {
+	__u32		tcp_info_len;		/* Length of each struct tcp_info in subflows pointer */
+	__u32		sub_len;		/* Total length of memory pointed to by subflows pointer */
+	__u32		meta_info_len;		/* Length of memory pointed to by meta_info */
+	__u32		sub_info_len;		/* Length of each struct mptcp_sub_info in subflow_info pointer */
+	__u32		total_sub_info_len;	/* Total length of memory pointed to by subflow_info */
+
+	__aligned_u64	meta_info;
+	__aligned_u64	initial;
+	__aligned_u64	subflows;
+	__aligned_u64	subflow_info;		/* mptcp_sub_info[] */
+};
+
+struct mptcp_sub_info {
+	union {
+		struct sockaddr src;
+		struct sockaddr_in src_v4;
+		struct sockaddr_in6 src_v6;
+	};
+	union {
+		struct sockaddr dst;
+		struct sockaddr_in dst_v4;
+		struct sockaddr_in6 dst_v6;
+	};
+};
+
+/* MPTCP socket options */
+#define MPTCP_INFO 1
+
 #endif /* _UAPI_MPTCP_H */
net/mptcp/sockopt.c +197 -0

215 lines not on this page · raw message

Signature · 1 line
2.31.1
#1 lore raw

Hi Florian,

Thank you for looking at that!

Quoted 10 lines · Florian Westphal

Its not compatible with mptcp.org kernel one:
1. mptcp.org defines 'struct mptcp_info', but we already have this for inet_diag.

2. 'struct mptcp_info', as defined by mptcp.org has different layout for 32/64 bit arches.
This is a problem for 32bit binaries on 64bit kernels.

For those reasons a new 'struct mptcp_info_opt' is added which contains aligned_u64 fields to store the userspace buffer addresses.

Thank you for the explanation, it makes sense!

Quoted 2 lines

'struct mptcp_sub_info' is added as per mptcp.org kernel, but I don't like this structure. I think 'src' and 'dst' are confusing terms.

Do you prefer 'local' and 'remote'?
Good time to change!

Quoted 5 lines

AFAICS 'src' really is 'local address' (getsockname) and 'dst' is peer (getpeername).

Given mptcp-next has IPPROTO_MPTCP, this adds SOL_MPTCP. This also gives ample space to add mptcp specific sockopts.

Very good idea!

Quoted 2 lines

In light of this, I wonder if it would make more sense to split the functionality.

Would it not start to be a bit costly to get all details if the userspace has do a few syscalls?
- general info
- get subflow IDs
- for each suflow, call subflow info + peer/sock names

With the full structure, we could eventually allow userspace programs to set some "*_len" members to 0 not to get all info, e.g. only get the general info without peer/sock names for all subflows. Or without TCP_INFO for each subflow. No?

Quoted 11 lines

Examples:

getsockopt(fd, SOL_MPTCP, SUBFLOW_ID, subflow_ids, &count);

sa.sa_family = subflow_ids[0];
getsockopt(fd, SOL_MPTCP, SUBFLOW_GETPEERNAME, &sa, &len); ..
sa.sa_family = subflow_ids[0];
getsockopt(fd, SOL_MPTCP, SUBFLOW_GETSOCKNAME, &sa, &len);

Quoted 6 lines

tcp_info.tcpi_state = subflow_ids[0]; getsockopt(fd, SOL_MPTCP, SUBFLOW_TCP_INFO, &tcp_info, &len);

...

and so on.

This could be interesting if there is a demand to get info only for some specific subflows :)

Quoted 4 lines

Alternatively one could overload e.g. the upper 8 bit:

getsockopt(fd, SOL_MPTCP, subflow_ids[0] << 24 | SUBFLOW_FOO, ...);

Maybe a "cleaner" workaround to set the ID? :)

Quoted 2 lines

In any case, here is a tentative patch to add a mptcp.org-alike MPTCP_INFO getsockopt.

Thank you for sharing that!

(I didn't check the code yet)

Cheers,
Matt

Signature · 2 lines
Tessares | Belgium | Hybrid Access Solutions
www.tessares.net
#2 lore raw
Quoted 16 lines · Matthieu Baerts

Hi Florian,

Thank you for looking at that!

Quoted 10 lines · Florian Westphal

Its not compatible with mptcp.org kernel one:
1. mptcp.org defines 'struct mptcp_info', but we already have this for inet_diag.

2. 'struct mptcp_info', as defined by mptcp.org has different layout for 32/64 bit arches.
This is a problem for 32bit binaries on 64bit kernels.

For those reasons a new 'struct mptcp_info_opt' is added which contains aligned_u64 fields to store the userspace buffer addresses.

Quoted 6 lines

Thank you for the explanation, it makes sense!

Quoted 2 lines

'struct mptcp_sub_info' is added as per mptcp.org kernel, but I don't like this structure. I think 'src' and 'dst' are confusing terms.

Do you prefer 'local' and 'remote'?

I thought about alternative names, I like your suggestion.

Quoted 8 lines
Quoted 2 lines

In light of this, I wonder if it would make more sense to split the functionality.

Would it not start to be a bit costly to get all details if the userspace has do a few syscalls?
- general info
- get subflow IDs
- for each suflow, call subflow info + peer/sock names

Yes, good point. Its unlikely one would ask for a specific subflow endpoint addresses.

So perhaps just keep that in mind for later and go with a mptcp.org alike approach.

Quoted 4 lines

With the full structure, we could eventually allow userspace programs to set some "*_len" members to 0 not to get all info, e.g. only get the general info without peer/sock names for all subflows. Or without TCP_INFO for each subflow. No?

Yes, userspace can set the address field to 0 and the kernel will skip that part.

One thing that I'd like to change is the way the length fields are handled. I propose that userspace can set the legnth field to 0 to request the kernel to fill in the required size.

This won't work reliably (there could be a new subflow arriving after the query and before the second getsockopt with the "properly sized" fields.

At the moment, there is no way to detect this.
Perhaps the info struct should contain two size versions, one with the "this is how much I copied to the buffer" and another with "this is how much i would have copied".

Quoted 9 lines
Quoted 9 lines

sa.sa_family = subflow_ids[0];
getsockopt(fd, SOL_MPTCP, SUBFLOW_GETPEERNAME, &sa, &len); ..
sa.sa_family = subflow_ids[0];
getsockopt(fd, SOL_MPTCP, SUBFLOW_GETSOCKNAME, &sa, &len);

tcp_info.tcpi_state = subflow_ids[0]; getsockopt(fd, SOL_MPTCP, SUBFLOW_TCP_INFO, &tcp_info, &len);

Quoted 6 lines
Quoted 3 lines

...

and so on.

This could be interesting if there is a demand to get info only for some specific subflows :)

Yes, as I wrote above your are probably right that the "give me the peer socket address for subflow 42" is a bit esoteric.

Quoted 5 lines
Quoted 3 lines

Alternatively one could overload e.g. the upper 8 bit:

getsockopt(fd, SOL_MPTCP, subflow_ids[0] << 24 | SUBFLOW_FOO, ...);

Maybe a "cleaner" workaround to set the ID? :)

It would be more consistent, at least. The "piggyback in the buffer" is very ugly...

Quoted 6 lines
Quoted 2 lines

In any case, here is a tentative patch to add a mptcp.org-alike MPTCP_INFO getsockopt.

Thank you for sharing that!

(I didn't check the code yet)

No need to have a super close look, its going to change anyway. Its also heavily lifted from the mptcp.org kernel.

I will add attributions as needed once a formal patch submission is made.

maintainer
#3 lore raw
In reply to Florian Westphal
Quoted 7 lines · Florian Westphal

Its not compatible with mptcp.org kernel one:
1. mptcp.org defines 'struct mptcp_info', but we already have this for inet_diag.

2. 'struct mptcp_info', as defined by mptcp.org has different layout for 32/64 bit arches.
This is a problem for 32bit binaries on 64bit kernels.

I think the lack of binary compatibility with the OoO tree kernel is nto a big deal - and anyhow is unsolvable, so we have to consider it not a big deal ;)

I think it could *possibly* be nice to tie togethar mptcp_info and mptcp_info_opt - e.g. pulling the second into the first and unifying the handling, as tcp does for diag and GET_INFO. But perhpas is just too much code/overhead?!?

Quoted 8 lines

For those reasons a new 'struct mptcp_info_opt' is added which contains aligned_u64 fields to store the userspace buffer addresses.

'struct mptcp_sub_info' is added as per mptcp.org kernel, but I don't like this structure. I think 'src' and 'dst' are confusing terms.

AFAICS 'src' really is 'local address' (getsockname) and 'dst' is peer (getpeername).

+1 for 'local' and 'remote'

Quoted 16 lines

Given mptcp-next has IPPROTO_MPTCP, this adds SOL_MPTCP. This also gives ample space to add mptcp specific sockopts.

In light of this, I wonder if it would make more sense to split the functionality.

Examples:

getsockopt(fd, SOL_MPTCP, SUBFLOW_ID, subflow_ids, &count);

sa.sa_family = subflow_ids[0];
getsockopt(fd, SOL_MPTCP, SUBFLOW_GETPEERNAME, &sa, &len); ..
sa.sa_family = subflow_ids[0];
getsockopt(fd, SOL_MPTCP, SUBFLOW_GETSOCKNAME, &sa, &len);

Quoted 6 lines

tcp_info.tcpi_state = subflow_ids[0]; getsockopt(fd, SOL_MPTCP, SUBFLOW_TCP_INFO, &tcp_info, &len);

...

and so on.

+1 to fetch all the subflow info with a single syscall

+1 to pass back the user 2 sizes: one with
the "this is how much I copied to the buffer" and another with "this is how much i would have copied".

Quoted 3 lines

Alternatively one could overload e.g. the upper 8 bit:

getsockopt(fd, SOL_MPTCP, subflow_ids[0] << 24 | SUBFLOW_FOO, ...);

What about using:

struct mptcp_subflow_info {
	u32 	id;
	struct tcp_info info;
};

mptcp_subflow_info.id = id;
getsockopt(fd, SOL_MPTCP, &mptcp_subflow_info, ...);

?

Cheers,

Paolo

#4 lore raw
Quoted 1 line · Paolo Abeni

On Thu, 2021-07-22 at 17:01 +0200, Florian Westphal wrote:

(...)

Quoted 3 lines

+1 to pass back the user 2 sizes: one with
the "this is how much I copied to the buffer" and another with "this is how much i would have copied".

+1

Quoted 6 lines
Quoted 3 lines

Alternatively one could overload e.g. the upper 8 bit:

getsockopt(fd, SOL_MPTCP, subflow_ids[0] << 24 | SUBFLOW_FOO, ...);

What about using:

Quoted 7 lines
struct mptcp_subflow_info {
	u32 	id;
	struct tcp_info info;
};

mptcp_subflow_info.id = id;
getsockopt(fd, SOL_MPTCP, &mptcp_subflow_info, ...);

Indeed, clearer if we can do that :)

Cheers,
Matt

Signature · 2 lines
Tessares | Belgium | Hybrid Access Solutions
www.tessares.net