mptcp: add MPTCP_INFO getsockopt
Subsystems (3)
RFC 1/1 mptcp: add MPTCP_INFO getsockopt
Its not compatible with mptcp.org kernel one:
1. mptcp.org defines 'struct mptcp_info', but we already have this for inet_diag.
2. 'struct mptcp_info', as defined by mptcp.org has different layout for 32/64 bit arches.
This is a problem for 32bit binaries on 64bit kernels.
For those reasons a new 'struct mptcp_info_opt' is added which contains aligned_u64 fields to store the userspace buffer addresses.
'struct mptcp_sub_info' is added as per mptcp.org kernel, but I don't like this structure. I think 'src' and 'dst' are confusing terms.
AFAICS 'src' really is 'local address' (getsockname) and 'dst' is peer (getpeername).
Given mptcp-next has IPPROTO_MPTCP, this adds SOL_MPTCP. This also gives ample space to add mptcp specific sockopts.
In light of this, I wonder if it would make more sense to split the functionality.
Examples:
getsockopt(fd, SOL_MPTCP, SUBFLOW_ID, subflow_ids, &count);
sa.sa_family = subflow_ids[0];getsockopt(fd, SOL_MPTCP, SUBFLOW_GETPEERNAME, &sa, &len); ..
sa.sa_family = subflow_ids[0];getsockopt(fd, SOL_MPTCP, SUBFLOW_GETSOCKNAME, &sa, &len);
tcp_info.tcpi_state = subflow_ids[0]; getsockopt(fd, SOL_MPTCP, SUBFLOW_TCP_INFO, &tcp_info, &len);
...
and so on.
Alternatively one could overload e.g. the upper 8 bit:
getsockopt(fd, SOL_MPTCP, subflow_ids[0] << 24 | SUBFLOW_FOO, ...);
In any case, here is a tentative patch to add a mptcp.org-alike MPTCP_INFO getsockopt.
It depends on a tiny extra patch to move part of inet_diag to the core.
--- include/linux/socket.h | 1 + include/uapi/linux/mptcp.h | 30 ++++++ net/mptcp/sockopt.c | 197 +++++++++++++++++++++++++++++++++++++ 3 files changed, 228 insertions(+)
include/linux/socket.h +1 -0
@@ -360,6 +360,7 @@ struct ucred {#define SOL_KCM 281#define SOL_TLS 282#define SOL_XDP 283 +#define SOL_MPTCP 284/* IPX options */ #define IPX_TYPE 1
include/uapi/linux/mptcp.h +30 -0
@@ -4,6 +4,7 @@#include <linux/const.h> #include <linux/types.h> +#include <linux/in.h> #define MPTCP_SUBFLOW_FLAG_MCAP_REM _BITUL(0)#define MPTCP_SUBFLOW_FLAG_MCAP_LOC _BITUL(1)
@@ -192,4 +193,33 @@ enum mptcp_event_attr {#define MPTCP_RST_EBADPERF 5#define MPTCP_RST_EMIDDLEBOX 6 +struct mptcp_info_opt { +__u32 tcp_info_len;/* Length of each struct tcp_info in subflows pointer */ +__u32 sub_len;/* Total length of memory pointed to by subflows pointer */ +__u32 meta_info_len;/* Length of memory pointed to by meta_info */ +__u32 sub_info_len;/* Length of each struct mptcp_sub_info in subflow_info pointer */ +__u32 total_sub_info_len;/* Total length of memory pointed to by subflow_info */ + +__aligned_u64 meta_info; +__aligned_u64 initial; +__aligned_u64 subflows; +__aligned_u64 subflow_info;/* mptcp_sub_info[] */ +}; + +struct mptcp_sub_info { +union { +struct sockaddr src; +struct sockaddr_in src_v4; +struct sockaddr_in6 src_v6; + }; +union { +struct sockaddr dst; +struct sockaddr_in dst_v4; +struct sockaddr_in6 dst_v6; + }; +}; + +/* MPTCP socket options */ +#define MPTCP_INFO 1 +#endif /* _UAPI_MPTCP_H */
net/mptcp/sockopt.c +197 -0
215 lines not on this page · raw message
Signature · 1 line
2.31.1
Hi Florian,
Thank you for looking at that!
Quoted 10 lines · Florian Westphal
Its not compatible with mptcp.org kernel one:
1. mptcp.org defines 'structmptcp_info', but we already have this forinet_diag.2. 'struct
mptcp_info', as defined by mptcp.org has different layout for 32/64 bit arches.
This is a problem for 32bit binaries on 64bit kernels.For those reasons a new 'struct
mptcp_info_opt' is added which containsaligned_u64fields to store the userspace buffer addresses.
Thank you for the explanation, it makes sense!
Quoted 2 lines
'struct
mptcp_sub_info' is added as per mptcp.org kernel, but I don't like this structure. I think 'src' and 'dst' are confusing terms.
Do you prefer 'local' and 'remote'?
Good time to change!
Quoted 5 lines
AFAICS 'src' really is 'local address' (getsockname) and 'dst' is peer (getpeername).
Given mptcp-next has
IPPROTO_MPTCP, this addsSOL_MPTCP. This also gives ample space to add mptcp specific sockopts.
Very good idea!
Quoted 2 lines
In light of this, I wonder if it would make more sense to split the functionality.
Would it not start to be a bit costly to get all details if the userspace has do a few syscalls?
- general info
- get subflow IDs
- for each suflow, call subflow info + peer/sock names
With the full structure, we could eventually allow userspace programs to set some "*_len" members to 0 not to get all info, e.g. only get the general info without peer/sock names for all subflows. Or without TCP_INFO for each subflow. No?
Quoted 11 lines
Examples:
getsockopt(fd, SOL_MPTCP, SUBFLOW_ID, subflow_ids, &count);sa.sa_family = subflow_ids[0];
getsockopt(fd, SOL_MPTCP, SUBFLOW_GETPEERNAME, &sa, &len); ..
sa.sa_family = subflow_ids[0];getsockopt(fd, SOL_MPTCP, SUBFLOW_GETSOCKNAME, &sa, &len);
Quoted 6 lines
tcp_info.tcpi_state = subflow_ids[0];
getsockopt(fd, SOL_MPTCP, SUBFLOW_TCP_INFO, &tcp_info, &len);...
and so on.
This could be interesting if there is a demand to get info only for some specific subflows :)
Quoted 4 lines
Alternatively one could overload e.g. the upper 8 bit:
getsockopt(fd, SOL_MPTCP, subflow_ids[0] << 24 | SUBFLOW_FOO, ...);
Maybe a "cleaner" workaround to set the ID? :)
Quoted 2 lines
In any case, here is a tentative patch to add a mptcp.org-alike
MPTCP_INFOgetsockopt.
Thank you for sharing that!
(I didn't check the code yet)
Cheers,
Matt
Signature · 2 lines
Tessares | Belgium | Hybrid Access Solutions www.tessares.net
Quoted 16 lines · Matthieu Baerts
Hi Florian,
Thank you for looking at that!
Quoted 10 lines · Florian Westphal
Its not compatible with mptcp.org kernel one:
1. mptcp.org defines 'structmptcp_info', but we already have this forinet_diag.2. 'struct
mptcp_info', as defined by mptcp.org has different layout for 32/64 bit arches.
This is a problem for 32bit binaries on 64bit kernels.For those reasons a new 'struct
mptcp_info_opt' is added which containsaligned_u64fields to store the userspace buffer addresses.
Quoted 6 lines
Thank you for the explanation, it makes sense!
Quoted 2 lines
'struct
mptcp_sub_info' is added as per mptcp.org kernel, but I don't like this structure. I think 'src' and 'dst' are confusing terms.Do you prefer 'local' and 'remote'?
I thought about alternative names, I like your suggestion.
Quoted 8 lines
Quoted 2 lines
In light of this, I wonder if it would make more sense to split the functionality.
Would it not start to be a bit costly to get all details if the userspace has do a few syscalls?
- general info
- get subflow IDs
- for each suflow, call subflow info + peer/sock names
Yes, good point. Its unlikely one would ask for a specific subflow endpoint addresses.
So perhaps just keep that in mind for later and go with a mptcp.org alike approach.
Quoted 4 lines
With the full structure, we could eventually allow userspace programs to set some "
*_len" members to 0 not to get all info, e.g. only get the general info without peer/sock names for all subflows. Or withoutTCP_INFOfor each subflow. No?
Yes, userspace can set the address field to 0 and the kernel will skip that part.
One thing that I'd like to change is the way the length fields are handled. I propose that userspace can set the legnth field to 0 to request the kernel to fill in the required size.
This won't work reliably (there could be a new subflow arriving after the query and before the second getsockopt with the "properly sized" fields.
At the moment, there is no way to detect this.
Perhaps the info struct should contain two size versions, one with the "this is how much I copied to the buffer" and another with "this is how much i would have copied".
Quoted 9 lines
Quoted 9 lines
sa.sa_family = subflow_ids[0];
getsockopt(fd, SOL_MPTCP, SUBFLOW_GETPEERNAME, &sa, &len); ..
sa.sa_family = subflow_ids[0];getsockopt(fd, SOL_MPTCP, SUBFLOW_GETSOCKNAME, &sa, &len);tcp_info.tcpi_state = subflow_ids[0];
getsockopt(fd, SOL_MPTCP, SUBFLOW_TCP_INFO, &tcp_info, &len);
Quoted 6 lines
Quoted 3 lines
...
and so on.
This could be interesting if there is a demand to get info only for some specific subflows :)
Yes, as I wrote above your are probably right that the "give me the peer socket address for subflow 42" is a bit esoteric.
Quoted 5 lines
Quoted 3 lines
Alternatively one could overload e.g. the upper 8 bit:
getsockopt(fd, SOL_MPTCP, subflow_ids[0] << 24 | SUBFLOW_FOO, ...);Maybe a "cleaner" workaround to set the ID? :)
It would be more consistent, at least. The "piggyback in the buffer" is very ugly...
Quoted 6 lines
Quoted 2 lines
In any case, here is a tentative patch to add a mptcp.org-alike
MPTCP_INFOgetsockopt.Thank you for sharing that!
(I didn't check the code yet)
No need to have a super close look, its going to change anyway. Its also heavily lifted from the mptcp.org kernel.
I will add attributions as needed once a formal patch submission is made.
Paolo Abeni
maintainerQuoted 7 lines · Florian Westphal
Its not compatible with mptcp.org kernel one:
1. mptcp.org defines 'structmptcp_info', but we already have this forinet_diag.2. 'struct
mptcp_info', as defined by mptcp.org has different layout for 32/64 bit arches.
This is a problem for 32bit binaries on 64bit kernels.
I think the lack of binary compatibility with the OoO tree kernel is nto a big deal - and anyhow is unsolvable, so we have to consider it not a big deal ;)
I think it could *possibly* be nice to tie togethar mptcp_info and mptcp_info_opt - e.g. pulling the second into the first and unifying the handling, as tcp does for diag and GET_INFO. But perhpas is just too much code/overhead?!?
Quoted 8 lines
For those reasons a new 'struct
mptcp_info_opt' is added which containsaligned_u64fields to store the userspace buffer addresses.'struct
mptcp_sub_info' is added as per mptcp.org kernel, but I don't like this structure. I think 'src' and 'dst' are confusing terms.AFAICS 'src' really is 'local address' (getsockname) and 'dst' is peer (getpeername).
+1 for 'local' and 'remote'
Quoted 16 lines
Given mptcp-next has
IPPROTO_MPTCP, this addsSOL_MPTCP. This also gives ample space to add mptcp specific sockopts.In light of this, I wonder if it would make more sense to split the functionality.
Examples:
getsockopt(fd, SOL_MPTCP, SUBFLOW_ID, subflow_ids, &count);sa.sa_family = subflow_ids[0];
getsockopt(fd, SOL_MPTCP, SUBFLOW_GETPEERNAME, &sa, &len); ..
sa.sa_family = subflow_ids[0];getsockopt(fd, SOL_MPTCP, SUBFLOW_GETSOCKNAME, &sa, &len);
Quoted 6 lines
tcp_info.tcpi_state = subflow_ids[0];
getsockopt(fd, SOL_MPTCP, SUBFLOW_TCP_INFO, &tcp_info, &len);...
and so on.
+1 to fetch all the subflow info with a single syscall
+1 to pass back the user 2 sizes: one with
the "this is how much I copied to the buffer" and another with "this is how much i would have copied".
Quoted 3 lines
Alternatively one could overload e.g. the upper 8 bit:
getsockopt(fd, SOL_MPTCP, subflow_ids[0] << 24 | SUBFLOW_FOO, ...);
What about using:
struct mptcp_subflow_info {
u32 id;
struct tcp_info info;
};mptcp_subflow_info.id = id;getsockopt(fd, SOL_MPTCP, &mptcp_subflow_info, ...);
?
Cheers,
Paolo
Quoted 1 line · Paolo Abeni
On Thu, 2021-07-22 at 17:01 +0200, Florian Westphal wrote:
(...)
Quoted 3 lines
+1 to pass back the user 2 sizes: one with
the "this is how much I copied to the buffer" and another with "this is how much i would have copied".
+1
Quoted 6 lines
Quoted 3 lines
Alternatively one could overload e.g. the upper 8 bit:
getsockopt(fd, SOL_MPTCP, subflow_ids[0] << 24 | SUBFLOW_FOO, ...);What about using:
Quoted 7 lines
struct mptcp_subflow_info { u32 id; struct tcp_info info; };mptcp_subflow_info.id = id;
getsockopt(fd, SOL_MPTCP, &mptcp_subflow_info, ...);
Indeed, clearer if we can do that :)
Cheers,
Matt
Signature · 2 lines
Tessares | Belgium | Hybrid Access Solutions www.tessares.net