net_tstamp: add SCM_TS_OPT_ID to provide OPT_ID in control message
Subsystems (5)
RFC PATCH 1/1 net_tstamp: add SCM_TS_OPT_ID to provide OPT_ID in control message
SOF_TIMESTAMPING_OPT_ID socket option flag gives a way to correlate TX timestamps and packets sent via socket. Unfortunately, there is no way to reliably predict socket timestamp ID value in case of error returned by sendmsg [1]. This patch adds new control message type to give user-space software an opportunity to control the mapping between packets and values by providing ID with each sendmsg. This works fine for UDP sockets only, and explicit check is added to control message parser. Also, there is no easy way to use 0 as provided ID, so this is value treated as invalid.
[1] https://lore.kernel.org/netdev/CALCETrU0jB+kg0mhV6A8mrHfTE1D1pr1SD_B9Eaa9aDPfgHdtA@mail.gmail.com/
--- include/net/inet_sock.h | 1 + include/net/sock.h | 1 + include/uapi/asm-generic/socket.h | 2 ++ net/core/sock.c | 14 ++++++++++++++ net/ipv4/ip_output.c | 11 +++++++++-- net/ipv6/ip6_output.c | 11 +++++++++-- 6 files changed, 36 insertions(+), 4 deletions(-)
include/net/inet_sock.h +1 -0
@@ -174,6 +174,7 @@ struct inet_cork {__s16 tos;char priority;__u16 gso_size; +u32 ts_opt_id;u64 transmit_time;u32 mark; };
include/net/sock.h +1 -0
@@ -1794,6 +1794,7 @@ struct sockcm_cookie {u64 transmit_time;u32 mark;u32 tsflags; +u32 ts_opt_id; }; static inline void sockcm_init(struct sockcm_cookie *sockc,
include/uapi/asm-generic/socket.h +2 -0
@@ -135,6 +135,8 @@#define SO_PASSPIDFD 76#define SO_PEERPIDFD 77 +#define SCM_TS_OPT_ID 78 +#if !defined(__KERNEL__)#if __BITS_PER_LONG ==64 || (defined(__x86_64__) && defined(__ILP32__))
net/core/sock.c +14 -0
@@ -2859,6 +2859,20 @@ int __sock_cmsg_send(struct sock *sk, struct cmsghdr *cmsg,return -EINVAL ; sockc->transmit_time =get_unaligned ((u64 *)CMSG_DATA (cmsg));break ; +case SCM_TS_OPT_ID : +/* allow this option for UDP sockets only */ +if (!sk_is_udp (sk)) +return -EINVAL ; + tsflags =READ_ONCE (sk->sk_tsflags); +if (!(tsflags &SOF_TIMESTAMPING_OPT_ID )) +return -EINVAL ; +if (cmsg->cmsg_len !=CMSG_LEN (sizeof (u32))) +return -EINVAL ; + sockc->ts_opt_id =get_unaligned ((u32 *)CMSG_DATA (cmsg)); +/* do not allow 0 as packet id for timestamp */ +if (!sockc->ts_opt_id) +return -EINVAL ; +break ;/* SCM_RIGHTS and SCM_CREDENTIALS are semantically in SOL_UNIX. */ case SCM_RIGHTS :case SCM_CREDENTIALS :
net/ipv4/ip_output.c +9 -2
@@ -1050,8 +1050,14 @@ static int __ip_append_data(struct sock *sk, hold_tskey = cork->tx_flags &SKBTX_ANY_TSTAMP &&READ_ONCE (sk->sk_tsflags) &SOF_TIMESTAMPING_OPT_ID ; -if (hold_tskey) - tskey =atomic_inc_return (&sk->sk_tskey) -1 ; +if (hold_tskey) { +if (cork->ts_opt_id) { + hold_tskey = false; + tskey = cork->ts_opt_id; + }else { + tskey =atomic_inc_return (&sk->sk_tskey) -1 ; + } + }/* So, what's going on in the loop below? *
@@ -1324,6 +1330,7 @@ static int ip_setup_cork(struct sock *sk, struct inet_cork *cork, cork->mark = ipc->sockc.mark; cork->priority = ipc->priority; cork->transmit_time = ipc->sockc.transmit_time; + cork->ts_opt_id = ipc->sockc.ts_opt_id; cork->tx_flags =0 ;sock_tx_timestamp (sk, ipc->sockc.tsflags, &cork->tx_flags);
net/ipv6/ip6_output.c +9 -2
@@ -1401,6 +1401,7 @@ static int ip6_setup_cork(struct sock *sk, struct inet_cork_full *cork, cork->base.gso_size = ipc6->gso_size; cork->base.tx_flags =0 ; cork->base.mark = ipc6->sockc.mark; + cork->base.ts_opt_id = ipc6->sockc.ts_opt_id;sock_tx_timestamp (sk, ipc6->sockc.tsflags, &cork->base.tx_flags); cork->base.length =0 ;
@@ -1545,8 +1546,14 @@ static int __ip6_append_data(struct sock *sk, hold_tskey = cork->tx_flags &SKBTX_ANY_TSTAMP &&READ_ONCE (sk->sk_tsflags) &SOF_TIMESTAMPING_OPT_ID ; -if (hold_tskey) - tskey =atomic_inc_return (&sk->sk_tskey) -1 ; +if (hold_tskey) { +if (cork->ts_opt_id) { + hold_tskey = false; + tskey = cork->ts_opt_id; + }else { + tskey =atomic_inc_return (&sk->sk_tskey) -1 ; + } + }/* * Let's try using as much space as possible.
Signature · 1 line
2.43.5
Quoted 2 lines · Vadim Fedorenko
SOF_TIMESTAMPING_OPT_IDsocket option flag gives a way to correlate TX timestamps
+1 on the feature. Few minor points only.
Not a hard requirement, but would be nice if there was a test, e.g., as a tools/testing/../txtimestamp.c extension.
Quoted 3 lines
and packets sent via socket. Unfortunately, there is no way to reliably predict socket timestamp ID value in case of error returned by sendmsg [1].
Might be good to copy more context from the discussion to explain why reliable OPT_ID is infeasible. For UDP, it is as simple as lockless transmit. For RAW, things like MSG_MORE come into play.
Quoted 6 lines
This patch adds new control message type to give user-space software an opportunity to control the mapping between packets and values by providing ID with each sendmsg. This works fine for UDP sockets only, and explicit check is added to control message parser. Also, there is no easy way to use 0 as provided ID, so this is value treated as invalid.
This is because the code branches on non-zero value in the cookie, else uses ts_key. Please make this explicit. Or perhaps better, add a bit in the cookie so that the full 32-bit space can be used.
Quoted 17 lines
[1] https://lore.kernel.org/netdev/CALCETrU0jB+kg0mhV6A8mrHfTE1D1pr1SD_B9Eaa9aDPfgHdtA@mail.gmail.com/
Signed-off-by Vadim Fedorenko <[email protected]>--- include/net/inet_sock.h | 1 + include/net/sock.h | 1 + include/uapi/asm-generic/socket.h | 2 ++ net/core/sock.c | 14 ++++++++++++++ net/ipv4/ip_output.c | 11 +++++++++-- net/ipv6/ip6_output.c | 11 +++++++++-- 6 files changed, 36 insertions(+), 4 deletions(-)include/net/inet_sock.h +0
-0
include/net/inet_sock.h struct inet_cork { · line 180
Quoted 7 lines
include/net/inet_sock.h +1
-0@@ -174,6 +174,7 @@ struct inet_cork {__s16 tos;char priority;__u16 gso_size; +u32 ts_opt_id;u64 transmit_time;u32 mark; };
Ah there's a hole here. Nice!
Quoted 33 lines
include/net/sock.h +1
-0@@ -1794,6 +1794,7 @@ struct sockcm_cookie {u64 transmit_time;u32 mark;u32 tsflags; +u32 ts_opt_id; }; static inline void sockcm_init(struct sockcm_cookie *sockc,include/uapi/asm-generic/socket.h +2
-0@@ -135,6 +135,8 @@#define SO_PASSPIDFD 76#define SO_PEERPIDFD 77 +#define SCM_TS_OPT_ID 78 +#if !defined(__KERNEL__)#if __BITS_PER_LONG ==64 || (defined(__x86_64__) && defined(__ILP32__))net/core/sock.c +0
-0@@ -2859,6 +2859,20 @@ int __sock_cmsg_send(struct sock *sk, struct cmsghdr *cmsg,return -EINVAL ; sockc->transmit_time =get_unaligned ((u64 *)CMSG_DATA (cmsg));break ;
net/core/sock.c int __sock_cmsg_send(struct sock *sk, struct cmsghdr *cmsg, · line 2871
Quoted 10 lines
net/core/sock.c +10
-0+case SCM_TS_OPT_ID : +/* allow this option for UDP sockets only */ +if (!sk_is_udp (sk)) +return -EINVAL ; + tsflags =READ_ONCE (sk->sk_tsflags); +if (!(tsflags &SOF_TIMESTAMPING_OPT_ID )) +return -EINVAL ; +if (cmsg->cmsg_len !=CMSG_LEN (sizeof (u32))) +return -EINVAL ; + sockc->ts_opt_id =get_unaligned ((u32 *)CMSG_DATA (cmsg));
Is the get_unaligned here needed? I don't usually see that on CMSG_DATA accesses. Even though they are indeed likely to be unaligned.
Quoted 68 lines
net/core/sock.c +4
-0+/* do not allow 0 as packet id for timestamp */ +if (!sockc->ts_opt_id) +return -EINVAL ; +break ;/* SCM_RIGHTS and SCM_CREDENTIALS are semantically in SOL_UNIX. */ case SCM_RIGHTS :case SCM_CREDENTIALS :net/ipv4/ip_output.c +9
-2@@ -1050,8 +1050,14 @@ static int __ip_append_data(struct sock *sk, hold_tskey = cork->tx_flags &SKBTX_ANY_TSTAMP &&READ_ONCE (sk->sk_tsflags) &SOF_TIMESTAMPING_OPT_ID ; -if (hold_tskey) - tskey =atomic_inc_return (&sk->sk_tskey) -1 ; +if (hold_tskey) { +if (cork->ts_opt_id) { + hold_tskey = false; + tskey = cork->ts_opt_id; + }else { + tskey =atomic_inc_return (&sk->sk_tskey) -1 ; + } + }/* So, what's going on in the loop below? * @@ -1324,6 +1330,7 @@ static int ip_setup_cork(struct sock *sk, struct inet_cork *cork, cork->mark = ipc->sockc.mark; cork->priority = ipc->priority; cork->transmit_time = ipc->sockc.transmit_time; + cork->ts_opt_id = ipc->sockc.ts_opt_id; cork->tx_flags =0 ;sock_tx_timestamp (sk, ipc->sockc.tsflags, &cork->tx_flags);net/ipv6/ip6_output.c +9
-2@@ -1401,6 +1401,7 @@ static int ip6_setup_cork(struct sock *sk, struct inet_cork_full *cork, cork->base.gso_size = ipc6->gso_size; cork->base.tx_flags =0 ; cork->base.mark = ipc6->sockc.mark; + cork->base.ts_opt_id = ipc6->sockc.ts_opt_id;sock_tx_timestamp (sk, ipc6->sockc.tsflags, &cork->base.tx_flags); cork->base.length =0 ;@@ -1545,8 +1546,14 @@ static int __ip6_append_data(struct sock *sk, hold_tskey = cork->tx_flags &SKBTX_ANY_TSTAMP &&READ_ONCE (sk->sk_tsflags) &SOF_TIMESTAMPING_OPT_ID ; -if (hold_tskey) - tskey =atomic_inc_return (&sk->sk_tskey) -1 ; +if (hold_tskey) { +if (cork->ts_opt_id) { + hold_tskey = false; + tskey = cork->ts_opt_id; + }else { + tskey =atomic_inc_return (&sk->sk_tskey) -1 ; + } + }/* * Let's try using as much space as possible. Signature · 1 line
2.43.5
Quoted 8 lines · Willem de Bruijn
Quoted 2 lines · Vadim Fedorenko
SOF_TIMESTAMPING_OPT_IDsocket option flag gives a way to correlate TX timestamps+1 on the feature. Few minor points only.
Not a hard requirement, but would be nice if there was a test, e.g., as a
tools/testing/../txtimestamp.cextension.
Sure, I'll add some tests in the next version.
Quoted 7 lines
Quoted 3 lines
and packets sent via socket. Unfortunately, there is no way to reliably predict socket timestamp ID value in case of error returned by sendmsg [1].
Might be good to copy more context from the discussion to explain why reliable
OPT_IDis infeasible. For UDP, it is as simple as lockless transmit. For RAW, things likeMSG_MOREcome into play.
Ok, I'll add it, thanks!
Quoted 10 lines
Quoted 6 lines
This patch adds new control message type to give user-space software an opportunity to control the mapping between packets and values by providing ID with each sendmsg. This works fine for UDP sockets only, and explicit check is added to control message parser. Also, there is no easy way to use 0 as provided ID, so this is value treated as invalid.
This is because the code branches on non-zero value in the cookie, else uses
ts_key. Please make this explicit. Or perhaps better, add a bit in the cookie so that the full 32-bit space can be used.
Adding a bit in the cookie is not enough, I have to add another flag toinet_cork. And we are running out of space for tx flags, inet_cork::tx_flags is u8 and we have only 1 bit left for SKBTX* enum. Do you think it's OK to use this last bit for OPT_ID feature?
Quoted 66 lines
Quoted 17 lines
[1] https://lore.kernel.org/netdev/CALCETrU0jB+kg0mhV6A8mrHfTE1D1pr1SD_B9Eaa9aDPfgHdtA@mail.gmail.com/
Signed-off-by Vadim Fedorenko <[email protected]>--- include/net/inet_sock.h | 1 + include/net/sock.h | 1 + include/uapi/asm-generic/socket.h | 2 ++ net/core/sock.c | 14 ++++++++++++++ net/ipv4/ip_output.c | 11 +++++++++-- net/ipv6/ip6_output.c | 11 +++++++++-- 6 files changed, 36 insertions(+), 4 deletions(-)include/net/inet_sock.h +0
-0include/net/inet_sock.h struct inet_cork { · line 180
Quoted 7 lines
include/net/inet_sock.h +1
-0@@ -174,6 +174,7 @@ struct inet_cork {__s16 tos;char priority;__u16 gso_size; +u32 ts_opt_id;u64 transmit_time;u32 mark; };Ah there's a hole here. Nice!
Quoted 39 lines
include/net/sock.h +1
-0@@ -1794,6 +1794,7 @@ struct sockcm_cookie {u64 transmit_time;u32 mark;u32 tsflags; +u32 ts_opt_id; }; static inline void sockcm_init(struct sockcm_cookie *sockc,include/uapi/asm-generic/socket.h +2
-0@@ -135,6 +135,8 @@#define SO_PASSPIDFD 76#define SO_PEERPIDFD 77 +#define SCM_TS_OPT_ID 78 +#if !defined(__KERNEL__)#if __BITS_PER_LONG ==64 || (defined(__x86_64__) && defined(__ILP32__))net/core/sock.c +6
-0@@ -2859,6 +2859,20 @@ int __sock_cmsg_send(struct sock *sk, struct cmsghdr *cmsg,return -EINVAL ; sockc->transmit_time =get_unaligned ((u64 *)CMSG_DATA (cmsg));break ; +case SCM_TS_OPT_ID : +/* allow this option for UDP sockets only */ +if (!sk_is_udp (sk)) +return -EINVAL ; + tsflags =READ_ONCE (sk->sk_tsflags); +if (!(tsflags &SOF_TIMESTAMPING_OPT_ID ))
Quoted 8 lines
net/core/sock.c int __sock_cmsg_send(struct sock *sk, struct cmsghdr *cmsg, · line 2871
Quoted 4 lines
net/core/sock.c +4
-0+return -EINVAL ; +if (cmsg->cmsg_len !=CMSG_LEN (sizeof (u32))) +return -EINVAL ; + sockc->ts_opt_id =get_unaligned ((u32 *)CMSG_DATA (cmsg));Is the
get_unalignedhere needed? I don't usually see that onCMSG_DATAaccesses. Even though they are indeed likely to be unaligned.
Well, maybe you are right and we don't need get_unaligned for u32 here, at least SO_MARK uses direct access. I have no strong opinion.
Quoted 70 lines
Quoted 68 lines
net/core/sock.c +4
-0+/* do not allow 0 as packet id for timestamp */ +if (!sockc->ts_opt_id) +return -EINVAL ; +break ;/* SCM_RIGHTS and SCM_CREDENTIALS are semantically in SOL_UNIX. */ case SCM_RIGHTS :case SCM_CREDENTIALS :net/ipv4/ip_output.c +9
-2@@ -1050,8 +1050,14 @@ static int __ip_append_data(struct sock *sk, hold_tskey = cork->tx_flags &SKBTX_ANY_TSTAMP &&READ_ONCE (sk->sk_tsflags) &SOF_TIMESTAMPING_OPT_ID ; -if (hold_tskey) - tskey =atomic_inc_return (&sk->sk_tskey) -1 ; +if (hold_tskey) { +if (cork->ts_opt_id) { + hold_tskey = false; + tskey = cork->ts_opt_id; + }else { + tskey =atomic_inc_return (&sk->sk_tskey) -1 ; + } + }/* So, what's going on in the loop below? * @@ -1324,6 +1330,7 @@ static int ip_setup_cork(struct sock *sk, struct inet_cork *cork, cork->mark = ipc->sockc.mark; cork->priority = ipc->priority; cork->transmit_time = ipc->sockc.transmit_time; + cork->ts_opt_id = ipc->sockc.ts_opt_id; cork->tx_flags =0 ;sock_tx_timestamp (sk, ipc->sockc.tsflags, &cork->tx_flags);net/ipv6/ip6_output.c +9
-2@@ -1401,6 +1401,7 @@ static int ip6_setup_cork(struct sock *sk, struct inet_cork_full *cork, cork->base.gso_size = ipc6->gso_size; cork->base.tx_flags =0 ; cork->base.mark = ipc6->sockc.mark; + cork->base.ts_opt_id = ipc6->sockc.ts_opt_id;sock_tx_timestamp (sk, ipc6->sockc.tsflags, &cork->base.tx_flags); cork->base.length =0 ;@@ -1545,8 +1546,14 @@ static int __ip6_append_data(struct sock *sk, hold_tskey = cork->tx_flags &SKBTX_ANY_TSTAMP &&READ_ONCE (sk->sk_tsflags) &SOF_TIMESTAMPING_OPT_ID ; -if (hold_tskey) - tskey =atomic_inc_return (&sk->sk_tskey) -1 ; +if (hold_tskey) { +if (cork->ts_opt_id) { + hold_tskey = false; + tskey = cork->ts_opt_id; + }else { + tskey =atomic_inc_return (&sk->sk_tskey) -1 ; + } + }/* * Let's try using as much space as possible. Signature · 1 line
2.43.5
Quoted 30 lines · Vadim Fedorenko
Quoted 8 lines · Willem de Bruijn
Quoted 2 lines · Vadim Fedorenko
SOF_TIMESTAMPING_OPT_IDsocket option flag gives a way to correlate TX timestamps+1 on the feature. Few minor points only.
Not a hard requirement, but would be nice if there was a test, e.g., as a
tools/testing/../txtimestamp.cextension.Sure, I'll add some tests in the next version.
Quoted 7 lines
Quoted 3 lines
and packets sent via socket. Unfortunately, there is no way to reliably predict socket timestamp ID value in case of error returned by sendmsg [1].
Might be good to copy more context from the discussion to explain why reliable
OPT_IDis infeasible. For UDP, it is as simple as lockless transmit. For RAW, things likeMSG_MOREcome into play.Ok, I'll add it, thanks!
Quoted 7 lines
Quoted 6 lines
This patch adds new control message type to give user-space software an opportunity to control the mapping between packets and values by providing ID with each sendmsg. This works fine for UDP sockets only, and explicit check is added to control message parser. Also, there is no easy way to use 0 as provided ID, so this is value treated as invalid.
Quoted 8 lines
Quoted 3 lines
This is because the code branches on non-zero value in the cookie, else uses
ts_key. Please make this explicit. Or perhaps better, add a bit in the cookie so that the full 32-bit space can be used.Adding a bit in the cookie is not enough, I have to add another flag to
inet_cork. And we are running out of space for tx flags,inet_cork::tx_flagsis u8 and we have only 1 bit left for SKBTX* enum. Do you think it's OK to use this last bit forOPT_IDfeature?
No, that space is particularly constrained in skb_shinfo.
Either a separate bit in inet_cork, or just keep as is.
Quoted 35 lines · Willem de Bruijn
Quoted 34 lines · Vadim Fedorenko
Quoted 8 lines · Willem de Bruijn
Quoted 2 lines · Vadim Fedorenko
SOF_TIMESTAMPING_OPT_IDsocket option flag gives a way to correlate TX timestamps+1 on the feature. Few minor points only.
Not a hard requirement, but would be nice if there was a test, e.g., as a
tools/testing/../txtimestamp.cextension.Sure, I'll add some tests in the next version.
Quoted 7 lines
Quoted 3 lines
and packets sent via socket. Unfortunately, there is no way to reliably predict socket timestamp ID value in case of error returned by sendmsg [1].
Might be good to copy more context from the discussion to explain why reliable
OPT_IDis infeasible. For UDP, it is as simple as lockless transmit. For RAW, things likeMSG_MOREcome into play.Ok, I'll add it, thanks!
Quoted 10 lines
Quoted 6 lines
This patch adds new control message type to give user-space software an opportunity to control the mapping between packets and values by providing ID with each sendmsg. This works fine for UDP sockets only, and explicit check is added to control message parser. Also, there is no easy way to use 0 as provided ID, so this is value treated as invalid.
This is because the code branches on non-zero value in the cookie, else uses
ts_key. Please make this explicit. Or perhaps better, add a bit in the cookie so that the full 32-bit space can be used.
Quoted 8 lines
Quoted 4 lines
Adding a bit in the cookie is not enough, I have to add another flag to
inet_cork. And we are running out of space for tx flags,inet_cork::tx_flagsis u8 and we have only 1 bit left for SKBTX* enum. Do you think it's OK to use this last bit forOPT_IDfeature?No, that space is particularly constrained in
skb_shinfo.Either a separate bit in
inet_cork, or just keep as is.
Ok, got it. I'll add IPCORK_TS_OPT_ID flag then. Thanks!