Skip to contentSkip to conversation

net-timestamp: bpf extension to equip applications transparently

PATCH Superseded Jason Xing 3 participants 45 messages Jan 28, 2025 – Feb 7, 2025

Subsystems (10)

v7 of 13 Superseded Latest is v13 →

lore
Share & tools
lore

Jason Xing

author
#0 lore raw

v7 cover net-timestamp: bpf extension to equip applications transparently

"Timestamping is key to debugging network stack latency. With SO_TIMESTAMPING, bugs that are otherwise incorrectly assumed to be network issues can be attributed to the kernel." This is extracted from the talk "SO_TIMESTAMPING: Powering Fleetwide RPC Monitoring" addressed by Willem de Bruijn at netdevconf 0x17).

There are a few areas that need optimization with the consideration of easier use and less performance impact, which I highlighted and mainly discussed at netconf 2024 with Willem de Bruijn and John Fastabend:
uAPI compatibility, extra system call overhead, and the need for application modification. I initially managed to solve these issues by writing a kernel module that hooks various key functions. However, this approach is not suitable for the next kernel release. Therefore, a BPF extension was proposed. During recent period, Martin KaFai Lau provides invaluable suggestions about BPF along the way. Many thanks here!

In this series, I only support foundamental codes and tx for TCP. This approach mostly relies on existing SO_TIMESTAMPING feature, users only needs to pass certain flags through bpf_setsocktopt() to a separate tsflags. Please see the last selftest patch in this series.

After this series, we could step by step implement more advanced functions/flags already in SO_TIMESTAMPING feature for bpf extension.

---
v7

1. target bpf-next tree
2. simplely and directly stop timestamping callbacks calling a few BPF CALLS due to safety concern.
3. add more new testcases and adjust the existing testcases
4. revise some comments of new timestamping callbacks
5. remove a few BPF CGROUP locks

RFC v6
In the meantime, any suggestions and reviews are welcome!

1. handle those safety problem by using the correct method.
2. support bpf_getsockopt.
3. adjust the position of BPF_SOCK_OPS_TS_TCP_SND_CB
4. fix mishandling the hardware timestamp error
5. add more corresponding tests

v5

1. handle the safety issus when someone tries to call unrelated bpf helpers.
2. avoid adding direct function call in the hot path like __dev_queue_xmit()
3. remove reporting the hardware timestamp and tskey since they can be fetched through the existing helper with the help of bpf_skops_init_skb(), please see the selftest.
4. add new sendmsg callback in tcp_sendmsg, and introduce tskey_bpf used by bpf program to correlate tcp_sendmsg with other hook points in patch [13/15].

v4

1. introduce sk->sk_bpf_cb_flags to let user use bpf_setsockopt() (Martin)
2. introduce SKBTX_BPF to enable the bpf SO_TIMESTAMPING feature (Martin)
3. introduce bpf map in tests (Martin)
4. I choose to make this series as simple as possible, so I only support most cases in the tx path for TCP protocol.

v3

1. support UDP proto by introducing a new generation point.
2. for OPT_ID, introducing sk_tskey_bpf_offset to compute the delta between the current socket key and bpf socket key. It is desiged for UDP, which also applies to TCP.
3. support bpf_getsockopt()
4. use cgroup static key instead.
5. add one simple bpf selftest to show how it can be used.
6. remove the rx support from v2 because the number of patches could exceed the limit of one series.

V2

1. Introduce tsflag requestors so that we are able to extend more in the future. Besides, it enables TX flags for bpf extension feature separately without breaking users. It is suggested by Vadim Fedorenko.
2. introduce a static key to control the whole feature. (Willem)
3. Open the gate of bpf_setsockopt for the SO_TIMESTAMPING feature in some TX/RX cases, not all the cases.

Jason Xing (13):
net-timestamp: add support for bpf_setsockopt()
net-timestamp: prepare for timestamping callbacks use
bpf: stop unsafely accessing TCP fields in bpf callbacks
bpf: stop calling some sock_op BPF CALLs in new timestamping callbacks
net-timestamp: prepare for isolating two modes of SO_TIMESTAMPING
net-timestamp: support SCM_TSTAMP_SCHED for bpf extension
net-timestamp: support sw SCM_TSTAMP_SND for bpf extension
net-timestamp: support hw SCM_TSTAMP_SND for bpf extension
net-timestamp: support SCM_TSTAMP_ACK for bpf extension
net-timestamp: make TCP tx timestamp bpf extension work
net-timestamp: add a new callback in tcp_tx_timestamp()
net-timestamp: introduce cgroup lock to avoid affecting non-bpf cases
bpf: add simple bpf tests in the tx path for so_timestamping feature

 include/linux/filter.h                        |   5 +
 include/linux/skbuff.h                        |  25 +-
 include/net/sock.h                            |  10 +
 include/net/tcp.h                             |   4 +-
 include/uapi/linux/bpf.h                      |  35 ++
 net/core/dev.c                                |   5 +-
 net/core/filter.c                             |  48 ++-
 net/core/skbuff.c                             |  65 +++-
 net/core/sock.c                               |  15 +
 net/dsa/user.c                                |   2 +-
 net/ipv4/tcp.c                                |  11 +
 net/ipv4/tcp_input.c                          |   8 +-
 net/ipv4/tcp_output.c                         |   7 +
 net/socket.c                                  |   2 +-
 tools/include/uapi/linux/bpf.h                |  28 ++
 .../bpf/prog_tests/so_timestamping.c          |  86 +++++
 .../selftests/bpf/progs/so_timestamping.c     | 299 ++++++++++++++++++
 17 files changed, 633 insertions(+), 22 deletions(-)
 create mode 100644 tools/testing/selftests/bpf/prog_tests/so_timestamping.c
 create mode 100644 tools/testing/selftests/bpf/progs/so_timestamping.c
Signature · 1 line
2.43.5
author
#1 lore raw

PATCH v7 1/13 net-timestamp: add support for bpf_setsockopt()

Users can write the following code to enable the bpf extension:
bpf_setsockopt(skops, SOL_SOCKET, SK_BPF_CB_FLAGS, &flags, sizeof(flags));

---
 include/net/sock.h             |  3 +++
 include/uapi/linux/bpf.h       |  8 ++++++++
 net/core/filter.c              | 23 +++++++++++++++++++++++
 tools/include/uapi/linux/bpf.h |  1 +
 4 files changed, 35 insertions(+)
include/net/sock.h +3 -0
index 8036b3b79cd8..7916982343c6 100644
@@ -303,6 +303,7 @@ struct sk_filter;
   *	@sk_stamp: time stamp of last packet received
   *	@sk_stamp_seq: lock for accessing sk_stamp on 32 bit architectures only
   *	@sk_tsflags: SO_TIMESTAMPING flags
+  *	@sk_bpf_cb_flags: used in bpf_setsockopt()
   *	@sk_use_task_frag: allow sk_page_frag() to use current->task_frag.
   *			   Sockets that can be used under memory reclaim should
   *			   set this to false.
@@ -445,6 +446,8 @@ struct sock {
 	u32			sk_reserved_mem;
 	int			sk_forward_alloc;
 	u32			sk_tsflags;
+#define SK_BPF_CB_FLAG_TEST(SK, FLAG) ((SK)->sk_bpf_cb_flags & (FLAG))
+	u32			sk_bpf_cb_flags;
 	__cacheline_group_end(sock_write_rxtx);
 
 	__cacheline_group_begin(sock_write_tx);
include/uapi/linux/bpf.h +8 -0
index 2acf9b336371..6116eb3d1515 100644
@@ -6913,6 +6913,13 @@ enum {
 	BPF_SOCK_OPS_ALL_CB_FLAGS       = 0x7F,
 };
 
+/* Definitions for bpf_sk_cb_flags */
+enum {
+	SK_BPF_CB_TX_TIMESTAMPING	= 1<<0,
+	SK_BPF_CB_MASK			= (SK_BPF_CB_TX_TIMESTAMPING - 1) |
+					   SK_BPF_CB_TX_TIMESTAMPING
+};
+
 /* List of known BPF sock_ops operators.
  * New entries can only be added at the end
  */
@@ -7091,6 +7098,7 @@ enum {
 	TCP_BPF_SYN_IP		= 1006, /* Copy the IP[46] and TCP header */
 	TCP_BPF_SYN_MAC         = 1007, /* Copy the MAC, IP[46], and TCP header */
 	TCP_BPF_SOCK_OPS_CB_FLAGS = 1008, /* Get or Set TCP sock ops flags */
+	SK_BPF_CB_FLAGS		= 1009, /* Used to set socket bpf flags */
 };
 
 enum {
net/core/filter.c +23 -0
index 2ec162dd83c4..1c6c07507a78 100644
@@ -5222,6 +5222,25 @@ static const struct bpf_func_proto bpf_get_socket_uid_proto = {
 	.arg1_type      = ARG_PTR_TO_CTX,
 };
 
+static int sk_bpf_set_get_cb_flags(struct sock *sk, char *optval, bool getopt)
+{
+	u32 sk_bpf_cb_flags;
+
+	if (getopt) {
+		*(u32 *)optval = sk->sk_bpf_cb_flags;
+		return 0;
+	}
+
+	sk_bpf_cb_flags = *(u32 *)optval;
+
+	if (sk_bpf_cb_flags & ~SK_BPF_CB_MASK)
+		return -EINVAL;
+
+	sk->sk_bpf_cb_flags = sk_bpf_cb_flags;
+
+	return 0;
+}
+
 static int sol_socket_sockopt(struct sock *sk, int optname,
 			      char *optval, int *optlen,
 			      bool getopt)
@@ -5238,6 +5257,7 @@ static int sol_socket_sockopt(struct sock *sk, int optname,
 	case SO_MAX_PACING_RATE:
 	case SO_BINDTOIFINDEX:
 	case SO_TXREHASH:
+	case SK_BPF_CB_FLAGS:
 		if (*optlen != sizeof(int))
 			return -EINVAL;
 		break;
@@ -5247,6 +5267,9 @@ static int sol_socket_sockopt(struct sock *sk, int optname,
 		return -EINVAL;
 	}
 
+	if (optname == SK_BPF_CB_FLAGS)
+		return sk_bpf_set_get_cb_flags(sk, optval, getopt);
+
 	if (getopt) {
 		if (optname == SO_BINDTODEVICE)
 			return -EINVAL;
tools/include/uapi/linux/bpf.h +1 -0
index 2acf9b336371..70366f74ef4e 100644
@@ -7091,6 +7091,7 @@ enum {
 	TCP_BPF_SYN_IP		= 1006, /* Copy the IP[46] and TCP header */
 	TCP_BPF_SYN_MAC         = 1007, /* Copy the MAC, IP[46], and TCP header */
 	TCP_BPF_SOCK_OPS_CB_FLAGS = 1008, /* Get or Set TCP sock ops flags */
+	SK_BPF_CB_FLAGS		= 1009, /* Used to set socket bpf flags */
 };
 
 enum {
Signature · 1 line
2.43.5
author
#2 lore raw

PATCH v7 2/13 net-timestamp: prepare for timestamping callbacks use

In reply to Jason Xing

Later, I would introduce four callback points to report information to user space based on this patch.

As to skb initialization here, people can follow these three steps as below to fetch the shared info from the exported skb in the bpf prog:
1. skops_kern = bpf_cast_to_kern_ctx(skops);
2. skb = skops_kern->skb;
3. shinfo = bpf_core_cast(skb->head + skb->end, struct skb_shared_info);

More details can be seen in the last selftest patch of the series.

---
 include/net/sock.h |  7 +++++++
 net/core/sock.c    | 15 +++++++++++++++
 2 files changed, 22 insertions(+)
include/net/sock.h +7 -0
index 7916982343c6..6f4d54faba92 100644
@@ -2923,6 +2923,13 @@ int sock_set_timestamping(struct sock *sk, int optname,
 			  struct so_timestamping timestamping);
 
 void sock_enable_timestamps(struct sock *sk);
+#if defined(CONFIG_CGROUP_BPF)
+void bpf_skops_tx_timestamping(struct sock *sk, struct sk_buff *skb, int op);
+#else
+static inline void bpf_skops_tx_timestamping(struct sock *sk, struct sk_buff *skb, int op)
+{
+}
+#endif
 void sock_no_linger(struct sock *sk);
 void sock_set_keepalive(struct sock *sk);
 void sock_set_priority(struct sock *sk, u32 priority);
net/core/sock.c +15 -0
index eae2ae70a2e0..41db6407e360 100644
@@ -948,6 +948,21 @@ int sock_set_timestamping(struct sock *sk, int optname,
 	return 0;
 }
 
+#if defined(CONFIG_CGROUP_BPF)
+void bpf_skops_tx_timestamping(struct sock *sk, struct sk_buff *skb, int op)
+{
+	struct bpf_sock_ops_kern sock_ops;
+
+	memset(&sock_ops, 0, offsetof(struct bpf_sock_ops_kern, temp));
+	sock_ops.op = op;
+	sock_ops.is_fullsock = 1;
+	sock_ops.sk = sk;
+	bpf_skops_init_skb(&sock_ops, skb, 0);
+	/* Timestamping bpf extension supports only TCP and UDP full socket */
+	__cgroup_bpf_run_filter_sock_ops(sk, &sock_ops, CGROUP_SOCK_OPS);
+}
+#endif
+
 void sock_set_keepalive(struct sock *sk)
 {
 	lock_sock(sk);
Signature · 1 line
2.43.5
author
#3 lore raw

PATCH v7 3/13 bpf: stop unsafely accessing TCP fields in bpf callbacks

In reply to Jason Xing

The "allow_tcp_access" flag is added to indicate that the callback site has a tcp_sock locked.

Applying the new member allow_tcp_access in the existing callbacks where is_fullsock is set to 1 can help us stop UDP socket accessing struct tcp_sock and stop TCP socket without sk lock protecting does the similar thing, or else it could be catastrophe leading to panic.

To keep it simple, instead of distinguishing between read and write access, we disallow all read/write access to the tcp_sock through the older bpf_sock_ops ctx. The new timestamping callbacks can use newer helpers to read everything from a sk (e.g. bpf_core_cast), so nothing is lost.

---
 include/linux/filter.h | 5 +++++
 include/net/tcp.h      | 1 +
 net/core/filter.c      | 8 ++++----
 net/ipv4/tcp_input.c   | 2 ++
 net/ipv4/tcp_output.c  | 2 ++
 5 files changed, 14 insertions(+), 4 deletions(-)
include/linux/filter.h +5 -0
index a3ea46281595..1569e9f31a8c 100644
@@ -1508,6 +1508,11 @@ struct bpf_sock_ops_kern {
 	void	*skb_data_end;
 	u8	op;
 	u8	is_fullsock;
+	u8	allow_tcp_access;	/* Indicate that the callback site
+					 * has a tcp_sock locked. Then it
+					 * would be safe to access struct
+					 * tcp_sock.
+					 */
 	u8	remaining_opt_len;
 	u64	temp;			/* temp and everything after is not
 					 * initialized to 0 before calling
include/net/tcp.h +1 -0
index 5b2b04835688..293047694710 100644
@@ -2649,6 +2649,7 @@ static inline int tcp_call_bpf(struct sock *sk, int op, u32 nargs, u32 *args)
 	memset(&sock_ops, 0, offsetof(struct bpf_sock_ops_kern, temp));
 	if (sk_fullsock(sk)) {
 		sock_ops.is_fullsock = 1;
+		sock_ops.allow_tcp_access = 1;
 		sock_owned_by_me(sk);
 	}
 
net/core/filter.c +4 -4
index 1c6c07507a78..dc0e67c5776a 100644
@@ -10381,10 +10381,10 @@ static u32 sock_ops_convert_ctx_access(enum bpf_access_type type,
 		}							      \
 		*insn++ = BPF_LDX_MEM(BPF_FIELD_SIZEOF(			      \
 						struct bpf_sock_ops_kern,     \
-						is_fullsock),		      \
+						allow_tcp_access),	      \
 				      fullsock_reg, si->src_reg,	      \
 				      offsetof(struct bpf_sock_ops_kern,      \
-					       is_fullsock));		      \
+					       allow_tcp_access));	      \
 		*insn++ = BPF_JMP_IMM(BPF_JEQ, fullsock_reg, 0, jmp);	      \
 		if (si->dst_reg == si->src_reg)				      \
 			*insn++ = BPF_LDX_MEM(BPF_DW, reg, si->src_reg,	      \
@@ -10469,10 +10469,10 @@ static u32 sock_ops_convert_ctx_access(enum bpf_access_type type,
 					       temp));			      \
 		*insn++ = BPF_LDX_MEM(BPF_FIELD_SIZEOF(			      \
 						struct bpf_sock_ops_kern,     \
-						is_fullsock),		      \
+						allow_tcp_access),	      \
 				      reg, si->dst_reg,			      \
 				      offsetof(struct bpf_sock_ops_kern,      \
-					       is_fullsock));		      \
+					       allow_tcp_access));	      \
 		*insn++ = BPF_JMP_IMM(BPF_JEQ, reg, 0, 2);		      \
 		*insn++ = BPF_LDX_MEM(BPF_FIELD_SIZEOF(			      \
 						struct bpf_sock_ops_kern, sk),\
net/ipv4/tcp_input.c +2 -0
index eb82e01da911..77185479ed5e 100644
@@ -169,6 +169,7 @@ static void bpf_skops_parse_hdr(struct sock *sk, struct sk_buff *skb)
 	memset(&sock_ops, 0, offsetof(struct bpf_sock_ops_kern, temp));
 	sock_ops.op = BPF_SOCK_OPS_PARSE_HDR_OPT_CB;
 	sock_ops.is_fullsock = 1;
+	sock_ops.allow_tcp_access = 1;
 	sock_ops.sk = sk;
 	bpf_skops_init_skb(&sock_ops, skb, tcp_hdrlen(skb));
 
@@ -185,6 +186,7 @@ static void bpf_skops_established(struct sock *sk, int bpf_op,
 	memset(&sock_ops, 0, offsetof(struct bpf_sock_ops_kern, temp));
 	sock_ops.op = bpf_op;
 	sock_ops.is_fullsock = 1;
+	sock_ops.allow_tcp_access = 1;
 	sock_ops.sk = sk;
 	/* sk with TCP_REPAIR_ON does not have skb in tcp_finish_connect */
 	if (skb)
net/ipv4/tcp_output.c +2 -0
index 0e5b9a654254..695749807c09 100644
@@ -522,6 +522,7 @@ static void bpf_skops_hdr_opt_len(struct sock *sk, struct sk_buff *skb,
 		sock_owned_by_me(sk);
 
 		sock_ops.is_fullsock = 1;
+		sock_ops.allow_tcp_access = 1;
 		sock_ops.sk = sk;
 	}
 
@@ -567,6 +568,7 @@ static void bpf_skops_write_hdr_opt(struct sock *sk, struct sk_buff *skb,
 		sock_owned_by_me(sk);
 
 		sock_ops.is_fullsock = 1;
+		sock_ops.allow_tcp_access = 1;
 		sock_ops.sk = sk;
 	}
 
Signature · 1 line
2.43.5
author
#4 lore raw

PATCH v7 4/13 bpf: stop calling some sock_op BPF CALLs in new timestamping callbacks

In reply to Jason Xing

Simply disallow calling bpf_sock_ops_setsockopt/getsockopt, bpf_sock_ops_cb_flags_set, and the bpf_sock_ops_load_hdr_opt for the new timestamping callbacks for the safety consideration.

Besides, In the next round, we will support the UDP proto for SO_TIMESTAMPING bpf extension, so we need to ensure there is no safety problem, which is usually caused by UDP socket trying to access TCP fields.

---
 net/core/filter.c | 17 +++++++++++++++++
 1 file changed, 17 insertions(+)
net/core/filter.c +17 -0
index dc0e67c5776a..d3395ffe058e 100644
@@ -5523,6 +5523,11 @@ static int __bpf_setsockopt(struct sock *sk, int level, int optname,
 	return -EINVAL;
 }
 
+static bool is_locked_tcp_sock_ops(struct bpf_sock_ops_kern *bpf_sock)
+{
+	return bpf_sock->op <= BPF_SOCK_OPS_WRITE_HDR_OPT_CB;
+}
+
 static int _bpf_setsockopt(struct sock *sk, int level, int optname,
 			   char *optval, int optlen)
 {
@@ -5673,6 +5678,9 @@ static const struct bpf_func_proto bpf_sock_addr_getsockopt_proto = {
 BPF_CALL_5(bpf_sock_ops_setsockopt, struct bpf_sock_ops_kern *, bpf_sock,
 	   int, level, int, optname, char *, optval, int, optlen)
 {
+	if (!is_locked_tcp_sock_ops(bpf_sock))
+		return -EOPNOTSUPP;
+
 	return _bpf_setsockopt(bpf_sock->sk, level, optname, optval, optlen);
 }
 
@@ -5758,6 +5766,9 @@ static int bpf_sock_ops_get_syn(struct bpf_sock_ops_kern *bpf_sock,
 BPF_CALL_5(bpf_sock_ops_getsockopt, struct bpf_sock_ops_kern *, bpf_sock,
 	   int, level, int, optname, char *, optval, int, optlen)
 {
+	if (!is_locked_tcp_sock_ops(bpf_sock))
+		return -EOPNOTSUPP;
+
 	if (IS_ENABLED(CONFIG_INET) && level == SOL_TCP &&
 	    optname >= TCP_BPF_SYN && optname <= TCP_BPF_SYN_MAC) {
 		int ret, copy_len = 0;
@@ -5800,6 +5811,9 @@ BPF_CALL_2(bpf_sock_ops_cb_flags_set, struct bpf_sock_ops_kern *, bpf_sock,
 	struct sock *sk = bpf_sock->sk;
 	int val = argval & BPF_SOCK_OPS_ALL_CB_FLAGS;
 
+	if (!is_locked_tcp_sock_ops(bpf_sock))
+		return -EOPNOTSUPP;
+
 	if (!IS_ENABLED(CONFIG_INET) || !sk_fullsock(sk))
 		return -EINVAL;
 
@@ -7609,6 +7623,9 @@ BPF_CALL_4(bpf_sock_ops_load_hdr_opt, struct bpf_sock_ops_kern *, bpf_sock,
 	u8 search_kind, search_len, copy_len, magic_len;
 	int ret;
 
+	if (!is_locked_tcp_sock_ops(bpf_sock))
+		return -EOPNOTSUPP;
+
 	/* 2 byte is the minimal option len except TCPOPT_NOP and
 	 * TCPOPT_EOL which are useless for the bpf prog to learn
 	 * and this helper disallow loading them also.
Signature · 1 line
2.43.5
author
#5 lore raw

PATCH v7 5/13 net-timestamp: prepare for isolating two modes of SO_TIMESTAMPING

In reply to Jason Xing

No functional changes here. I add skb_enable_app_tstamp() to test if the orig_skb matches the usage of application SO_TIMESTAMPING and skb_sw_tstamp_tx() to distinguish the software and hardware timestamp when tsflag is SCM_TSTAMP_SND.

Also, I deliberately distinguish the the software and hardware SCM_TSTAMP_SND timestamp by passing 'sw' parameter in order to avoid such a case where hardware may go wrong and pass a NULL hwstamps, which is even though unlikely to happen. If it really happens, bpf prog will finally consider it as a software timestamp. It will be hardly recognized. Let's make the timestamping part more robust.

After this patch, I will soon add checks about bpf SO_TIMESTAMPING. In this way, we can support two modes parallelly.

---
 include/linux/skbuff.h | 13 +++++++------
 net/core/dev.c         |  2 +-
 net/core/skbuff.c      | 32 ++++++++++++++++++++++++++++++--
 net/ipv4/tcp_input.c   |  3 ++-
 4 files changed, 40 insertions(+), 10 deletions(-)
include/linux/skbuff.h +7 -6
index bb2b751d274a..dfc419281cc9 100644
@@ -39,6 +39,7 @@
 #include <net/net_debug.h>
 #include <net/dropreason-core.h>
 #include <net/netmem.h>
+#include <uapi/linux/errqueue.h>
 
 /**
  * DOC: skb checksums
@@ -4533,18 +4534,18 @@ void skb_complete_tx_timestamp(struct sk_buff *skb,
 
 void __skb_tstamp_tx(struct sk_buff *orig_skb, const struct sk_buff *ack_skb,
 		     struct skb_shared_hwtstamps *hwtstamps,
-		     struct sock *sk, int tstype);
+		     struct sock *sk, bool sw, int tstype);
 
 /**
- * skb_tstamp_tx - queue clone of skb with send time stamps
+ * skb_tstamp_tx - queue clone of skb with send HARDWARE timestamps
  * @orig_skb:	the original outgoing packet
  * @hwtstamps:	hardware time stamps, may be NULL if not available
  *
  * If the skb has a socket associated, then this function clones the
  * skb (thus sharing the actual data and optional structures), stores
- * the optional hardware time stamping information (if non NULL) or
- * generates a software time stamp (otherwise), then queues the clone
- * to the error queue of the socket.  Errors are silently ignored.
+ * the optional hardware time stamping information (if non NULL) then
+ * queues the clone to the error queue of the socket.  Errors are
+ * silently ignored.
  */
 void skb_tstamp_tx(struct sk_buff *orig_skb,
 		   struct skb_shared_hwtstamps *hwtstamps);
@@ -4565,7 +4566,7 @@ static inline void skb_tx_timestamp(struct sk_buff *skb)
 {
 	skb_clone_tx_timestamp(skb);
 	if (skb_shinfo(skb)->tx_flags & SKBTX_SW_TSTAMP)
-		skb_tstamp_tx(skb, NULL);
+		__skb_tstamp_tx(skb, NULL, NULL, skb->sk, true, SCM_TSTAMP_SND);
 }
 
 /**
net/core/dev.c +1 -1
index afa2282f2604..d77b8389753e 100644
@@ -4501,7 +4501,7 @@ int __dev_queue_xmit(struct sk_buff *skb, struct net_device *sb_dev)
 	skb_assert_len(skb);
 
 	if (unlikely(skb_shinfo(skb)->tx_flags & SKBTX_SCHED_TSTAMP))
-		__skb_tstamp_tx(skb, NULL, NULL, skb->sk, SCM_TSTAMP_SCHED);
+		__skb_tstamp_tx(skb, NULL, NULL, skb->sk, true, SCM_TSTAMP_SCHED);
 
 	/* Disable soft irqs for various locks below. Also
 	 * stops preemption for RCU.
net/core/skbuff.c +30 -2
index a441613a1e6c..6042961dfc02 100644
@@ -5539,10 +5539,35 @@ void skb_complete_tx_timestamp(struct sk_buff *skb,
 }
 EXPORT_SYMBOL_GPL(skb_complete_tx_timestamp);
 
+static bool skb_enable_app_tstamp(struct sk_buff *skb, int tstype, bool sw)
+{
+	int flag;
+
+	switch (tstype) {
+	case SCM_TSTAMP_SCHED:
+		flag = SKBTX_SCHED_TSTAMP;
+		break;
+	case SCM_TSTAMP_SND:
+		flag = sw ? SKBTX_SW_TSTAMP : SKBTX_HW_TSTAMP;
+		break;
+	case SCM_TSTAMP_ACK:
+		if (TCP_SKB_CB(skb)->txstamp_ack)
+			return true;
+		fallthrough;
+	default:
+		return false;
+	}
+
+	if (skb_shinfo(skb)->tx_flags & flag)
+		return true;
+
+	return false;
+}
+
 void __skb_tstamp_tx(struct sk_buff *orig_skb,
 		     const struct sk_buff *ack_skb,
 		     struct skb_shared_hwtstamps *hwtstamps,
-		     struct sock *sk, int tstype)
+		     struct sock *sk, bool sw, int tstype)
 {
 	struct sk_buff *skb;
 	bool tsonly, opt_stats = false;
@@ -5551,6 +5576,9 @@ void __skb_tstamp_tx(struct sk_buff *orig_skb,
 	if (!sk)
 		return;
 
+	if (!skb_enable_app_tstamp(orig_skb, tstype, sw))
+		return;
+
 	tsflags = READ_ONCE(sk->sk_tsflags);
 	if (!hwtstamps && !(tsflags & SOF_TIMESTAMPING_OPT_TX_SWHW) &&
 	    skb_shinfo(orig_skb)->tx_flags & SKBTX_IN_PROGRESS)
@@ -5599,7 +5627,7 @@ EXPORT_SYMBOL_GPL(__skb_tstamp_tx);
 void skb_tstamp_tx(struct sk_buff *orig_skb,
 		   struct skb_shared_hwtstamps *hwtstamps)
 {
-	return __skb_tstamp_tx(orig_skb, NULL, hwtstamps, orig_skb->sk,
+	return __skb_tstamp_tx(orig_skb, NULL, hwtstamps, orig_skb->sk, false,
 			       SCM_TSTAMP_SND);
 }
 EXPORT_SYMBOL_GPL(skb_tstamp_tx);
net/ipv4/tcp_input.c +2 -1
index 77185479ed5e..62252702929d 100644
@@ -3330,7 +3330,8 @@ static void tcp_ack_tstamp(struct sock *sk, struct sk_buff *skb,
 	if (!before(shinfo->tskey, prior_snd_una) &&
 	    before(shinfo->tskey, tcp_sk(sk)->snd_una)) {
 		tcp_skb_tsorted_save(skb) {
-			__skb_tstamp_tx(skb, ack_skb, NULL, sk, SCM_TSTAMP_ACK);
+			__skb_tstamp_tx(skb, ack_skb, NULL, sk, true,
+					SCM_TSTAMP_ACK);
 		} tcp_skb_tsorted_restore(skb);
 	}
 }
Signature · 1 line
2.43.5
author
#8 lore raw

PATCH v7 6/13 net-timestamp: support SCM_TSTAMP_SCHED for bpf extension

In reply to Jason Xing

Introducing SKBTX_BPF is used as an indicator telling us whether the skb should be traced by the bpf prog.

---
 include/linux/skbuff.h         |  6 +++++-
 include/uapi/linux/bpf.h       |  4 ++++
 net/core/dev.c                 |  3 ++-
 net/core/skbuff.c              | 23 +++++++++++++++++++++++
 tools/include/uapi/linux/bpf.h |  4 ++++
 5 files changed, 38 insertions(+), 2 deletions(-)
include/linux/skbuff.h +5 -1
index dfc419281cc9..35c2e864dd4b 100644
@@ -490,10 +490,14 @@ enum {
 
 	/* generate software time stamp when entering packet scheduling */
 	SKBTX_SCHED_TSTAMP = 1 << 6,
+
+	/* used for bpf extension when a bpf program is loaded */
+	SKBTX_BPF = 1 << 7,
 };
 
 #define SKBTX_ANY_SW_TSTAMP	(SKBTX_SW_TSTAMP    | \
-				 SKBTX_SCHED_TSTAMP)
+				 SKBTX_SCHED_TSTAMP | \
+				 SKBTX_BPF)
 #define SKBTX_ANY_TSTAMP	(SKBTX_HW_TSTAMP | \
 				 SKBTX_HW_TSTAMP_USE_CYCLES | \
 				 SKBTX_ANY_SW_TSTAMP)
include/uapi/linux/bpf.h +4 -0
index 6116eb3d1515..30d2c078966b 100644
@@ -7032,6 +7032,10 @@ enum {
 					 * by the kernel or the
 					 * earlier bpf-progs.
 					 */
+	BPF_SOCK_OPS_TS_SCHED_OPT_CB,	/* Called when skb is passing through
+					 * dev layer when SK_BPF_CB_TX_TIMESTAMPING
+					 * feature is on.
+					 */
 };
 
 /* List of TCP states. There is a build check in net/ipv4/tcp.c to detect
net/core/dev.c +2 -1
index d77b8389753e..4f291459d6b1 100644
@@ -4500,7 +4500,8 @@ int __dev_queue_xmit(struct sk_buff *skb, struct net_device *sb_dev)
 	skb_reset_mac_header(skb);
 	skb_assert_len(skb);
 
-	if (unlikely(skb_shinfo(skb)->tx_flags & SKBTX_SCHED_TSTAMP))
+	if (unlikely(skb_shinfo(skb)->tx_flags &
+		     (SKBTX_SCHED_TSTAMP | SKBTX_BPF)))
 		__skb_tstamp_tx(skb, NULL, NULL, skb->sk, true, SCM_TSTAMP_SCHED);
 
 	/* Disable soft irqs for various locks below. Also
net/core/skbuff.c +23 -0
index 6042961dfc02..d19d577b996f 100644
@@ -5564,6 +5564,24 @@ static bool skb_enable_app_tstamp(struct sk_buff *skb, int tstype, bool sw)
 	return false;
 }
 
+static void skb_tstamp_tx_bpf(struct sk_buff *skb, struct sock *sk, int tstype)
+{
+	int op;
+
+	if (!sk)
+		return;
+
+	switch (tstype) {
+	case SCM_TSTAMP_SCHED:
+		op = BPF_SOCK_OPS_TS_SCHED_OPT_CB;
+		break;
+	default:
+		return;
+	}
+
+	bpf_skops_tx_timestamping(sk, skb, op);
+}
+
 void __skb_tstamp_tx(struct sk_buff *orig_skb,
 		     const struct sk_buff *ack_skb,
 		     struct skb_shared_hwtstamps *hwtstamps,
@@ -5576,6 +5594,11 @@ void __skb_tstamp_tx(struct sk_buff *orig_skb,
 	if (!sk)
 		return;
 
+	/* bpf extension feature entry */
+	if (skb_shinfo(orig_skb)->tx_flags & SKBTX_BPF)
+		skb_tstamp_tx_bpf(orig_skb, sk, tstype);
+
+	/* application feature entry */
 	if (!skb_enable_app_tstamp(orig_skb, tstype, sw))
 		return;
 
tools/include/uapi/linux/bpf.h +4 -0
index 70366f74ef4e..eed91b7296b7 100644
@@ -7025,6 +7025,10 @@ enum {
 					 * by the kernel or the
 					 * earlier bpf-progs.
 					 */
+	BPF_SOCK_OPS_TS_SCHED_OPT_CB,	/* Called when skb is passing through
+					 * dev layer when SK_BPF_CB_TX_TIMESTAMPING
+					 * feature is on.
+					 */
 };
 
 /* List of TCP states. There is a build check in net/ipv4/tcp.c to detect
Signature · 1 line
2.43.5
author
#11 lore raw

PATCH v7 7/13 net-timestamp: support sw SCM_TSTAMP_SND for bpf extension

In reply to Jason Xing

Support SCM_TSTAMP_SND case. Then we will get the software timestamp when the driver is about to send the skb. Later, I will support the hardware timestamp.

---
 include/linux/skbuff.h         |  2 +-
 include/uapi/linux/bpf.h       |  4 ++++
 net/core/skbuff.c              | 10 ++++++++--
 tools/include/uapi/linux/bpf.h |  4 ++++
 4 files changed, 17 insertions(+), 3 deletions(-)
include/linux/skbuff.h +1 -1
index 35c2e864dd4b..de8d3bd311f5 100644
@@ -4569,7 +4569,7 @@ void skb_tstamp_tx(struct sk_buff *orig_skb,
 static inline void skb_tx_timestamp(struct sk_buff *skb)
 {
 	skb_clone_tx_timestamp(skb);
-	if (skb_shinfo(skb)->tx_flags & SKBTX_SW_TSTAMP)
+	if (skb_shinfo(skb)->tx_flags & (SKBTX_SW_TSTAMP | SKBTX_BPF))
 		__skb_tstamp_tx(skb, NULL, NULL, skb->sk, true, SCM_TSTAMP_SND);
 }
 
include/uapi/linux/bpf.h +4 -0
index 30d2c078966b..6a1083bcf779 100644
@@ -7036,6 +7036,10 @@ enum {
 					 * dev layer when SK_BPF_CB_TX_TIMESTAMPING
 					 * feature is on.
 					 */
+	BPF_SOCK_OPS_TS_SW_OPT_CB,	/* Called when skb is about to send
+					 * to the nic when SK_BPF_CB_TX_TIMESTAMPING
+					 * feature is on.
+					 */
 };
 
 /* List of TCP states. There is a build check in net/ipv4/tcp.c to detect
net/core/skbuff.c +8 -2
index d19d577b996f..288eb9869827 100644
@@ -5564,7 +5564,8 @@ static bool skb_enable_app_tstamp(struct sk_buff *skb, int tstype, bool sw)
 	return false;
 }
 
-static void skb_tstamp_tx_bpf(struct sk_buff *skb, struct sock *sk, int tstype)
+static void skb_tstamp_tx_bpf(struct sk_buff *skb, struct sock *sk,
+			      int tstype, bool sw)
 {
 	int op;
 
@@ -5575,6 +5576,11 @@ static void skb_tstamp_tx_bpf(struct sk_buff *skb, struct sock *sk, int tstype)
 	case SCM_TSTAMP_SCHED:
 		op = BPF_SOCK_OPS_TS_SCHED_OPT_CB;
 		break;
+	case SCM_TSTAMP_SND:
+		if (!sw)
+			return;
+		op = BPF_SOCK_OPS_TS_SW_OPT_CB;
+		break;
 	default:
 		return;
 	}
@@ -5596,7 +5602,7 @@ void __skb_tstamp_tx(struct sk_buff *orig_skb,
 
 	/* bpf extension feature entry */
 	if (skb_shinfo(orig_skb)->tx_flags & SKBTX_BPF)
-		skb_tstamp_tx_bpf(orig_skb, sk, tstype);
+		skb_tstamp_tx_bpf(orig_skb, sk, tstype, sw);
 
 	/* application feature entry */
 	if (!skb_enable_app_tstamp(orig_skb, tstype, sw))
tools/include/uapi/linux/bpf.h +4 -0
index eed91b7296b7..9bd1c7c77b17 100644
@@ -7029,6 +7029,10 @@ enum {
 					 * dev layer when SK_BPF_CB_TX_TIMESTAMPING
 					 * feature is on.
 					 */
+	BPF_SOCK_OPS_TS_SW_OPT_CB,	/* Called when skb is about to send
+					 * to the nic when SK_BPF_CB_TX_TIMESTAMPING
+					 * feature is on.
+					 */
 };
 
 /* List of TCP states. There is a build check in net/ipv4/tcp.c to detect
Signature · 1 line
2.43.5
author
#12 lore raw

PATCH v7 8/13 net-timestamp: support hw SCM_TSTAMP_SND for bpf extension

In reply to Jason Xing

In this patch, we finish the hardware part. Then bpf program can fetch the hwstamp from skb directly.

To avoid changing so many callers using SKBTX_HW_TSTAMP from drivers, use this simple modification like this patch does to support printing hardware timestamp.

---
 include/linux/skbuff.h         |  4 +++-
 include/uapi/linux/bpf.h       |  7 +++++++
 net/core/skbuff.c              | 11 ++++++-----
 net/dsa/user.c                 |  2 +-
 net/socket.c                   |  2 +-
 tools/include/uapi/linux/bpf.h |  7 +++++++
 6 files changed, 25 insertions(+), 8 deletions(-)
include/linux/skbuff.h +3 -1
index de8d3bd311f5..df2d790ae36b 100644
@@ -471,7 +471,7 @@ struct skb_shared_hwtstamps {
 /* Definitions for tx_flags in struct skb_shared_info */
 enum {
 	/* generate hardware time stamp */
-	SKBTX_HW_TSTAMP = 1 << 0,
+	__SKBTX_HW_TSTAMP = 1 << 0,
 
 	/* generate software time stamp when queueing packet to NIC */
 	SKBTX_SW_TSTAMP = 1 << 1,
@@ -495,6 +495,8 @@ enum {
 	SKBTX_BPF = 1 << 7,
 };
 
+#define SKBTX_HW_TSTAMP		(__SKBTX_HW_TSTAMP | SKBTX_BPF)
+
 #define SKBTX_ANY_SW_TSTAMP	(SKBTX_SW_TSTAMP    | \
 				 SKBTX_SCHED_TSTAMP | \
 				 SKBTX_BPF)
include/uapi/linux/bpf.h +7 -0
index 6a1083bcf779..4c3566f623c2 100644
@@ -7040,6 +7040,13 @@ enum {
 					 * to the nic when SK_BPF_CB_TX_TIMESTAMPING
 					 * feature is on.
 					 */
+	BPF_SOCK_OPS_TS_HW_OPT_CB,	/* Called in hardware phase when
+					 * SK_BPF_CB_TX_TIMESTAMPING feature
+					 * is on. At the same time, hwtstamps
+					 * of skb is initialized as the
+					 * timestamp that hardware just
+					 * generates.
+					 */
 };
 
 /* List of TCP states. There is a build check in net/ipv4/tcp.c to detect
net/core/skbuff.c +6 -5
index 288eb9869827..c769feae5162 100644
@@ -5548,7 +5548,7 @@ static bool skb_enable_app_tstamp(struct sk_buff *skb, int tstype, bool sw)
 		flag = SKBTX_SCHED_TSTAMP;
 		break;
 	case SCM_TSTAMP_SND:
-		flag = sw ? SKBTX_SW_TSTAMP : SKBTX_HW_TSTAMP;
+		flag = sw ? SKBTX_SW_TSTAMP : __SKBTX_HW_TSTAMP;
 		break;
 	case SCM_TSTAMP_ACK:
 		if (TCP_SKB_CB(skb)->txstamp_ack)
@@ -5565,7 +5565,8 @@ static bool skb_enable_app_tstamp(struct sk_buff *skb, int tstype, bool sw)
 }
 
 static void skb_tstamp_tx_bpf(struct sk_buff *skb, struct sock *sk,
-			      int tstype, bool sw)
+			      int tstype, bool sw,
+			      struct skb_shared_hwtstamps *hwtstamps)
 {
 	int op;
 
@@ -5577,9 +5578,9 @@ static void skb_tstamp_tx_bpf(struct sk_buff *skb, struct sock *sk,
 		op = BPF_SOCK_OPS_TS_SCHED_OPT_CB;
 		break;
 	case SCM_TSTAMP_SND:
+		op = sw ? BPF_SOCK_OPS_TS_SW_OPT_CB : BPF_SOCK_OPS_TS_HW_OPT_CB;
 		if (!sw)
-			return;
-		op = BPF_SOCK_OPS_TS_SW_OPT_CB;
+			*skb_hwtstamps(skb) = *hwtstamps;
 		break;
 	default:
 		return;
@@ -5602,7 +5603,7 @@ void __skb_tstamp_tx(struct sk_buff *orig_skb,
 
 	/* bpf extension feature entry */
 	if (skb_shinfo(orig_skb)->tx_flags & SKBTX_BPF)
-		skb_tstamp_tx_bpf(orig_skb, sk, tstype, sw);
+		skb_tstamp_tx_bpf(orig_skb, sk, tstype, sw, hwtstamps);
 
 	/* application feature entry */
 	if (!skb_enable_app_tstamp(orig_skb, tstype, sw))
net/dsa/user.c +1 -1
index 291ab1b4acc4..ae715bf0ae75 100644
@@ -897,7 +897,7 @@ static void dsa_skb_tx_timestamp(struct dsa_user_priv *p,
 {
 	struct dsa_switch *ds = p->dp->ds;
 
-	if (!(skb_shinfo(skb)->tx_flags & SKBTX_HW_TSTAMP))
+	if (!(skb_shinfo(skb)->tx_flags & __SKBTX_HW_TSTAMP))
 		return;
 
 	if (!ds->ops->port_txtstamp)
net/socket.c +1 -1
index 262a28b59c7f..70eabb510ce6 100644
@@ -676,7 +676,7 @@ void __sock_tx_timestamp(__u32 tsflags, __u8 *tx_flags)
 	u8 flags = *tx_flags;
 
 	if (tsflags & SOF_TIMESTAMPING_TX_HARDWARE) {
-		flags |= SKBTX_HW_TSTAMP;
+		flags |= __SKBTX_HW_TSTAMP;
 
 		/* PTP hardware clocks can provide a free running cycle counter
 		 * as a time base for virtual clocks. Tell driver to use the
tools/include/uapi/linux/bpf.h +7 -0
index 9bd1c7c77b17..974b7f61d11f 100644
@@ -7033,6 +7033,13 @@ enum {
 					 * to the nic when SK_BPF_CB_TX_TIMESTAMPING
 					 * feature is on.
 					 */
+	BPF_SOCK_OPS_TS_HW_OPT_CB,	/* Called in hardware phase when
+					 * SK_BPF_CB_TX_TIMESTAMPING feature
+					 * is on. At the same time, hwtstamps
+					 * of skb is initialized as the
+					 * timestamp that hardware just
+					 * generates.
+					 */
 };
 
 /* List of TCP states. There is a build check in net/ipv4/tcp.c to detect
Signature · 1 line
2.43.5
author
#15 lore raw

PATCH v7 9/13 net-timestamp: support SCM_TSTAMP_ACK for bpf extension

In reply to Jason Xing

Handle the ACK timestamp case. Actually testing SKBTX_BPF flag can work, but we need to Introduce a new txstamp_ack_bpf to avoid cache line misses in tcp_ack_tstamp(). To be more specific, in most cases, normal flows would not access skb_shinfo as txstamp_ack is zero, so that this function won't appear in the hot spot lists. Introducing a new member txstamp_ack_bpf works similarly.

---
 include/net/tcp.h              | 3 ++-
 include/uapi/linux/bpf.h       | 5 +++++
 net/core/skbuff.c              | 3 +++
 net/ipv4/tcp_input.c           | 3 ++-
 net/ipv4/tcp_output.c          | 5 +++++
 tools/include/uapi/linux/bpf.h | 5 +++++
 6 files changed, 22 insertions(+), 2 deletions(-)
include/net/tcp.h +2 -1
index 293047694710..88429e422301 100644
@@ -959,9 +959,10 @@ struct tcp_skb_cb {
 	__u8		sacked;		/* State flags for SACK.	*/
 	__u8		ip_dsfield;	/* IPv4 tos or IPv6 dsfield	*/
 	__u8		txstamp_ack:1,	/* Record TX timestamp for ack? */
+			txstamp_ack_bpf:1,	/* ack timestamp for bpf use */
 			eor:1,		/* Is skb MSG_EOR marked? */
 			has_rxtstamp:1,	/* SKB has a RX timestamp	*/
-			unused:5;
+			unused:4;
 	__u32		ack_seq;	/* Sequence number ACK'd	*/
 	union {
 		struct {
include/uapi/linux/bpf.h +5 -0
index 4c3566f623c2..800122a8abe5 100644
@@ -7047,6 +7047,11 @@ enum {
 					 * timestamp that hardware just
 					 * generates.
 					 */
+	BPF_SOCK_OPS_TS_ACK_OPT_CB,	/* Called when all the skbs in the
+					 * same sendmsg call are acked
+					 * when SK_BPF_CB_TX_TIMESTAMPING
+					 * feature is on.
+					 */
 };
 
 /* List of TCP states. There is a build check in net/ipv4/tcp.c to detect
net/core/skbuff.c +3 -0
index c769feae5162..33340e0b094f 100644
@@ -5582,6 +5582,9 @@ static void skb_tstamp_tx_bpf(struct sk_buff *skb, struct sock *sk,
 		if (!sw)
 			*skb_hwtstamps(skb) = *hwtstamps;
 		break;
+	case SCM_TSTAMP_ACK:
+		op = BPF_SOCK_OPS_TS_ACK_OPT_CB;
+		break;
 	default:
 		return;
 	}
net/ipv4/tcp_input.c +2 -1
index 62252702929d..c8945f5be31b 100644
@@ -3323,7 +3323,8 @@ static void tcp_ack_tstamp(struct sock *sk, struct sk_buff *skb,
 	const struct skb_shared_info *shinfo;
 
 	/* Avoid cache line misses to get skb_shinfo() and shinfo->tx_flags */
-	if (likely(!TCP_SKB_CB(skb)->txstamp_ack))
+	if (likely(!TCP_SKB_CB(skb)->txstamp_ack &&
+		   !TCP_SKB_CB(skb)->txstamp_ack_bpf))
 		return;
 
 	shinfo = skb_shinfo(skb);
net/ipv4/tcp_output.c +5 -0
index 695749807c09..fc84ca669b76 100644
@@ -1556,6 +1556,7 @@ static void tcp_adjust_pcount(struct sock *sk, const struct sk_buff *skb, int de
 static bool tcp_has_tx_tstamp(const struct sk_buff *skb)
 {
 	return TCP_SKB_CB(skb)->txstamp_ack ||
+	       TCP_SKB_CB(skb)->txstamp_ack_bpf ||
 		(skb_shinfo(skb)->tx_flags & SKBTX_ANY_TSTAMP);
 }
 
@@ -1572,7 +1573,9 @@ static void tcp_fragment_tstamp(struct sk_buff *skb, struct sk_buff *skb2)
 		shinfo2->tx_flags |= tsflags;
 		swap(shinfo->tskey, shinfo2->tskey);
 		TCP_SKB_CB(skb2)->txstamp_ack = TCP_SKB_CB(skb)->txstamp_ack;
+		TCP_SKB_CB(skb2)->txstamp_ack_bpf = TCP_SKB_CB(skb)->txstamp_ack_bpf;
 		TCP_SKB_CB(skb)->txstamp_ack = 0;
+		TCP_SKB_CB(skb)->txstamp_ack_bpf = 0;
 	}
 }
 
@@ -3213,6 +3216,8 @@ void tcp_skb_collapse_tstamp(struct sk_buff *skb,
 		shinfo->tskey = next_shinfo->tskey;
 		TCP_SKB_CB(skb)->txstamp_ack |=
 			TCP_SKB_CB(next_skb)->txstamp_ack;
+		TCP_SKB_CB(skb)->txstamp_ack_bpf |=
+			TCP_SKB_CB(next_skb)->txstamp_ack_bpf;
 	}
 }
 
tools/include/uapi/linux/bpf.h +5 -0
index 974b7f61d11f..06e68d772989 100644
@@ -7040,6 +7040,11 @@ enum {
 					 * timestamp that hardware just
 					 * generates.
 					 */
+	BPF_SOCK_OPS_TS_ACK_OPT_CB,	/* Called when all the skbs in the
+					 * same sendmsg call are acked
+					 * when SK_BPF_CB_TX_TIMESTAMPING
+					 * feature is on.
+					 */
 };
 
 /* List of TCP states. There is a build check in net/ipv4/tcp.c to detect
Signature · 1 line
2.43.5
author
#16 lore raw

PATCH v7 10/13 net-timestamp: make TCP tx timestamp bpf extension work

In reply to Jason Xing

Make partial of the feature work finally. After this, user can fully use the bpf prog to trace the tx path for TCP type.

---
 net/ipv4/tcp.c | 9 +++++++++
 1 file changed, 9 insertions(+)
net/ipv4/tcp.c +9 -0
index 0d704bda6c41..0a41006b10d1 100644
@@ -492,6 +492,15 @@ static void tcp_tx_timestamp(struct sock *sk, struct sockcm_cookie *sockc)
 		if (tsflags & SOF_TIMESTAMPING_TX_RECORD_MASK)
 			shinfo->tskey = TCP_SKB_CB(skb)->seq + skb->len - 1;
 	}
+
+	if (SK_BPF_CB_FLAG_TEST(sk, SK_BPF_CB_TX_TIMESTAMPING) && skb) {
+		struct skb_shared_info *shinfo = skb_shinfo(skb);
+		struct tcp_skb_cb *tcb = TCP_SKB_CB(skb);
+
+		tcb->txstamp_ack_bpf = 1;
+		shinfo->tx_flags |= SKBTX_BPF;
+		shinfo->tskey = TCP_SKB_CB(skb)->seq + skb->len - 1;
+	}
 }
 
 static bool tcp_stream_is_readable(struct sock *sk, int target)
Signature · 1 line
2.43.5
author
#19 lore raw

PATCH v7 11/13 net-timestamp: add a new callback in tcp_tx_timestamp()

In reply to Jason Xing

Introduce the callback to correlate tcp_sendmsg timestamp with other points, like SND/SW/ACK. We can let bpf trace the beginning of tcp_sendmsg_locked() and fetch the socket addr, so that in tcp_tx_timestamp() we can correlate the tskey with the socket addr. It is accurate since they are under the protect of socket lock. More details can be found in the selftest.

---
 include/uapi/linux/bpf.h       | 7 +++++++
 net/ipv4/tcp.c                 | 1 +
 tools/include/uapi/linux/bpf.h | 7 +++++++
 3 files changed, 15 insertions(+)
include/uapi/linux/bpf.h +7 -0
index 800122a8abe5..accb3b314fff 100644
@@ -7052,6 +7052,13 @@ enum {
 					 * when SK_BPF_CB_TX_TIMESTAMPING
 					 * feature is on.
 					 */
+	BPF_SOCK_OPS_TS_SND_CB,		/* Called when every sendmsg syscall
+					 * is triggered. For TCP, it stays
+					 * in the last send process to
+					 * correlate with tcp_sendmsg timestamp
+					 * with other timestamping callbacks,
+					 * like SND/SW/ACK.
+					 */
 };
 
 /* List of TCP states. There is a build check in net/ipv4/tcp.c to detect
net/ipv4/tcp.c +1 -0
index 0a41006b10d1..b2f1fd216df1 100644
@@ -500,6 +500,7 @@ static void tcp_tx_timestamp(struct sock *sk, struct sockcm_cookie *sockc)
 		tcb->txstamp_ack_bpf = 1;
 		shinfo->tx_flags |= SKBTX_BPF;
 		shinfo->tskey = TCP_SKB_CB(skb)->seq + skb->len - 1;
+		bpf_skops_tx_timestamping(sk, skb, BPF_SOCK_OPS_TS_SND_CB);
 	}
 }
 
tools/include/uapi/linux/bpf.h +7 -0
index 06e68d772989..384502996cdd 100644
@@ -7045,6 +7045,13 @@ enum {
 					 * when SK_BPF_CB_TX_TIMESTAMPING
 					 * feature is on.
 					 */
+	BPF_SOCK_OPS_TS_SND_CB,		/* Called when every sendmsg syscall
+					 * is triggered. For TCP, it stays
+					 * in the last send process to
+					 * correlate with tcp_sendmsg timestamp
+					 * with other timestamping callbacks,
+					 * like SND/SW/ACK.
+					 */
 };
 
 /* List of TCP states. There is a build check in net/ipv4/tcp.c to detect
Signature · 1 line
2.43.5
author
#34 lore raw

PATCH v7 12/13 net-timestamp: introduce cgroup lock to avoid affecting non-bpf cases

In reply to Jason Xing

Introducing the lock to avoid affecting the applications which are not using timestamping bpf feature.

---
 net/ipv4/tcp.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)
net/ipv4/tcp.c +2 -1
index b2f1fd216df1..a2ac57543b6d 100644
@@ -493,7 +493,8 @@ static void tcp_tx_timestamp(struct sock *sk, struct sockcm_cookie *sockc)
 			shinfo->tskey = TCP_SKB_CB(skb)->seq + skb->len - 1;
 	}
 
-	if (SK_BPF_CB_FLAG_TEST(sk, SK_BPF_CB_TX_TIMESTAMPING) && skb) {
+	if (cgroup_bpf_enabled(CGROUP_SOCK_OPS) &&
+	    SK_BPF_CB_FLAG_TEST(sk, SK_BPF_CB_TX_TIMESTAMPING) && skb) {
 		struct skb_shared_info *shinfo = skb_shinfo(skb);
 		struct tcp_skb_cb *tcb = TCP_SKB_CB(skb);
 
Signature · 1 line
2.43.5
author
#37 lore raw

PATCH v7 13/13 bpf: add simple bpf tests in the tx path for so_timestamping feature

In reply to Jason Xing

Only check if we pass those three key points after we enable the bpf extension for so_timestamping. During each point, we can choose whether to print the current timestamp.

---
 .../bpf/prog_tests/so_timestamping.c          |  86 +++++
 .../selftests/bpf/progs/so_timestamping.c     | 299 ++++++++++++++++++
 2 files changed, 385 insertions(+)
 create mode 100644 tools/testing/selftests/bpf/prog_tests/so_timestamping.c
 create mode 100644 tools/testing/selftests/bpf/progs/so_timestamping.c
tools/testing/selftests/bpf/prog_tests/so_timestamping.c new file +86 -0
new file mode 100644
index 000000000000..ee7fdc381609
@@ -0,0 +1,86 @@
+#define _GNU_SOURCE
+#include <sched.h>
+#include <linux/socket.h>
+#include <linux/tls.h>
+#include <net/if.h>
+
+#include "test_progs.h"
+#include "cgroup_helpers.h"
+#include "network_helpers.h"
+
+#include "so_timestamping.skel.h"
+
+#define CG_NAME "/so-timestamping-test"
+
+static const char addr4_str[] = "127.0.0.1";
+static const char addr6_str[] = "::1";
+static struct so_timestamping *skel;
+static int cg_fd;
+
+static void test_tcp(int family)
+{
+	struct so_timestamping__bss *bss = skel->bss;
+	char buf[] = "testing testing";
+	int sfd = -1, cfd = -1;
+	int n;
+
+	memset(bss, 0, sizeof(*bss));
+
+	sfd = start_server(family, SOCK_STREAM,
+			   family == AF_INET6 ? addr6_str : addr4_str, 0, 0);
+	if (!ASSERT_OK_FD(sfd, "start_server"))
+		goto out;
+
+	cfd = connect_to_fd(sfd, 0);
+	if (!ASSERT_OK_FD(cfd, "connect_to_fd_server"))
+		goto out;
+
+	n = write(cfd, buf, sizeof(buf));
+	if (!ASSERT_EQ(n, sizeof(buf), "send to server"))
+		goto out;
+
+	ASSERT_EQ(bss->nr_active, 1, "nr_active");
+	ASSERT_EQ(bss->nr_snd, 2, "nr_snd");
+	ASSERT_EQ(bss->nr_sched, 1, "nr_sched");
+	ASSERT_EQ(bss->nr_txsw, 1, "nr_txsw");
+	ASSERT_EQ(bss->nr_ack, 1, "nr_ack");
+
+out:
+	if (sfd >= 0)
+		close(sfd);
+	if (cfd >= 0)
+		close(cfd);
+}
+
+void test_so_timestamping(void)
+{
+	struct netns_obj *ns;
+
+	cg_fd = test__join_cgroup(CG_NAME);
+	if (cg_fd < 0)
+		return;
+
+	ns = netns_new("so_timestamping_ns", true);
+	if (!ASSERT_OK_PTR(ns, "create ns"))
+		return;
+
+	skel = so_timestamping__open_and_load();
+	if (!ASSERT_OK_PTR(skel, "open and load skel"))
+		goto done;
+
+	if (!ASSERT_OK(so_timestamping__attach(skel), "attach skel"))
+		goto done;
+
+	skel->links.skops_sockopt =
+		bpf_program__attach_cgroup(skel->progs.skops_sockopt, cg_fd);
+	if (!ASSERT_OK_PTR(skel->links.skops_sockopt, "attach cgroup"))
+		goto done;
+
+	test_tcp(AF_INET6);
+	test_tcp(AF_INET);
+
+done:
+	so_timestamping__destroy(skel);
+	netns_free(ns);
+	close(cg_fd);
+}
tools/testing/selftests/bpf/progs/so_timestamping.c new file +299 -0

299 lines not on this page · raw message

Signature · 1 line
2.43.5
#6 lore raw

Re: [PATCH bpf-next v7 05/13] net-timestamp: prepare for isolating two modes of SO_TIMESTAMPING

In reply to Jason Xing
Quoted 3 lines · Jason Xing

No functional changes here. I add skb_enable_app_tstamp() to test if the orig_skb matches the usage of application SO_TIMESTAMPING and skb_sw_tstamp_tx() to distinguish the software and hardware

There is no skb_sw_tstamp_tx() in the code. An outdated commit message?

Quoted 11 lines

timestamp when tsflag is SCM_TSTAMP_SND.

Also, I deliberately distinguish the the software and hardware SCM_TSTAMP_SND timestamp by passing 'sw' parameter in order to avoid such a case where hardware may go wrong and pass a NULL hwstamps, which is even though unlikely to happen. If it really happens, bpf prog will finally consider it as a software timestamp. It will be hardly recognized. Let's make the timestamping part more robust.

After this patch, I will soon add checks about bpf SO_TIMESTAMPING.

This needs to be updated also. BPF does not use the SO_TIMESTAMPING socket option.

Quoted 1 line

In this way, we can support two modes parallelly.

s/parallely/in parallel/

Quoted 28 lines
---
  include/linux/skbuff.h | 13 +++++++------
  net/core/dev.c         |  2 +-
  net/core/skbuff.c      | 32 ++++++++++++++++++++++++++++++--
  net/ipv4/tcp_input.c   |  3 ++-
  4 files changed, 40 insertions(+), 10 deletions(-)
include/linux/skbuff.h +2 -1
index bb2b751d274a..dfc419281cc9 100644
@@ -39,6 +39,7 @@
  #include <net/net_debug.h>
  #include <net/dropreason-core.h>
  #include <net/netmem.h>
+#include <uapi/linux/errqueue.h>
 
  /**
   * DOC: skb checksums
@@ -4533,18 +4534,18 @@ void skb_complete_tx_timestamp(struct sk_buff *skb,
 
  void __skb_tstamp_tx(struct sk_buff *orig_skb, const struct sk_buff *ack_skb,
  		     struct skb_shared_hwtstamps *hwtstamps,
-		     struct sock *sk, int tstype);
+		     struct sock *sk, bool sw, int tstype);
 

include/linux/skbuff.h void skb_complete_tx_timestamp(struct sk_buff *skb, · line 4546

Quoted 10 lines
include/linux/skbuff.h +1 -3
  /**
- * skb_tstamp_tx - queue clone of skb with send time stamps
+ * skb_tstamp_tx - queue clone of skb with send HARDWARE timestamps
   * @orig_skb:	the original outgoing packet
   * @hwtstamps:	hardware time stamps, may be NULL if not available
   *
   * If the skb has a socket associated, then this function clones the
   * skb (thus sharing the actual data and optional structures), stores
- * the optional hardware time stamping information (if non NULL) or
- * generates a software time stamp (otherwise), then queues the clone

This line is removed. Does it mean no software timestamp now after this change?

Quoted 14 lines
include/linux/skbuff.h +4 -2
- * to the error queue of the socket.  Errors are silently ignored.
+ * the optional hardware time stamping information (if non NULL) then
+ * queues the clone to the error queue of the socket.  Errors are
+ * silently ignored.
   */
  void skb_tstamp_tx(struct sk_buff *orig_skb,
  		   struct skb_shared_hwtstamps *hwtstamps);
@@ -4565,7 +4566,7 @@ static inline void skb_tx_timestamp(struct sk_buff *skb)
  {
  	skb_clone_tx_timestamp(skb);
  	if (skb_shinfo(skb)->tx_flags & SKBTX_SW_TSTAMP)
-		skb_tstamp_tx(skb, NULL);
+		__skb_tstamp_tx(skb, NULL, NULL, skb->sk, true, SCM_TSTAMP_SND);
  }
#9 lore raw

Re: [PATCH bpf-next v7 06/13] net-timestamp: support SCM_TSTAMP_SCHED for bpf extension

In reply to Jason Xing

net/core/skbuff.c static bool skb_enable_app_tstamp(struct sk_buff *skb, int tstype, bool sw) · line 5571

Quoted 13 lines · Jason Xing
net/core/skbuff.c +5 -0
index 6042961dfc02..d19d577b996f 100644
@@ -5564,6 +5564,24 @@ static bool skb_enable_app_tstamp(struct sk_buff *skb, int tstype, bool sw)
  	return false;
  }
 
+static void skb_tstamp_tx_bpf(struct sk_buff *skb, struct sock *sk, int tstype)
+{
+	int op;
+
+	if (!sk)

This check is redundant.

Quoted 17 lines
net/core/skbuff.c +13 -0
+		return;
+
+	switch (tstype) {
+	case SCM_TSTAMP_SCHED:
+		op = BPF_SOCK_OPS_TS_SCHED_OPT_CB;
+		break;
+	default:
+		return;
+	}
+
+	bpf_skops_tx_timestamping(sk, skb, op);
+}
+
  void __skb_tstamp_tx(struct sk_buff *orig_skb,
  		     const struct sk_buff *ack_skb,
  		     struct skb_shared_hwtstamps *hwtstamps,

net/core/skbuff.c void __skb_tstamp_tx(struct sk_buff *orig_skb, · line 5594

Quoted 1 line
net/core/skbuff.c +0 -0
@@ -5576,6 +5594,11 @@ void __skb_tstamp_tx(struct sk_buff *orig_skb,
  	if (!sk)

It has been tested here...

net/core/skbuff.c void __skb_tstamp_tx(struct sk_buff *orig_skb, · line 5599

Quoted 5 lines
net/core/skbuff.c +3 -0
  		return;
 
+	/* bpf extension feature entry */
+	if (skb_shinfo(orig_skb)->tx_flags & SKBTX_BPF)
+		skb_tstamp_tx_bpf(orig_skb, sk, tstype);

...before calling this.

Quoted 4 lines
net/core/skbuff.c +2 -0
+
+	/* application feature entry */
  	if (!skb_enable_app_tstamp(orig_skb, tstype, sw))
  		return;
author
#7 lore raw

Re: [PATCH bpf-next v7 05/13] net-timestamp: prepare for isolating two modes of SO_TIMESTAMPING

In reply to Martin KaFai Lau
Quoted 7 lines · Martin KaFai Lau
Quoted 3 lines · Jason Xing

No functional changes here. I add skb_enable_app_tstamp() to test if the orig_skb matches the usage of application SO_TIMESTAMPING and skb_sw_tstamp_tx() to distinguish the software and hardware

There is no skb_sw_tstamp_tx() in the code. An outdated commit message?

Thanks. I'll update it and double check before reposting.

Quoted 11 lines
Quoted 11 lines

timestamp when tsflag is SCM_TSTAMP_SND.

Also, I deliberately distinguish the the software and hardware SCM_TSTAMP_SND timestamp by passing 'sw' parameter in order to avoid such a case where hardware may go wrong and pass a NULL hwstamps, which is even though unlikely to happen. If it really happens, bpf prog will finally consider it as a software timestamp. It will be hardly recognized. Let's make the timestamping part more robust.

Quoted 7 lines
Quoted 1 line

After this patch, I will soon add checks about bpf SO_TIMESTAMPING.

This needs to be updated also. BPF does not use the SO_TIMESTAMPING socket option.

Quoted 1 line

In this way, we can support two modes parallelly.

s/parallely/in parallel/

Will fix it.

Quoted 29 lines
Quoted 29 lines
---
  include/linux/skbuff.h | 13 +++++++------
  net/core/dev.c         |  2 +-
  net/core/skbuff.c      | 32 ++++++++++++++++++++++++++++++--
  net/ipv4/tcp_input.c   |  3 ++-
  4 files changed, 40 insertions(+), 10 deletions(-)
include/linux/skbuff.h +2 -1
index bb2b751d274a..dfc419281cc9 100644
@@ -39,6 +39,7 @@
  #include <net/net_debug.h>
  #include <net/dropreason-core.h>
  #include <net/netmem.h>
+#include <uapi/linux/errqueue.h>
 
  /**
   * DOC: skb checksums
@@ -4533,18 +4534,18 @@ void skb_complete_tx_timestamp(struct sk_buff *skb,
 
  void __skb_tstamp_tx(struct sk_buff *orig_skb, const struct sk_buff *ack_skb,
                   struct skb_shared_hwtstamps *hwtstamps,
-                  struct sock *sk, int tstype);
+                  struct sock *sk, bool sw, int tstype);
 
Quoted 12 lines

include/linux/skbuff.h void skb_complete_tx_timestamp(struct sk_buff *skb, · line 4546

Quoted 10 lines
include/linux/skbuff.h +1 -3
  /**
- * skb_tstamp_tx - queue clone of skb with send time stamps
+ * skb_tstamp_tx - queue clone of skb with send HARDWARE timestamps
   * @orig_skb:       the original outgoing packet
   * @hwtstamps:      hardware time stamps, may be NULL if not available
   *
   * If the skb has a socket associated, then this function clones the
   * skb (thus sharing the actual data and optional structures), stores
- * the optional hardware time stamping information (if non NULL) or
- * generates a software time stamp (otherwise), then queues the clone

This line is removed. Does it mean no software timestamp now after this change?

Right, _software_ timestamp will enter skb_tx_timestamp() then call __skb_tstamp_tx() instead of this skb_tx_timestamp().

Quoted 17 lines
Quoted 15 lines
include/linux/skbuff.h +4 -2
- * to the error queue of the socket.  Errors are silently ignored.
+ * the optional hardware time stamping information (if non NULL) then
+ * queues the clone to the error queue of the socket.  Errors are
+ * silently ignored.
   */
  void skb_tstamp_tx(struct sk_buff *orig_skb,
                 struct skb_shared_hwtstamps *hwtstamps);
@@ -4565,7 +4566,7 @@ static inline void skb_tx_timestamp(struct sk_buff *skb)
  {
      skb_clone_tx_timestamp(skb);
      if (skb_shinfo(skb)->tx_flags & SKBTX_SW_TSTAMP)
-             skb_tstamp_tx(skb, NULL);
+             __skb_tstamp_tx(skb, NULL, NULL, skb->sk, true, SCM_TSTAMP_SND);
  }
author
#10 lore raw

Re: [PATCH bpf-next v7 06/13] net-timestamp: support SCM_TSTAMP_SCHED for bpf extension

In reply to Martin KaFai Lau
Quoted 10 lines · Martin KaFai Lau
Quoted 8 lines · Jason Xing
net/core/skbuff.c +0 -0
index 6042961dfc02..d19d577b996f 100644
@@ -5564,6 +5564,24 @@ static bool skb_enable_app_tstamp(struct sk_buff *skb, int tstype, bool sw)
      return false;
  }
 
Quoted 7 lines

net/core/skbuff.c static bool skb_enable_app_tstamp(struct sk_buff *skb, int tstype, bool sw) · line 5571

Quoted 5 lines
net/core/skbuff.c +5 -0
+static void skb_tstamp_tx_bpf(struct sk_buff *skb, struct sock *sk, int tstype)
+{
+     int op;
+
+     if (!sk)

This check is redundant.

Thanks. Will remove it.

Quoted 34 lines
Quoted 18 lines
net/core/skbuff.c +13 -0
+             return;
+
+     switch (tstype) {
+     case SCM_TSTAMP_SCHED:
+             op = BPF_SOCK_OPS_TS_SCHED_OPT_CB;
+             break;
+     default:
+             return;
+     }
+
+     bpf_skops_tx_timestamping(sk, skb, op);
+}
+
  void __skb_tstamp_tx(struct sk_buff *orig_skb,
                   const struct sk_buff *ack_skb,
                   struct skb_shared_hwtstamps *hwtstamps,

net/core/skbuff.c void __skb_tstamp_tx(struct sk_buff *orig_skb, · line 5594

Quoted 1 line
net/core/skbuff.c +0 -0
@@ -5576,6 +5594,11 @@ void __skb_tstamp_tx(struct sk_buff *orig_skb,
      if (!sk)

It has been tested here...

net/core/skbuff.c void __skb_tstamp_tx(struct sk_buff *orig_skb, · line 5599

Quoted 5 lines
net/core/skbuff.c +3 -0
              return;
 
+     /* bpf extension feature entry */
+     if (skb_shinfo(orig_skb)->tx_flags & SKBTX_BPF)
+             skb_tstamp_tx_bpf(orig_skb, sk, tstype);

...before calling this.

Quoted 4 lines
net/core/skbuff.c +2 -0
+
+     /* application feature entry */
      if (!skb_enable_app_tstamp(orig_skb, tstype, sw))
              return;
#13 lore raw

Re: [PATCH bpf-next v7 08/13] net-timestamp: support hw SCM_TSTAMP_SND for bpf extension

In reply to Jason Xing
Quoted 81 lines · Jason Xing

In this patch, we finish the hardware part. Then bpf program can fetch the hwstamp from skb directly.

To avoid changing so many callers using SKBTX_HW_TSTAMP from drivers, use this simple modification like this patch does to support printing hardware timestamp.

---
  include/linux/skbuff.h         |  4 +++-
  include/uapi/linux/bpf.h       |  7 +++++++
  net/core/skbuff.c              | 11 ++++++-----
  net/dsa/user.c                 |  2 +-
  net/socket.c                   |  2 +-
  tools/include/uapi/linux/bpf.h |  7 +++++++
  6 files changed, 25 insertions(+), 8 deletions(-)
include/linux/skbuff.h +3 -1
index de8d3bd311f5..df2d790ae36b 100644
@@ -471,7 +471,7 @@ struct skb_shared_hwtstamps {
  /* Definitions for tx_flags in struct skb_shared_info */
  enum {
  	/* generate hardware time stamp */
-	SKBTX_HW_TSTAMP = 1 << 0,
+	__SKBTX_HW_TSTAMP = 1 << 0,
 
  	/* generate software time stamp when queueing packet to NIC */
  	SKBTX_SW_TSTAMP = 1 << 1,
@@ -495,6 +495,8 @@ enum {
  	SKBTX_BPF = 1 << 7,
  };
 
+#define SKBTX_HW_TSTAMP		(__SKBTX_HW_TSTAMP | SKBTX_BPF)
+
  #define SKBTX_ANY_SW_TSTAMP	(SKBTX_SW_TSTAMP    | \
  				 SKBTX_SCHED_TSTAMP | \
  				 SKBTX_BPF)
include/uapi/linux/bpf.h +7 -0
index 6a1083bcf779..4c3566f623c2 100644
@@ -7040,6 +7040,13 @@ enum {
  					 * to the nic when SK_BPF_CB_TX_TIMESTAMPING
  					 * feature is on.
  					 */
+	BPF_SOCK_OPS_TS_HW_OPT_CB,	/* Called in hardware phase when
+					 * SK_BPF_CB_TX_TIMESTAMPING feature
+					 * is on. At the same time, hwtstamps
+					 * of skb is initialized as the
+					 * timestamp that hardware just
+					 * generates.
+					 */
  };
 
  /* List of TCP states. There is a build check in net/ipv4/tcp.c to detect
net/core/skbuff.c +3 -2
index 288eb9869827..c769feae5162 100644
@@ -5548,7 +5548,7 @@ static bool skb_enable_app_tstamp(struct sk_buff *skb, int tstype, bool sw)
  		flag = SKBTX_SCHED_TSTAMP;
  		break;
  	case SCM_TSTAMP_SND:
-		flag = sw ? SKBTX_SW_TSTAMP : SKBTX_HW_TSTAMP;
+		flag = sw ? SKBTX_SW_TSTAMP : __SKBTX_HW_TSTAMP;
  		break;
  	case SCM_TSTAMP_ACK:
  		if (TCP_SKB_CB(skb)->txstamp_ack)
@@ -5565,7 +5565,8 @@ static bool skb_enable_app_tstamp(struct sk_buff *skb, int tstype, bool sw)
  }
 
  static void skb_tstamp_tx_bpf(struct sk_buff *skb, struct sock *sk,
-			      int tstype, bool sw)
+			      int tstype, bool sw,
+			      struct skb_shared_hwtstamps *hwtstamps)
  {
  	int op;
 

net/core/skbuff.c static void skb_tstamp_tx_bpf(struct sk_buff *skb, struct sock *sk, · line 5582

Quoted 5 lines
net/core/skbuff.c +1 -0
@@ -5577,9 +5578,9 @@ static void skb_tstamp_tx_bpf(struct sk_buff *skb, struct sock *sk,
  		op = BPF_SOCK_OPS_TS_SCHED_OPT_CB;
  		break;
  	case SCM_TSTAMP_SND:
+		op = sw ? BPF_SOCK_OPS_TS_SW_OPT_CB : BPF_SOCK_OPS_TS_HW_OPT_CB;
  		if (!sw)

Patch 5 mentioned hwtstamps could be NULL, so this should be "if (hwtstamps)" here.

net/core/skbuff.c static void skb_tstamp_tx_bpf(struct sk_buff *skb, struct sock *sk, · line 5583

Quoted 3 lines
net/core/skbuff.c +1 -2
-			return;
-		op = BPF_SOCK_OPS_TS_SW_OPT_CB;
+			*skb_hwtstamps(skb) = *hwtstamps;

Otherwise, this will crash.

pw-bot: cr

Quoted 3 lines
net/core/skbuff.c +0 -0
  		break;
  	default:
  		return;
#17 lore raw

Re: [PATCH bpf-next v7 10/13] net-timestamp: make TCP tx timestamp bpf extension work

In reply to Jason Xing
Quoted 1 line · Jason Xing

Make partial of the feature work finally. After this, user can

If it is "partial"-ly done, what is still missing?

My understanding is after this patch, the BPF program can fully support the TX timestamping in TCP.

Quoted 27 lines

fully use the bpf prog to trace the tx path for TCP type.

---
  net/ipv4/tcp.c | 9 +++++++++
  1 file changed, 9 insertions(+)
net/ipv4/tcp.c +9 -0
index 0d704bda6c41..0a41006b10d1 100644
@@ -492,6 +492,15 @@ static void tcp_tx_timestamp(struct sock *sk, struct sockcm_cookie *sockc)
  		if (tsflags & SOF_TIMESTAMPING_TX_RECORD_MASK)
  			shinfo->tskey = TCP_SKB_CB(skb)->seq + skb->len - 1;
  	}
+
+	if (SK_BPF_CB_FLAG_TEST(sk, SK_BPF_CB_TX_TIMESTAMPING) && skb) {
+		struct skb_shared_info *shinfo = skb_shinfo(skb);
+		struct tcp_skb_cb *tcb = TCP_SKB_CB(skb);
+
+		tcb->txstamp_ack_bpf = 1;
+		shinfo->tx_flags |= SKBTX_BPF;
+		shinfo->tskey = TCP_SKB_CB(skb)->seq + skb->len - 1;
+	}
  }
 
  static bool tcp_stream_is_readable(struct sock *sk, int target)
author
#14 lore raw

Re: [PATCH bpf-next v7 08/13] net-timestamp: support hw SCM_TSTAMP_SND for bpf extension

In reply to Martin KaFai Lau
Quoted 91 lines · Martin KaFai Lau
Quoted 81 lines · Jason Xing

In this patch, we finish the hardware part. Then bpf program can fetch the hwstamp from skb directly.

To avoid changing so many callers using SKBTX_HW_TSTAMP from drivers, use this simple modification like this patch does to support printing hardware timestamp.

---
  include/linux/skbuff.h         |  4 +++-
  include/uapi/linux/bpf.h       |  7 +++++++
  net/core/skbuff.c              | 11 ++++++-----
  net/dsa/user.c                 |  2 +-
  net/socket.c                   |  2 +-
  tools/include/uapi/linux/bpf.h |  7 +++++++
  6 files changed, 25 insertions(+), 8 deletions(-)
include/linux/skbuff.h +3 -1
index de8d3bd311f5..df2d790ae36b 100644
@@ -471,7 +471,7 @@ struct skb_shared_hwtstamps {
  /* Definitions for tx_flags in struct skb_shared_info */
  enum {
      /* generate hardware time stamp */
-     SKBTX_HW_TSTAMP = 1 << 0,
+     __SKBTX_HW_TSTAMP = 1 << 0,
 
      /* generate software time stamp when queueing packet to NIC */
      SKBTX_SW_TSTAMP = 1 << 1,
@@ -495,6 +495,8 @@ enum {
      SKBTX_BPF = 1 << 7,
  };
 
+#define SKBTX_HW_TSTAMP              (__SKBTX_HW_TSTAMP | SKBTX_BPF)
+
  #define SKBTX_ANY_SW_TSTAMP (SKBTX_SW_TSTAMP    | \
                               SKBTX_SCHED_TSTAMP | \
                               SKBTX_BPF)
include/uapi/linux/bpf.h +7 -0
index 6a1083bcf779..4c3566f623c2 100644
@@ -7040,6 +7040,13 @@ enum {
                                       * to the nic when SK_BPF_CB_TX_TIMESTAMPING
                                       * feature is on.
                                       */
+     BPF_SOCK_OPS_TS_HW_OPT_CB,      /* Called in hardware phase when
+                                      * SK_BPF_CB_TX_TIMESTAMPING feature
+                                      * is on. At the same time, hwtstamps
+                                      * of skb is initialized as the
+                                      * timestamp that hardware just
+                                      * generates.
+                                      */
  };
 
  /* List of TCP states. There is a build check in net/ipv4/tcp.c to detect
net/core/skbuff.c +3 -2
index 288eb9869827..c769feae5162 100644
@@ -5548,7 +5548,7 @@ static bool skb_enable_app_tstamp(struct sk_buff *skb, int tstype, bool sw)
              flag = SKBTX_SCHED_TSTAMP;
              break;
      case SCM_TSTAMP_SND:
-             flag = sw ? SKBTX_SW_TSTAMP : SKBTX_HW_TSTAMP;
+             flag = sw ? SKBTX_SW_TSTAMP : __SKBTX_HW_TSTAMP;
              break;
      case SCM_TSTAMP_ACK:
              if (TCP_SKB_CB(skb)->txstamp_ack)
@@ -5565,7 +5565,8 @@ static bool skb_enable_app_tstamp(struct sk_buff *skb, int tstype, bool sw)
  }
 
  static void skb_tstamp_tx_bpf(struct sk_buff *skb, struct sock *sk,
-                           int tstype, bool sw)
+                           int tstype, bool sw,
+                           struct skb_shared_hwtstamps *hwtstamps)
  {
      int op;
 

net/core/skbuff.c static void skb_tstamp_tx_bpf(struct sk_buff *skb, struct sock *sk, · line 5582

Quoted 5 lines
net/core/skbuff.c +1 -0
@@ -5577,9 +5578,9 @@ static void skb_tstamp_tx_bpf(struct sk_buff *skb, struct sock *sk,
              op = BPF_SOCK_OPS_TS_SCHED_OPT_CB;
              break;
      case SCM_TSTAMP_SND:
+             op = sw ? BPF_SOCK_OPS_TS_SW_OPT_CB : BPF_SOCK_OPS_TS_HW_OPT_CB;
              if (!sw)

Patch 5 mentioned hwtstamps could be NULL, so this should be "if (hwtstamps)" here.

Quoted 5 lines

net/core/skbuff.c static void skb_tstamp_tx_bpf(struct sk_buff *skb, struct sock *sk, · line 5583

Quoted 3 lines
net/core/skbuff.c +1 -2
-                     return;
-             op = BPF_SOCK_OPS_TS_SW_OPT_CB;
+                     *skb_hwtstamps(skb) = *hwtstamps;

Otherwise, this will crash.

Sorry, I tested it in the wrong way... Will fix it soon!

Quoted 7 lines

pw-bot: cr

Quoted 3 lines
net/core/skbuff.c +0 -0
              break;
      default:
              return;
author
#18 lore raw

Re: [PATCH bpf-next v7 10/13] net-timestamp: make TCP tx timestamp bpf extension work

In reply to Martin KaFai Lau
Quoted 8 lines · Martin KaFai Lau
Quoted 1 line · Jason Xing

Make partial of the feature work finally. After this, user can

If it is "partial"-ly done, what is still missing?

My understanding is after this patch, the BPF program can fully support the TX timestamping in TCP.

I'm going to make the change in the next version. My thought was a big project supporting various protocols.

Quoted 29 lines
Quoted 28 lines

fully use the bpf prog to trace the tx path for TCP type.

---
  net/ipv4/tcp.c | 9 +++++++++
  1 file changed, 9 insertions(+)
net/ipv4/tcp.c +9 -0
index 0d704bda6c41..0a41006b10d1 100644
@@ -492,6 +492,15 @@ static void tcp_tx_timestamp(struct sock *sk, struct sockcm_cookie *sockc)
              if (tsflags & SOF_TIMESTAMPING_TX_RECORD_MASK)
                      shinfo->tskey = TCP_SKB_CB(skb)->seq + skb->len - 1;
      }
+
+     if (SK_BPF_CB_FLAG_TEST(sk, SK_BPF_CB_TX_TIMESTAMPING) && skb) {
+             struct skb_shared_info *shinfo = skb_shinfo(skb);
+             struct tcp_skb_cb *tcb = TCP_SKB_CB(skb);
+
+             tcb->txstamp_ack_bpf = 1;
+             shinfo->tx_flags |= SKBTX_BPF;
+             shinfo->tskey = TCP_SKB_CB(skb)->seq + skb->len - 1;
+     }
  }
 
  static bool tcp_stream_is_readable(struct sock *sk, int target)
#20 lore raw

Re: [PATCH bpf-next v7 11/13] net-timestamp: add a new callback in tcp_tx_timestamp()

In reply to Jason Xing
Quoted 3 lines · Jason Xing

Introduce the callback to correlate tcp_sendmsg timestamp with other points, like SND/SW/ACK. We can let bpf trace the beginning of tcp_sendmsg_locked() and fetch the socket addr, so that in

Instead of "fetch the socket addr...", should be "store the sendmsg timestamp at the bpf_sk_storage ...".

Quoted 5 lines

tcp_tx_timestamp() we can correlate the tskey with the socket addr.

It is accurate since they are under the protect of socket lock. More details can be found in the selftest.

The selftest uses the bpf_sk_storage to store the sendmsg timestamp at fentry/tcp_sendmsg_locked and retrieves it back at tcp_tx_timestamp (i.e. BPF_SOCK_OPS_TS_SND_CB added in this patch).

Quoted 13 lines
---
  include/uapi/linux/bpf.h       | 7 +++++++
  net/ipv4/tcp.c                 | 1 +
  tools/include/uapi/linux/bpf.h | 7 +++++++
  3 files changed, 15 insertions(+)
include/uapi/linux/bpf.h +0 -0
index 800122a8abe5..accb3b314fff 100644

include/uapi/linux/bpf.h enum { · line 7060

Quoted 9 lines
include/uapi/linux/bpf.h +6 -0
@@ -7052,6 +7052,13 @@ enum {
  					 * when SK_BPF_CB_TX_TIMESTAMPING
  					 * feature is on.
  					 */
+	BPF_SOCK_OPS_TS_SND_CB,		/* Called when every sendmsg syscall
+					 * is triggered. For TCP, it stays
+					 * in the last send process to
+					 * correlate with tcp_sendmsg timestamp
+					 * with other timestamping callbacks,
+					 * like SND/SW/ACK.

Do you have a chance to look at how this will work at UDP?

#35 lore raw

Re: [PATCH bpf-next v7 12/13] net-timestamp: introduce cgroup lock to avoid affecting non-bpf cases

In reply to Jason Xing
Quoted 1 line · Jason Xing

Introducing the lock to avoid affecting the applications which

s/lock/static key/

Unless it needs more static-key guards in the next re-spin, I would squash this one liner with patch 10.

Quoted 21 lines

are not using timestamping bpf feature.

---
  net/ipv4/tcp.c | 3 ++-
  1 file changed, 2 insertions(+), 1 deletion(-)
net/ipv4/tcp.c +2 -1
index b2f1fd216df1..a2ac57543b6d 100644
@@ -493,7 +493,8 @@ static void tcp_tx_timestamp(struct sock *sk, struct sockcm_cookie *sockc)
  			shinfo->tskey = TCP_SKB_CB(skb)->seq + skb->len - 1;
  	}
 
-	if (SK_BPF_CB_FLAG_TEST(sk, SK_BPF_CB_TX_TIMESTAMPING) && skb) {
+	if (cgroup_bpf_enabled(CGROUP_SOCK_OPS) &&
+	    SK_BPF_CB_FLAG_TEST(sk, SK_BPF_CB_TX_TIMESTAMPING) && skb) {
  		struct skb_shared_info *shinfo = skb_shinfo(skb);
  		struct tcp_skb_cb *tcb = TCP_SKB_CB(skb);
 
author
#21 lore raw

Re: [PATCH bpf-next v7 11/13] net-timestamp: add a new callback in tcp_tx_timestamp()

In reply to Martin KaFai Lau
Quoted 8 lines · Martin KaFai Lau
Quoted 3 lines · Jason Xing

Introduce the callback to correlate tcp_sendmsg timestamp with other points, like SND/SW/ACK. We can let bpf trace the beginning of tcp_sendmsg_locked() and fetch the socket addr, so that in

Instead of "fetch the socket addr...", should be "store the sendmsg timestamp at the bpf_sk_storage ...".

I will revise it. Thanks.

Quoted 19 lines
Quoted 6 lines

tcp_tx_timestamp() we can correlate the tskey with the socket addr.

It is accurate since they are under the protect of socket lock. More details can be found in the selftest.

The selftest uses the bpf_sk_storage to store the sendmsg timestamp at fentry/tcp_sendmsg_locked and retrieves it back at tcp_tx_timestamp (i.e. BPF_SOCK_OPS_TS_SND_CB added in this patch).

Quoted 8 lines
---
  include/uapi/linux/bpf.h       | 7 +++++++
  net/ipv4/tcp.c                 | 1 +
  tools/include/uapi/linux/bpf.h | 7 +++++++
  3 files changed, 15 insertions(+)
Quoted 16 lines
Quoted 5 lines
include/uapi/linux/bpf.h +0 -0
index 800122a8abe5..accb3b314fff 100644

include/uapi/linux/bpf.h enum { · line 7060

Quoted 9 lines
include/uapi/linux/bpf.h +6 -0
@@ -7052,6 +7052,13 @@ enum {
                                       * when SK_BPF_CB_TX_TIMESTAMPING
                                       * feature is on.
                                       */
+     BPF_SOCK_OPS_TS_SND_CB,         /* Called when every sendmsg syscall
+                                      * is triggered. For TCP, it stays
+                                      * in the last send process to
+                                      * correlate with tcp_sendmsg timestamp
+                                      * with other timestamping callbacks,
+                                      * like SND/SW/ACK.

Do you have a chance to look at how this will work at UDP?

Sure, I feel like it could not be useful for UDP. Well, things get strange because I did write a long paragraph about this thing which apparently disappeared...

I manage to find what I wrote:
For UDP type, BPF_SOCK_OPS_TS_SND_CB may be not suitable because there are two sending process, 1) lockless path, 2) lock path, which should be handled carefully later. For the former, even though it's unlikely multiple threads access the socket to call sendmsg at the same time, I think we'd better not correlate it like what we do to the TCP case because of the lack of sock lock protection. Considering SND_CB is uapi flag, I think we don't need to forcely add the 'TCP_' prefix in case we need to use it someday.

And one more thing is I'd like to use the v5[1] method in the next round to introduce a new tskey_bpf which is good for UDP type. The new field will not conflict with the tskey in shared info which is generated by sk->sk_tskey in __ip_append_data(). It hardly works if both features (so_timestamping and its bpf extension) exists at the same time. Users could get confused because sometimes they fetch the tskey from skb, sometimes they don't, especially when we have cmsg feature to turn it on/ off per sendmsg. A standalone tskey for bpf extension will be needed. With this tskey_bpf, we can easily correlate the timestamp in sendmsg syscall with other tx points(SND/SW/ACK...).

[1]: https://lore.kernel.org/all/[email protected]/

If possible, we can leave this question until the UDP support series shows up. I will figure out a better solution :)

In conclusion, it probably won't be used by the UDP type. It's uAPI flag so I consider the compatibility reason.

author
#36 lore raw

Re: [PATCH bpf-next v7 12/13] net-timestamp: introduce cgroup lock to avoid affecting non-bpf cases

In reply to Martin KaFai Lau
Quoted 8 lines · Martin KaFai Lau
Quoted 1 line · Jason Xing

Introducing the lock to avoid affecting the applications which

s/lock/static key/

Unless it needs more static-key guards in the next re-spin, I would squash this one liner with patch 10.

Got it. Will do that. Thanks.

Quoted 23 lines
Quoted 22 lines

are not using timestamping bpf feature.

---
  net/ipv4/tcp.c | 3 ++-
  1 file changed, 2 insertions(+), 1 deletion(-)
net/ipv4/tcp.c +2 -1
index b2f1fd216df1..a2ac57543b6d 100644
@@ -493,7 +493,8 @@ static void tcp_tx_timestamp(struct sock *sk, struct sockcm_cookie *sockc)
                      shinfo->tskey = TCP_SKB_CB(skb)->seq + skb->len - 1;
      }
 
-     if (SK_BPF_CB_FLAG_TEST(sk, SK_BPF_CB_TX_TIMESTAMPING) && skb) {
+     if (cgroup_bpf_enabled(CGROUP_SOCK_OPS) &&
+         SK_BPF_CB_FLAG_TEST(sk, SK_BPF_CB_TX_TIMESTAMPING) && skb) {
              struct skb_shared_info *shinfo = skb_shinfo(skb);
              struct tcp_skb_cb *tcb = TCP_SKB_CB(skb);
 
#38 lore raw

Re: [PATCH bpf-next v7 13/13] bpf: add simple bpf tests in the tx path for so_timestamping feature

In reply to Jason Xing
Quoted 3 lines · Jason Xing

Only check if we pass those three key points after we enable the bpf extension for so_timestamping. During each point, we can choose whether to print the current timestamp.

The commit message also needs to be updated...

Quoted 15 lines
---
  .../bpf/prog_tests/so_timestamping.c          |  86 +++++
  .../selftests/bpf/progs/so_timestamping.c     | 299 ++++++++++++++++++
  2 files changed, 385 insertions(+)

create mode 100644 tools/testing/selftests/bpf/prog_tests/so_timestamping.c create mode 100644 tools/testing/selftests/bpf/progs/so_timestamping.c

tools/testing/selftests/bpf/prog_tests/so_timestamping.c new file +0 -0
new file mode 100644
index 000000000000..ee7fdc381609

tools/testing/selftests/bpf/prog_tests/so_timestamping.c line 4

Quoted 4 lines
tools/testing/selftests/bpf/prog_tests/so_timestamping.c new file +4 -0
@@ -0,0 +1,86 @@
+#define _GNU_SOURCE
+#include <sched.h>
+#include <linux/socket.h>
+#include <linux/tls.h>

tls.h?

tools/testing/selftests/bpf/prog_tests/so_timestamping.c line 5

Quoted 1 line
tools/testing/selftests/bpf/prog_tests/so_timestamping.c new file +1 -0
+#include <net/if.h>

I suspect most of the above #define and #include are not needed. Please clean up.

tools/testing/selftests/bpf/prog_tests/so_timestamping.c line 18

Quoted 13 lines
tools/testing/selftests/bpf/prog_tests/so_timestamping.c new file +13 -0
+
+#include "test_progs.h"
+#include "cgroup_helpers.h"
+#include "network_helpers.h"
+
+#include "so_timestamping.skel.h"
+
+#define CG_NAME "/so-timestamping-test"
+
+static const char addr4_str[] = "127.0.0.1";
+static const char addr6_str[] = "::1";
+static struct so_timestamping *skel;
+static int cg_fd;

nit. cg_fd does not need to be global.

Quoted 32 lines
tools/testing/selftests/bpf/prog_tests/so_timestamping.c new file +32 -0
+
+static void test_tcp(int family)
+{
+	struct so_timestamping__bss *bss = skel->bss;
+	char buf[] = "testing testing";
+	int sfd = -1, cfd = -1;
+	int n;
+
+	memset(bss, 0, sizeof(*bss));
+
+	sfd = start_server(family, SOCK_STREAM,
+			   family == AF_INET6 ? addr6_str : addr4_str, 0, 0);
+	if (!ASSERT_OK_FD(sfd, "start_server"))
+		goto out;
+
+	cfd = connect_to_fd(sfd, 0);
+	if (!ASSERT_OK_FD(cfd, "connect_to_fd_server"))
+		goto out;
+
+	n = write(cfd, buf, sizeof(buf));
+	if (!ASSERT_EQ(n, sizeof(buf), "send to server"))
+		goto out;
+
+	ASSERT_EQ(bss->nr_active, 1, "nr_active");
+	ASSERT_EQ(bss->nr_snd, 2, "nr_snd");
+	ASSERT_EQ(bss->nr_sched, 1, "nr_sched");
+	ASSERT_EQ(bss->nr_txsw, 1, "nr_txsw");
+	ASSERT_EQ(bss->nr_ack, 1, "nr_ack");
+
+out:
+	if (sfd >= 0)
+		close(sfd);

tools/testing/selftests/bpf/prog_tests/so_timestamping.c line 60

Quoted 10 lines
tools/testing/selftests/bpf/prog_tests/so_timestamping.c new file +10 -0
+	if (cfd >= 0)
+		close(cfd);
+}
+
+void test_so_timestamping(void)
+{
+	struct netns_obj *ns;
+
+	cg_fd = test__join_cgroup(CG_NAME);
+	if (cg_fd < 0)

ASSERT_OK_FD. The existing setget_sockopt test should probably be fixed also but that will be a separate patch.

tools/testing/selftests/bpf/prog_tests/so_timestamping.c line 64

Quoted 4 lines
tools/testing/selftests/bpf/prog_tests/so_timestamping.c new file +4 -0
+		return;
+
+	ns = netns_new("so_timestamping_ns", true);
+	if (!ASSERT_OK_PTR(ns, "create ns"))

cg_fd is leaked.

tools/testing/selftests/bpf/prog_tests/so_timestamping.c line 65

Quoted 1 line
tools/testing/selftests/bpf/prog_tests/so_timestamping.c new file +1 -0
+		return;

goto done;

netns_free() and so_timestamping__destroy() can handle NULL.

Quoted 75 lines
tools/testing/selftests/bpf/prog_tests/so_timestamping.c new file +21 -0
+
+	skel = so_timestamping__open_and_load();
+	if (!ASSERT_OK_PTR(skel, "open and load skel"))
+		goto done;
+
+	if (!ASSERT_OK(so_timestamping__attach(skel), "attach skel"))
+		goto done;
+
+	skel->links.skops_sockopt =
+		bpf_program__attach_cgroup(skel->progs.skops_sockopt, cg_fd);
+	if (!ASSERT_OK_PTR(skel->links.skops_sockopt, "attach cgroup"))
+		goto done;
+
+	test_tcp(AF_INET6);
+	test_tcp(AF_INET);
+
+done:
+	so_timestamping__destroy(skel);
+	netns_free(ns);
+	close(cg_fd);
+}
tools/testing/selftests/bpf/progs/so_timestamping.c new file +48 -0
new file mode 100644
index 000000000000..a893859ffe32
@@ -0,0 +1,299 @@
+#include "vmlinux.h"
+#include "bpf_tracing_net.h"
+#include <bpf/bpf_core_read.h>
+#include <bpf/bpf_helpers.h>
+#include <bpf/bpf_tracing.h>
+#include "bpf_misc.h"
+#include "bpf_kfuncs.h"
+#define BPF_PROG_TEST_TCP_HDR_OPTIONS
+#include "test_tcp_hdr_options.h"
+#include <errno.h>
+
+#define SK_BPF_CB_FLAGS 1009
+#define SK_BPF_CB_TX_TIMESTAMPING 1
+
+int nr_active;
+int nr_snd;
+int nr_passive;
+int nr_sched;
+int nr_txsw;
+int nr_ack;
+
+struct sockopt_test {
+	int opt;
+	int new;
+};
+
+static const struct sockopt_test sol_socket_tests[] = {
+	{ .opt = SK_BPF_CB_FLAGS, .new = SK_BPF_CB_TX_TIMESTAMPING, },
+	{ .opt = 0, },
+};
+
+struct loop_ctx {
+	void *ctx;
+	const struct sock *sk;
+};
+
+struct sk_stg {
+	__u64 sendmsg_ns;	/* record ts when sendmsg is called */
+};
+
+struct {
+	__uint(type, BPF_MAP_TYPE_SK_STORAGE);
+	__uint(map_flags, BPF_F_NO_PREALLOC);
+	__type(key, int);
+	__type(value, struct sk_stg);
+} sk_stg_map SEC(".maps");
+
+

tools/testing/selftests/bpf/progs/so_timestamping.c line 58

Quoted 10 lines
tools/testing/selftests/bpf/progs/so_timestamping.c new file +10 -0
+struct delay_info {
+	u64 sendmsg_ns;		/* record ts when sendmsg is called */
+	u32 sched_delay;	/* SCHED_OPT_CB - sendmsg_ns */
+	u32 sw_snd_delay;	/* SW_OPT_CB - SCHED_OPT_CB */
+	u32 ack_delay;		/* ACK_OPT_CB - SW_OPT_CB */
+};
+
+struct {
+	__uint(type, BPF_MAP_TYPE_HASH);
+	__type(key, u32);

I just noticed there are two tcp connections in the test. One v4 and one v6. Unlikely to collide on seqno, still better to add a sk_cookie to the key of the map, like:

struct sk_tskey {
	u64 sk_cookie;
	u32 tskey;
};

Use bpf_get_sokcet_cookie(ctx) to get a unique socket cookie.

Quoted 35 lines
tools/testing/selftests/bpf/progs/so_timestamping.c new file +35 -0
+	__type(value, struct delay_info);
+	__uint(max_entries, 1024);
+} time_map SEC(".maps");
+
+static u64 delay_tolerance_nsec = 10000000000; /* 10 second as an example */
+
+static int bpf_test_sockopt_int(void *ctx, const struct sock *sk,
+				const struct sockopt_test *t,
+				int level)
+{
+	int new, opt, tmp;
+
+	opt = t->opt;
+	new = t->new;
+
+	if (bpf_setsockopt(ctx, level, opt, &new, sizeof(new)))
+		return 1;
+
+	if (bpf_getsockopt(ctx, level, opt, &tmp, sizeof(tmp)) ||
+	    tmp != new)
+		return 1;
+
+	return 0;
+}
+
+static int bpf_test_socket_sockopt(__u32 i, struct loop_ctx *lc)
+{
+	const struct sockopt_test *t;
+
+	if (i >= ARRAY_SIZE(sol_socket_tests))
+		return 1;
+
+	t = &sol_socket_tests[i];
+	if (!t->opt)
+		return 1;

tools/testing/selftests/bpf/progs/so_timestamping.c line 103

Quoted 10 lines
tools/testing/selftests/bpf/progs/so_timestamping.c new file +10 -0
+
+	return bpf_test_sockopt_int(lc->ctx, lc->sk, t, SOL_SOCKET);
+}
+
+static int bpf_test_sockopt(void *ctx, const struct sock *sk)
+{
+	struct loop_ctx lc = { .ctx = ctx, .sk = sk, };
+	int n;
+
+	n = bpf_loop(ARRAY_SIZE(sol_socket_tests), bpf_test_socket_sockopt, &lc, 0);

There is only one SK_BPF_CB_FLAGS optname to test, so no need to bpf_loop. Directly do one bpf_setsockopt and one bpf_getsockopt.

We can see if there is a need to refactor this timestamp test back to the setget_sockopt.c test later if loop will be needed after adding the UDP support. The setget_sockopt.c does use a loop to test many options at once which is probably where this piece of code (bpf_loop) is borrowed from.

Quoted 5 lines
tools/testing/selftests/bpf/progs/so_timestamping.c new file +5 -0
+	if (n != ARRAY_SIZE(sol_socket_tests))
+		return -1;
+
+	return 0;
+}

tools/testing/selftests/bpf/progs/so_timestamping.c line 118

Quoted 10 lines
tools/testing/selftests/bpf/progs/so_timestamping.c new file +10 -0
+
+static bool bpf_test_access_sockopt(void *ctx)
+{
+	const struct sockopt_test *t;
+	int tmp, ret, i = 0;
+	int level = SOL_SOCKET;
+
+	t = &sol_socket_tests[i];
+
+	for (; t->opt;) {

Same here. Directly do one bpf_setsockopt and one bpf_getsockopt instead of looping.

Quoted 38 lines
tools/testing/selftests/bpf/progs/so_timestamping.c new file +38 -0
+		ret = bpf_setsockopt(ctx, level, t->opt, (void *)&t->new, sizeof(t->new));
+		if (ret != -EOPNOTSUPP)
+			return true;
+
+		ret = bpf_getsockopt(ctx, level, t->opt, &tmp, sizeof(tmp));
+		if (ret != -EOPNOTSUPP)
+			return true;
+
+		if (++i >= ARRAY_SIZE(sol_socket_tests))
+			break;
+	}
+
+	return false;
+}
+
+/* Adding a simple test to see if we can get an expected value */
+static bool bpf_test_access_load_hdr_opt(struct bpf_sock_ops *skops)
+{
+	struct tcp_opt reg_opt;
+	int load_flags = 0;
+	int ret;
+
+	reg_opt.kind = TCPOPT_EXP;
+	reg_opt.len = 0;
+	reg_opt.data32 = 0;
+	ret = bpf_load_hdr_opt(skops, &reg_opt, sizeof(reg_opt), load_flags);
+	if (ret != -EOPNOTSUPP)
+		return true;
+
+	return false;
+}
+
+/* Adding a simple test to see if we can get an expected value */
+static bool bpf_test_access_cb_flags_set(struct bpf_sock_ops *skops)
+{
+	int ret;
+
+	ret = bpf_sock_ops_cb_flags_set(skops, 0);

tools/testing/selftests/bpf/progs/so_timestamping.c line 166

Quoted 10 lines
tools/testing/selftests/bpf/progs/so_timestamping.c new file +10 -0
+	if (ret != -EOPNOTSUPP)
+		return true;
+
+	return false;
+}
+
+/* In the timestamping callbacks, we're not allowed to call the following
+ * BPF CALLs for the safety concern. Return false if expected.
+ */
+static int bpf_test_access_bpf_calls(struct bpf_sock_ops *skops,

nit. The return value is true/false. Stay with "bool" as the return type.

tools/testing/selftests/bpf/progs/so_timestamping.c line 176

Quoted 10 lines
tools/testing/selftests/bpf/progs/so_timestamping.c new file +10 -0
+				     const struct sock *sk)
+{
+	if (bpf_test_access_sockopt(skops))
+		return true;
+
+	if (bpf_test_access_load_hdr_opt(skops))
+		return true;
+
+	if (bpf_test_access_cb_flags_set(skops))
+		return true;

Thanks for adding these negative tests.

Quoted 4 lines
tools/testing/selftests/bpf/progs/so_timestamping.c new file +4 -0
+
+	return false;
+}
+
#40 lore raw
In reply to Jason Xing
Quoted 17 lines · Jason Xing

"Timestamping is key to debugging network stack latency. With SO_TIMESTAMPING, bugs that are otherwise incorrectly assumed to be network issues can be attributed to the kernel." This is extracted from the talk "SO_TIMESTAMPING: Powering Fleetwide RPC Monitoring" addressed by Willem de Bruijn at netdevconf 0x17).

There are a few areas that need optimization with the consideration of easier use and less performance impact, which I highlighted and mainly discussed at netconf 2024 with Willem de Bruijn and John Fastabend:
uAPI compatibility, extra system call overhead, and the need for application modification. I initially managed to solve these issues by writing a kernel module that hooks various key functions. However, this approach is not suitable for the next kernel release. Therefore, a BPF extension was proposed. During recent period, Martin KaFai Lau provides invaluable suggestions about BPF along the way. Many thanks here!

Quoted 1 line

In this series, I only support foundamental codes and tx for TCP.

*fundamental*.

May be just "only tx time stamping for TCP is supported..."

Quoted 6 lines

This approach mostly relies on existing SO_TIMESTAMPING feature, users only needs to pass certain flags through bpf_setsocktopt() to a separate tsflags. Please see the last selftest patch in this series.

After this series, we could step by step implement more advanced functions/flags already in SO_TIMESTAMPING feature for bpf extension.

Patch 1-4 and 6-11 can use an extra "bpf:" tag in the subject line. Patch 13 should be "selftests/bpf:" instead of "bpf:" in the subject.

Please revisit the commit messages of this patch set to check for outdated comments from the earlier revisions. I may have missed some of them.

Overall, it looks close. I will review at your replies later.

Willem, could you also take a look? Thanks.

author
#41 lore raw
Quoted 29 lines · Martin KaFai Lau
Quoted 17 lines · Jason Xing

"Timestamping is key to debugging network stack latency. With SO_TIMESTAMPING, bugs that are otherwise incorrectly assumed to be network issues can be attributed to the kernel." This is extracted from the talk "SO_TIMESTAMPING: Powering Fleetwide RPC Monitoring" addressed by Willem de Bruijn at netdevconf 0x17).

There are a few areas that need optimization with the consideration of easier use and less performance impact, which I highlighted and mainly discussed at netconf 2024 with Willem de Bruijn and John Fastabend:
uAPI compatibility, extra system call overhead, and the need for application modification. I initially managed to solve these issues by writing a kernel module that hooks various key functions. However, this approach is not suitable for the next kernel release. Therefore, a BPF extension was proposed. During recent period, Martin KaFai Lau provides invaluable suggestions about BPF along the way. Many thanks here!

Quoted 1 line

In this series, I only support foundamental codes and tx for TCP.

*fundamental*.

May be just "only tx time stamping for TCP is supported..."

Quoted 4 lines

This approach mostly relies on existing SO_TIMESTAMPING feature, users only needs to pass certain flags through bpf_setsocktopt() to a separate tsflags. Please see the last selftest patch in this series.

Quoted 8 lines
Quoted 2 lines

After this series, we could step by step implement more advanced functions/flags already in SO_TIMESTAMPING feature for bpf extension.

Patch 1-4 and 6-11 can use an extra "bpf:" tag in the subject line. Patch 13 should be "selftests/bpf:" instead of "bpf:" in the subject.

Please revisit the commit messages of this patch set to check for outdated comments from the earlier revisions. I may have missed some of them.

Roger that, sir. Thanks for your help!

Quoted 4 lines

Overall, it looks close. I will review at your replies later.

Willem, could you also take a look? Thanks.

Right, some related parts need reviews from netdev experts as well.

Willem, please help me review this when you're available. No rush :)

Thanks,
Jason

author
#39 lore raw

Re: [PATCH bpf-next v7 13/13] bpf: add simple bpf tests in the tx path for so_timestamping feature

In reply to Martin KaFai Lau
Quoted 7 lines · Martin KaFai Lau
Quoted 3 lines · Jason Xing

Only check if we pass those three key points after we enable the bpf extension for so_timestamping. During each point, we can choose whether to print the current timestamp.

The commit message also needs to be updated...

I will revise it.

Quoted 23 lines
Quoted 16 lines
---
  .../bpf/prog_tests/so_timestamping.c          |  86 +++++
  .../selftests/bpf/progs/so_timestamping.c     | 299 ++++++++++++++++++
  2 files changed, 385 insertions(+)

create mode 100644 tools/testing/selftests/bpf/prog_tests/so_timestamping.c create mode 100644 tools/testing/selftests/bpf/progs/so_timestamping.c

tools/testing/selftests/bpf/prog_tests/so_timestamping.c new file +0 -0
new file mode 100644
index 000000000000..ee7fdc381609

tools/testing/selftests/bpf/prog_tests/so_timestamping.c line 4

Quoted 4 lines
tools/testing/selftests/bpf/prog_tests/so_timestamping.c new file +4 -0
@@ -0,0 +1,86 @@
+#define _GNU_SOURCE
+#include <sched.h>
+#include <linux/socket.h>
+#include <linux/tls.h>

tls.h?

Quoted 3 lines

tools/testing/selftests/bpf/prog_tests/so_timestamping.c line 5

Quoted 1 line
tools/testing/selftests/bpf/prog_tests/so_timestamping.c new file +1 -0
+#include <net/if.h>

I suspect most of the above #define and #include are not needed. Please clean up.

I found all the above codes unnecessary.

Quoted 16 lines
Quoted 4 lines
tools/testing/selftests/bpf/prog_tests/so_timestamping.c new file +3 -0
+
+#include "test_progs.h"
+#include "cgroup_helpers.h"

tools/testing/selftests/bpf/prog_tests/so_timestamping.c line 18

Quoted 10 lines
tools/testing/selftests/bpf/prog_tests/so_timestamping.c new file +10 -0
+#include "network_helpers.h"
+
+#include "so_timestamping.skel.h"
+
+#define CG_NAME "/so-timestamping-test"
+
+static const char addr4_str[] = "127.0.0.1";
+static const char addr6_str[] = "::1";
+static struct so_timestamping *skel;
+static int cg_fd;

nit. cg_fd does not need to be global.

Got it.

Quoted 38 lines
Quoted 38 lines
tools/testing/selftests/bpf/prog_tests/so_timestamping.c new file +37 -0
+
+static void test_tcp(int family)
+{
+     struct so_timestamping__bss *bss = skel->bss;
+     char buf[] = "testing testing";
+     int sfd = -1, cfd = -1;
+     int n;
+
+     memset(bss, 0, sizeof(*bss));
+
+     sfd = start_server(family, SOCK_STREAM,
+                        family == AF_INET6 ? addr6_str : addr4_str, 0, 0);
+     if (!ASSERT_OK_FD(sfd, "start_server"))
+             goto out;
+
+     cfd = connect_to_fd(sfd, 0);
+     if (!ASSERT_OK_FD(cfd, "connect_to_fd_server"))
+             goto out;
+
+     n = write(cfd, buf, sizeof(buf));
+     if (!ASSERT_EQ(n, sizeof(buf), "send to server"))
+             goto out;
+
+     ASSERT_EQ(bss->nr_active, 1, "nr_active");
+     ASSERT_EQ(bss->nr_snd, 2, "nr_snd");
+     ASSERT_EQ(bss->nr_sched, 1, "nr_sched");
+     ASSERT_EQ(bss->nr_txsw, 1, "nr_txsw");
+     ASSERT_EQ(bss->nr_ack, 1, "nr_ack");
+
+out:
+     if (sfd >= 0)
+             close(sfd);
+     if (cfd >= 0)
+             close(cfd);
+}
+
+void test_so_timestamping(void)
Quoted 8 lines

tools/testing/selftests/bpf/prog_tests/so_timestamping.c line 60

Quoted 5 lines
tools/testing/selftests/bpf/prog_tests/so_timestamping.c new file +5 -0
+{
+     struct netns_obj *ns;
+
+     cg_fd = test__join_cgroup(CG_NAME);
+     if (cg_fd < 0)

ASSERT_OK_FD. The existing setget_sockopt test should probably be fixed also but that will be a separate patch.

Will fix it. And I just now sent a standalone patch according to your suggestion.

Quoted 11 lines

tools/testing/selftests/bpf/prog_tests/so_timestamping.c line 64

Quoted 5 lines
tools/testing/selftests/bpf/prog_tests/so_timestamping.c new file +4 -0
+             return;
+
+     ns = netns_new("so_timestamping_ns", true);
+     if (!ASSERT_OK_PTR(ns, "create ns"))

cg_fd is leaked.

tools/testing/selftests/bpf/prog_tests/so_timestamping.c line 65

Quoted 1 line
tools/testing/selftests/bpf/prog_tests/so_timestamping.c new file +1 -0
+             return;

goto done;

Will take care of it.

Quoted 93 lines

netns_free() and so_timestamping__destroy() can handle NULL.

Quoted 75 lines
tools/testing/selftests/bpf/prog_tests/so_timestamping.c new file +21 -0
+
+     skel = so_timestamping__open_and_load();
+     if (!ASSERT_OK_PTR(skel, "open and load skel"))
+             goto done;
+
+     if (!ASSERT_OK(so_timestamping__attach(skel), "attach skel"))
+             goto done;
+
+     skel->links.skops_sockopt =
+             bpf_program__attach_cgroup(skel->progs.skops_sockopt, cg_fd);
+     if (!ASSERT_OK_PTR(skel->links.skops_sockopt, "attach cgroup"))
+             goto done;
+
+     test_tcp(AF_INET6);
+     test_tcp(AF_INET);
+
+done:
+     so_timestamping__destroy(skel);
+     netns_free(ns);
+     close(cg_fd);
+}
tools/testing/selftests/bpf/progs/so_timestamping.c new file +48 -0
new file mode 100644
index 000000000000..a893859ffe32
@@ -0,0 +1,299 @@
+#include "vmlinux.h"
+#include "bpf_tracing_net.h"
+#include <bpf/bpf_core_read.h>
+#include <bpf/bpf_helpers.h>
+#include <bpf/bpf_tracing.h>
+#include "bpf_misc.h"
+#include "bpf_kfuncs.h"
+#define BPF_PROG_TEST_TCP_HDR_OPTIONS
+#include "test_tcp_hdr_options.h"
+#include <errno.h>
+
+#define SK_BPF_CB_FLAGS 1009
+#define SK_BPF_CB_TX_TIMESTAMPING 1
+
+int nr_active;
+int nr_snd;
+int nr_passive;
+int nr_sched;
+int nr_txsw;
+int nr_ack;
+
+struct sockopt_test {
+     int opt;
+     int new;
+};
+
+static const struct sockopt_test sol_socket_tests[] = {
+     { .opt = SK_BPF_CB_FLAGS, .new = SK_BPF_CB_TX_TIMESTAMPING, },
+     { .opt = 0, },
+};
+
+struct loop_ctx {
+     void *ctx;
+     const struct sock *sk;
+};
+
+struct sk_stg {
+     __u64 sendmsg_ns;       /* record ts when sendmsg is called */
+};
+
+struct {
+     __uint(type, BPF_MAP_TYPE_SK_STORAGE);
+     __uint(map_flags, BPF_F_NO_PREALLOC);
+     __type(key, int);
+     __type(value, struct sk_stg);
+} sk_stg_map SEC(".maps");
+
+

tools/testing/selftests/bpf/progs/so_timestamping.c line 58

Quoted 10 lines
tools/testing/selftests/bpf/progs/so_timestamping.c new file +10 -0
+struct delay_info {
+     u64 sendmsg_ns;         /* record ts when sendmsg is called */
+     u32 sched_delay;        /* SCHED_OPT_CB - sendmsg_ns */
+     u32 sw_snd_delay;       /* SW_OPT_CB - SCHED_OPT_CB */
+     u32 ack_delay;          /* ACK_OPT_CB - SW_OPT_CB */
+};
+
+struct {
+     __uint(type, BPF_MAP_TYPE_HASH);
+     __type(key, u32);

I just noticed there are two tcp connections in the test. One v4 and one v6. Unlikely to collide on seqno, still better to add a sk_cookie to the key of the map, like:

Quoted 6 lines
struct sk_tskey {
        u64 sk_cookie;
        u32 tskey;
};

Use bpf_get_sokcet_cookie(ctx) to get a unique socket cookie.

I will try :)

Quoted 41 lines
Quoted 41 lines
tools/testing/selftests/bpf/progs/so_timestamping.c new file +40 -0
+     __type(value, struct delay_info);
+     __uint(max_entries, 1024);
+} time_map SEC(".maps");
+
+static u64 delay_tolerance_nsec = 10000000000; /* 10 second as an example */
+
+static int bpf_test_sockopt_int(void *ctx, const struct sock *sk,
+                             const struct sockopt_test *t,
+                             int level)
+{
+     int new, opt, tmp;
+
+     opt = t->opt;
+     new = t->new;
+
+     if (bpf_setsockopt(ctx, level, opt, &new, sizeof(new)))
+             return 1;
+
+     if (bpf_getsockopt(ctx, level, opt, &tmp, sizeof(tmp)) ||
+         tmp != new)
+             return 1;
+
+     return 0;
+}
+
+static int bpf_test_socket_sockopt(__u32 i, struct loop_ctx *lc)
+{
+     const struct sockopt_test *t;
+
+     if (i >= ARRAY_SIZE(sol_socket_tests))
+             return 1;
+
+     t = &sol_socket_tests[i];
+     if (!t->opt)
+             return 1;
+
+     return bpf_test_sockopt_int(lc->ctx, lc->sk, t, SOL_SOCKET);
+}
+
+static int bpf_test_sockopt(void *ctx, const struct sock *sk)
Quoted 8 lines

tools/testing/selftests/bpf/progs/so_timestamping.c line 103

Quoted 5 lines
tools/testing/selftests/bpf/progs/so_timestamping.c new file +5 -0
+{
+     struct loop_ctx lc = { .ctx = ctx, .sk = sk, };
+     int n;
+
+     n = bpf_loop(ARRAY_SIZE(sol_socket_tests), bpf_test_socket_sockopt, &lc, 0);

There is only one SK_BPF_CB_FLAGS optname to test, so no need to bpf_loop. Directly do one bpf_setsockopt and one bpf_getsockopt.

I believe in the short run I will support the rx for TCP, so there will be a SK_BPF_CB_RX_TIMESTAMPING flag. So I still want to keep it.

Quoted 66 lines

We can see if there is a need to refactor this timestamp test back to the setget_sockopt.c test later if loop will be needed after adding the UDP support. The setget_sockopt.c does use a loop to test many options at once which is probably where this piece of code (bpf_loop) is borrowed from.

Quoted 5 lines
tools/testing/selftests/bpf/progs/so_timestamping.c new file +5 -0
+     if (n != ARRAY_SIZE(sol_socket_tests))
+             return -1;
+
+     return 0;
+}

tools/testing/selftests/bpf/progs/so_timestamping.c line 118

Quoted 10 lines
tools/testing/selftests/bpf/progs/so_timestamping.c new file +10 -0
+
+static bool bpf_test_access_sockopt(void *ctx)
+{
+     const struct sockopt_test *t;
+     int tmp, ret, i = 0;
+     int level = SOL_SOCKET;
+
+     t = &sol_socket_tests[i];
+
+     for (; t->opt;) {

Same here. Directly do one bpf_setsockopt and one bpf_getsockopt instead of looping.

Quoted 42 lines
tools/testing/selftests/bpf/progs/so_timestamping.c new file +42 -0
+             ret = bpf_setsockopt(ctx, level, t->opt, (void *)&t->new, sizeof(t->new));
+             if (ret != -EOPNOTSUPP)
+                     return true;
+
+             ret = bpf_getsockopt(ctx, level, t->opt, &tmp, sizeof(tmp));
+             if (ret != -EOPNOTSUPP)
+                     return true;
+
+             if (++i >= ARRAY_SIZE(sol_socket_tests))
+                     break;
+     }
+
+     return false;
+}
+
+/* Adding a simple test to see if we can get an expected value */
+static bool bpf_test_access_load_hdr_opt(struct bpf_sock_ops *skops)
+{
+     struct tcp_opt reg_opt;
+     int load_flags = 0;
+     int ret;
+
+     reg_opt.kind = TCPOPT_EXP;
+     reg_opt.len = 0;
+     reg_opt.data32 = 0;
+     ret = bpf_load_hdr_opt(skops, &reg_opt, sizeof(reg_opt), load_flags);
+     if (ret != -EOPNOTSUPP)
+             return true;
+
+     return false;
+}
+
+/* Adding a simple test to see if we can get an expected value */
+static bool bpf_test_access_cb_flags_set(struct bpf_sock_ops *skops)
+{
+     int ret;
+
+     ret = bpf_sock_ops_cb_flags_set(skops, 0);
+     if (ret != -EOPNOTSUPP)
+             return true;
+
+     return false;
Quoted 8 lines

tools/testing/selftests/bpf/progs/so_timestamping.c line 166

Quoted 6 lines
tools/testing/selftests/bpf/progs/so_timestamping.c new file +6 -0
+}
+
+/* In the timestamping callbacks, we're not allowed to call the following
+ * BPF CALLs for the safety concern. Return false if expected.
+ */
+static int bpf_test_access_bpf_calls(struct bpf_sock_ops *skops,

nit. The return value is true/false. Stay with "bool" as the return type.

Oh, my bad.

Quoted 14 lines

tools/testing/selftests/bpf/progs/so_timestamping.c line 176

Quoted 12 lines
tools/testing/selftests/bpf/progs/so_timestamping.c new file +10 -0
+                                  const struct sock *sk)
+{
+     if (bpf_test_access_sockopt(skops))
+             return true;
+
+     if (bpf_test_access_load_hdr_opt(skops))
+             return true;
+
+     if (bpf_test_access_cb_flags_set(skops))
+             return true;

Thanks for adding these negative tests.

Thanks for the careful review.

Thanks,
Jason

Quoted 5 lines
Quoted 5 lines
tools/testing/selftests/bpf/progs/so_timestamping.c new file +4 -0
+
+     return false;
+}
+
#44 lore raw
In reply to Martin KaFai Lau
Quoted 34 lines · Martin KaFai Lau
Quoted 17 lines · Jason Xing

"Timestamping is key to debugging network stack latency. With SO_TIMESTAMPING, bugs that are otherwise incorrectly assumed to be network issues can be attributed to the kernel." This is extracted from the talk "SO_TIMESTAMPING: Powering Fleetwide RPC Monitoring" addressed by Willem de Bruijn at netdevconf 0x17).

There are a few areas that need optimization with the consideration of easier use and less performance impact, which I highlighted and mainly discussed at netconf 2024 with Willem de Bruijn and John Fastabend:
uAPI compatibility, extra system call overhead, and the need for application modification. I initially managed to solve these issues by writing a kernel module that hooks various key functions. However, this approach is not suitable for the next kernel release. Therefore, a BPF extension was proposed. During recent period, Martin KaFai Lau provides invaluable suggestions about BPF along the way. Many thanks here!

Quoted 1 line

In this series, I only support foundamental codes and tx for TCP.

*fundamental*.

May be just "only tx time stamping for TCP is supported..."

Quoted 6 lines

This approach mostly relies on existing SO_TIMESTAMPING feature, users only needs to pass certain flags through bpf_setsocktopt() to a separate tsflags. Please see the last selftest patch in this series.

After this series, we could step by step implement more advanced functions/flags already in SO_TIMESTAMPING feature for bpf extension.

Patch 1-4 and 6-11 can use an extra "bpf:" tag in the subject line. Patch 13 should be "selftests/bpf:" instead of "bpf:" in the subject.

Quoted 6 lines

Please revisit the commit messages of this patch set to check for outdated comments from the earlier revisions. I may have missed some of them.

Overall, it looks close. I will review at your replies later.

Willem, could you also take a look? Thanks.

Will do. Traveling, but took a first quick skim.

#22 lore raw

Re: [PATCH bpf-next v7 11/13] net-timestamp: add a new callback in tcp_tx_timestamp()

In reply to Jason Xing
Quoted 73 lines · Jason Xing
Quoted 8 lines · Martin KaFai Lau
Quoted 3 lines · Jason Xing

Introduce the callback to correlate tcp_sendmsg timestamp with other points, like SND/SW/ACK. We can let bpf trace the beginning of tcp_sendmsg_locked() and fetch the socket addr, so that in

Instead of "fetch the socket addr...", should be "store the sendmsg timestamp at the bpf_sk_storage ...".

I will revise it. Thanks.

Quoted 19 lines
Quoted 6 lines

tcp_tx_timestamp() we can correlate the tskey with the socket addr.

It is accurate since they are under the protect of socket lock. More details can be found in the selftest.

The selftest uses the bpf_sk_storage to store the sendmsg timestamp at fentry/tcp_sendmsg_locked and retrieves it back at tcp_tx_timestamp (i.e. BPF_SOCK_OPS_TS_SND_CB added in this patch).

Quoted 8 lines
---
  include/uapi/linux/bpf.h       | 7 +++++++
  net/ipv4/tcp.c                 | 1 +
  tools/include/uapi/linux/bpf.h | 7 +++++++
  3 files changed, 15 insertions(+)
Quoted 16 lines
Quoted 5 lines
include/uapi/linux/bpf.h +0 -0
index 800122a8abe5..accb3b314fff 100644

include/uapi/linux/bpf.h enum { · line 7060

Quoted 9 lines
include/uapi/linux/bpf.h +6 -0
@@ -7052,6 +7052,13 @@ enum {
                                       * when SK_BPF_CB_TX_TIMESTAMPING
                                       * feature is on.
                                       */
+     BPF_SOCK_OPS_TS_SND_CB,         /* Called when every sendmsg syscall
+                                      * is triggered. For TCP, it stays
+                                      * in the last send process to
+                                      * correlate with tcp_sendmsg timestamp
+                                      * with other timestamping callbacks,
+                                      * like SND/SW/ACK.

Do you have a chance to look at how this will work at UDP?

Sure, I feel like it could not be useful for UDP. Well, things get strange because I did write a long paragraph about this thing which apparently disappeared...

I manage to find what I wrote:
For UDP type, BPF_SOCK_OPS_TS_SND_CB may be not suitable because there are two sending process, 1) lockless path, 2) lock path, which should be handled carefully later. For the former, even though it's unlikely multiple threads access the socket to call sendmsg at the same time, I think we'd better not correlate it like what we do to the TCP case because of the lack of sock lock protection. Considering SND_CB is uapi flag, I think we don't need to forcely add the 'TCP_' prefix in case we need to use it someday.

And one more thing is I'd like to use the v5[1] method in the next round to introduce a new tskey_bpf which is good for UDP type. The new field will not conflict with the tskey in shared info which is generated by sk->sk_tskey in __ip_append_data(). It hardly works if both features (so_timestamping and its bpf extension) exists at the same time. Users could get confused because sometimes they fetch the tskey from skb, sometimes they don't, especially when we have cmsg feature to turn it on/ off per sendmsg. A standalone tskey for bpf extension will be needed. With this tskey_bpf, we can easily correlate the timestamp in sendmsg syscall with other tx points(SND/SW/ACK...).

Quoted 7 lines

[1]: https://lore.kernel.org/all/[email protected]/

If possible, we can leave this question until the UDP support series shows up. I will figure out a better solution :)

In conclusion, it probably won't be used by the UDP type. It's uAPI flag so I consider the compatibility reason.

I don't think this is acceptable. We should aim for an API that can easily be used across protocols, like SO_TIMESTAMPING. Taking a timestamp at sendmsg entry is a useful property for all such protocols.

#42 lore raw
In reply to Jason Xing
Quoted 42 lines · Jason Xing
Quoted 29 lines · Martin KaFai Lau
Quoted 17 lines · Jason Xing

"Timestamping is key to debugging network stack latency. With SO_TIMESTAMPING, bugs that are otherwise incorrectly assumed to be network issues can be attributed to the kernel." This is extracted from the talk "SO_TIMESTAMPING: Powering Fleetwide RPC Monitoring" addressed by Willem de Bruijn at netdevconf 0x17).

There are a few areas that need optimization with the consideration of easier use and less performance impact, which I highlighted and mainly discussed at netconf 2024 with Willem de Bruijn and John Fastabend:
uAPI compatibility, extra system call overhead, and the need for application modification. I initially managed to solve these issues by writing a kernel module that hooks various key functions. However, this approach is not suitable for the next kernel release. Therefore, a BPF extension was proposed. During recent period, Martin KaFai Lau provides invaluable suggestions about BPF along the way. Many thanks here!

Quoted 1 line

In this series, I only support foundamental codes and tx for TCP.

*fundamental*.

May be just "only tx time stamping for TCP is supported..."

Quoted 4 lines

This approach mostly relies on existing SO_TIMESTAMPING feature, users only needs to pass certain flags through bpf_setsocktopt() to a separate tsflags. Please see the last selftest patch in this series.

Quoted 8 lines
Quoted 2 lines

After this series, we could step by step implement more advanced functions/flags already in SO_TIMESTAMPING feature for bpf extension.

Patch 1-4 and 6-11 can use an extra "bpf:" tag in the subject line. Patch 13 should be "selftests/bpf:" instead of "bpf:" in the subject.

Please revisit the commit messages of this patch set to check for outdated comments from the earlier revisions. I may have missed some of them.

Roger that, sir. Thanks for your help!

Quoted 1 line
Quoted 7 lines
Quoted 3 lines

Overall, it looks close. I will review at your replies later.

Willem, could you also take a look? Thanks.

Right, some related parts need reviews from netdev experts as well.

Willem, please help me review this when you're available. No rush :)

I won't have much to add for the BPF side, to be clear.

One small high level commit message point: as submitting-patches suggests, use imperative mood: "adds X" when the patch introduces a feature, not "I add". And "caller gets" rather than "we get".

Specific case, with capitalization issue: "we need to Introduce".

I'll respond to a few inline code elements later. Nothing huge. Also feel free to post the next version and I'll respond to that, if you prefer.

author
#23 lore raw

Re: [PATCH bpf-next v7 11/13] net-timestamp: add a new callback in tcp_tx_timestamp()

In reply to Willem de Bruijn
Quoted 77 lines · Willem de Bruijn
Quoted 75 lines · Jason Xing
Quoted 8 lines · Martin KaFai Lau
Quoted 3 lines · Jason Xing

Introduce the callback to correlate tcp_sendmsg timestamp with other points, like SND/SW/ACK. We can let bpf trace the beginning of tcp_sendmsg_locked() and fetch the socket addr, so that in

Instead of "fetch the socket addr...", should be "store the sendmsg timestamp at the bpf_sk_storage ...".

I will revise it. Thanks.

Quoted 19 lines
Quoted 6 lines

tcp_tx_timestamp() we can correlate the tskey with the socket addr.

It is accurate since they are under the protect of socket lock. More details can be found in the selftest.

The selftest uses the bpf_sk_storage to store the sendmsg timestamp at fentry/tcp_sendmsg_locked and retrieves it back at tcp_tx_timestamp (i.e. BPF_SOCK_OPS_TS_SND_CB added in this patch).

Quoted 8 lines
---
  include/uapi/linux/bpf.h       | 7 +++++++
  net/ipv4/tcp.c                 | 1 +
  tools/include/uapi/linux/bpf.h | 7 +++++++
  3 files changed, 15 insertions(+)
Quoted 16 lines
Quoted 5 lines
include/uapi/linux/bpf.h +0 -0
index 800122a8abe5..accb3b314fff 100644

include/uapi/linux/bpf.h enum { · line 7060

Quoted 9 lines
include/uapi/linux/bpf.h +6 -0
@@ -7052,6 +7052,13 @@ enum {
                                       * when SK_BPF_CB_TX_TIMESTAMPING
                                       * feature is on.
                                       */
+     BPF_SOCK_OPS_TS_SND_CB,         /* Called when every sendmsg syscall
+                                      * is triggered. For TCP, it stays
+                                      * in the last send process to
+                                      * correlate with tcp_sendmsg timestamp
+                                      * with other timestamping callbacks,
+                                      * like SND/SW/ACK.

Do you have a chance to look at how this will work at UDP?

Sure, I feel like it could not be useful for UDP. Well, things get strange because I did write a long paragraph about this thing which apparently disappeared...

I manage to find what I wrote:
For UDP type, BPF_SOCK_OPS_TS_SND_CB may be not suitable because there are two sending process, 1) lockless path, 2) lock path, which should be handled carefully later. For the former, even though it's unlikely multiple threads access the socket to call sendmsg at the same time, I think we'd better not correlate it like what we do to the TCP case because of the lack of sock lock protection. Considering SND_CB is uapi flag, I think we don't need to forcely add the 'TCP_' prefix in case we need to use it someday.

And one more thing is I'd like to use the v5[1] method in the next round to introduce a new tskey_bpf which is good for UDP type. The new field will not conflict with the tskey in shared info which is generated by sk->sk_tskey in __ip_append_data(). It hardly works if both features (so_timestamping and its bpf extension) exists at the same time. Users could get confused because sometimes they fetch the tskey from skb, sometimes they don't, especially when we have cmsg feature to turn it on/ off per sendmsg. A standalone tskey for bpf extension will be needed. With this tskey_bpf, we can easily correlate the timestamp in sendmsg syscall with other tx points(SND/SW/ACK...).

[1]: https://lore.kernel.org/all/[email protected]/

Quoted 8 lines
Quoted 5 lines

If possible, we can leave this question until the UDP support series shows up. I will figure out a better solution :)

In conclusion, it probably won't be used by the UDP type. It's uAPI flag so I consider the compatibility reason.

I don't think this is acceptable. We should aim for an API that can easily be used across protocols, like SO_TIMESTAMPING.

After I revisit the UDP SO_TIMESTAMPING again, my thoughts are adjusted like below:

It's hard to provide an absolutely uniform interface or usage to users for TCP and UDP and even more protocols. Cases can be handled one by one. The main obstacle is how we can correlate the timestamp in sendmsg syscall with other sending timestamps. It's worth noticing that for SO_TIMESTAMPING the sendmsg timestamp is collected in the userspace. For instance, while skb enters the qdisc, we fail to know which skb belongs to which sendmsg.

An idea coming up is to introduce BPF_SOCK_OPS_TS_SND_CB to correlate the sendmsg timestamp with tskey (in tcp_tx_timestamp()) under the protection of socket lock + syscall as the current patch does. But for UDP, it can be lockless. IIUC, there is a very special case where even SO_TIMESTAMPING may get lost: if multiple threads accessing the same socket send UDP packets in parallel, then users could be confused which tskey matches which sendmsg. IIUC, I will not consider this unlikely case, then the UDP case is quite similar to the TCP case.

The scenario for the UDP case is:
1) using fentry bpf to hook the udp_sendmsg() to get the timestamp like TCP does in this series.
2) insert BPF_SOCK_OPS_TS_SND_CB into __ip_append_data() near the SO_TIMESTAMPING code snippets to let bpf program correlate the tskey with timestamp.
Note: tskey in UDP will be handled carefully in a different way because we should support both modes for socket timestamping at the same time.
It's really similar to TCP regardless of handling tskey.

I feel like I have to change back the name to BPF_SOCK_OPS_TS_TCP_SND_CB?

Thanks,
Jason

Quoted 3 lines

Taking a
timestamp at sendmsg entry is a useful property for all such protocols.

author
#43 lore raw
In reply to Willem de Bruijn
Quoted 52 lines · Willem de Bruijn
Quoted 42 lines · Jason Xing
Quoted 29 lines · Martin KaFai Lau
Quoted 17 lines · Jason Xing

"Timestamping is key to debugging network stack latency. With SO_TIMESTAMPING, bugs that are otherwise incorrectly assumed to be network issues can be attributed to the kernel." This is extracted from the talk "SO_TIMESTAMPING: Powering Fleetwide RPC Monitoring" addressed by Willem de Bruijn at netdevconf 0x17).

There are a few areas that need optimization with the consideration of easier use and less performance impact, which I highlighted and mainly discussed at netconf 2024 with Willem de Bruijn and John Fastabend:
uAPI compatibility, extra system call overhead, and the need for application modification. I initially managed to solve these issues by writing a kernel module that hooks various key functions. However, this approach is not suitable for the next kernel release. Therefore, a BPF extension was proposed. During recent period, Martin KaFai Lau provides invaluable suggestions about BPF along the way. Many thanks here!

Quoted 1 line

In this series, I only support foundamental codes and tx for TCP.

*fundamental*.

May be just "only tx time stamping for TCP is supported..."

Quoted 4 lines

This approach mostly relies on existing SO_TIMESTAMPING feature, users only needs to pass certain flags through bpf_setsocktopt() to a separate tsflags. Please see the last selftest patch in this series.

Quoted 8 lines
Quoted 2 lines

After this series, we could step by step implement more advanced functions/flags already in SO_TIMESTAMPING feature for bpf extension.

Patch 1-4 and 6-11 can use an extra "bpf:" tag in the subject line. Patch 13 should be "selftests/bpf:" instead of "bpf:" in the subject.

Please revisit the commit messages of this patch set to check for outdated comments from the earlier revisions. I may have missed some of them.

Roger that, sir. Thanks for your help!

Quoted 1 line
Quoted 7 lines
Quoted 3 lines

Overall, it looks close. I will review at your replies later.

Willem, could you also take a look? Thanks.

Right, some related parts need reviews from netdev experts as well.

Willem, please help me review this when you're available. No rush :)

Quoted 7 lines

I won't have much to add for the BPF side, to be clear.

One small high level commit message point: as submitting-patches suggests, use imperative mood: "adds X" when the patch introduces a feature, not "I add". And "caller gets" rather than "we get".

Specific case, with capitalization issue: "we need to Introduce".

Thanks for learning a new lesson. I will adjust them.

Quoted 4 lines

I'll respond to a few inline code elements later. Nothing huge. Also feel free to post the next version and I'll respond to that, if you prefer.

I will post v8 soon. Thanks for your precious time. Have fun with your trip :p

Thanks,
Jason

author
#24 lore raw

Re: [PATCH bpf-next v7 11/13] net-timestamp: add a new callback in tcp_tx_timestamp()

In reply to Jason Xing
Quoted 109 lines · Jason Xing
Quoted 77 lines · Willem de Bruijn
Quoted 75 lines · Jason Xing
Quoted 8 lines · Martin KaFai Lau
Quoted 3 lines · Jason Xing

Introduce the callback to correlate tcp_sendmsg timestamp with other points, like SND/SW/ACK. We can let bpf trace the beginning of tcp_sendmsg_locked() and fetch the socket addr, so that in

Instead of "fetch the socket addr...", should be "store the sendmsg timestamp at the bpf_sk_storage ...".

I will revise it. Thanks.

Quoted 19 lines
Quoted 6 lines

tcp_tx_timestamp() we can correlate the tskey with the socket addr.

It is accurate since they are under the protect of socket lock. More details can be found in the selftest.

The selftest uses the bpf_sk_storage to store the sendmsg timestamp at fentry/tcp_sendmsg_locked and retrieves it back at tcp_tx_timestamp (i.e. BPF_SOCK_OPS_TS_SND_CB added in this patch).

Quoted 8 lines
---
  include/uapi/linux/bpf.h       | 7 +++++++
  net/ipv4/tcp.c                 | 1 +
  tools/include/uapi/linux/bpf.h | 7 +++++++
  3 files changed, 15 insertions(+)
Quoted 16 lines
Quoted 5 lines
include/uapi/linux/bpf.h +0 -0
index 800122a8abe5..accb3b314fff 100644

include/uapi/linux/bpf.h enum { · line 7060

Quoted 9 lines
include/uapi/linux/bpf.h +6 -0
@@ -7052,6 +7052,13 @@ enum {
                                       * when SK_BPF_CB_TX_TIMESTAMPING
                                       * feature is on.
                                       */
+     BPF_SOCK_OPS_TS_SND_CB,         /* Called when every sendmsg syscall
+                                      * is triggered. For TCP, it stays
+                                      * in the last send process to
+                                      * correlate with tcp_sendmsg timestamp
+                                      * with other timestamping callbacks,
+                                      * like SND/SW/ACK.

Do you have a chance to look at how this will work at UDP?

Sure, I feel like it could not be useful for UDP. Well, things get strange because I did write a long paragraph about this thing which apparently disappeared...

I manage to find what I wrote:
For UDP type, BPF_SOCK_OPS_TS_SND_CB may be not suitable because there are two sending process, 1) lockless path, 2) lock path, which should be handled carefully later. For the former, even though it's unlikely multiple threads access the socket to call sendmsg at the same time, I think we'd better not correlate it like what we do to the TCP case because of the lack of sock lock protection. Considering SND_CB is uapi flag, I think we don't need to forcely add the 'TCP_' prefix in case we need to use it someday.

And one more thing is I'd like to use the v5[1] method in the next round to introduce a new tskey_bpf which is good for UDP type. The new field will not conflict with the tskey in shared info which is generated by sk->sk_tskey in __ip_append_data(). It hardly works if both features (so_timestamping and its bpf extension) exists at the same time. Users could get confused because sometimes they fetch the tskey from skb, sometimes they don't, especially when we have cmsg feature to turn it on/ off per sendmsg. A standalone tskey for bpf extension will be needed. With this tskey_bpf, we can easily correlate the timestamp in sendmsg syscall with other tx points(SND/SW/ACK...).

[1]: https://lore.kernel.org/all/[email protected]/

Quoted 8 lines
Quoted 5 lines

If possible, we can leave this question until the UDP support series shows up. I will figure out a better solution :)

In conclusion, it probably won't be used by the UDP type. It's uAPI flag so I consider the compatibility reason.

I don't think this is acceptable. We should aim for an API that can easily be used across protocols, like SO_TIMESTAMPING.

After I revisit the UDP SO_TIMESTAMPING again, my thoughts are adjusted like below:

It's hard to provide an absolutely uniform interface or usage to users for TCP and UDP and even more protocols. Cases can be handled one by one. The main obstacle is how we can correlate the timestamp in sendmsg syscall with other sending timestamps. It's worth noticing that for SO_TIMESTAMPING the sendmsg timestamp is collected in the userspace. For instance, while skb enters the qdisc, we fail to know which skb belongs to which sendmsg.

An idea coming up is to introduce BPF_SOCK_OPS_TS_SND_CB to correlate the sendmsg timestamp with tskey (in tcp_tx_timestamp()) under the protection of socket lock + syscall as the current patch does. But for UDP, it can be lockless. IIUC, there is a very special case where even SO_TIMESTAMPING may get lost: if multiple threads accessing the same socket send UDP packets in parallel, then users could be confused which tskey matches which sendmsg. IIUC, I will not consider this unlikely case, then the UDP case is quite similar to the TCP case.

Quoted 11 lines

The scenario for the UDP case is:
1) using fentry bpf to hook the udp_sendmsg() to get the timestamp like TCP does in this series.
2) insert BPF_SOCK_OPS_TS_SND_CB into __ip_append_data() near the SO_TIMESTAMPING code snippets to let bpf program correlate the tskey with timestamp.
Note: tskey in UDP will be handled carefully in a different way because we should support both modes for socket timestamping at the same time.
It's really similar to TCP regardless of handling tskey.

To be more precise in case you don't have much time to read the above long paragraph, BPF_SOCK_OPS_TS_SND_CB is mainly used to correlate sendmsg timestamp with corresponding tskey.

1. For TCP, we can correlate it in tcp_tx_timestamp() like this patch does.

2. For UDP, we can correlate in __ip_append_data() along with those tskey initialization, assuming there are no multiple threads calling locklessly ip_make_skb(). Locked path (udp_sendmsg()->ip_append_data()) works like TCP under the socket lock protection, so it can be easily handled. Lockless path (udp_sendmsg()->ip_make_skb()) can be visited by multiple threads at the same time, which should be handled properly. I prefer to implement the bpf extension for IPCORK_TS_OPT_ID, which should be another topic, I think. This might be the only one corner case, IIUC?

Overall I think BPF_SOCK_OPS_TS_SND_CB can work across protocols to do the correlation job.

To be on the safe side, I can change the name BPF_SOCK_OPS_TS_SND_CB to BPF_SOCK_OPS_TS_TCP_SND_CB just in case this approach is not the best one. What do you think about this?

[1]
commit 4aecca4c76808f3736056d18ff510df80424bc9f

Author: Vadim Fedorenko <[email protected]>
Date:   Tue Oct 1 05:57:14 2024 -0700

net_tstamp: add SCM_TS_OPT_ID to provide OPT_ID in control message

SOF_TIMESTAMPING_OPT_ID socket option flag gives a way to correlate TX timestamps and packets sent via socket. Unfortunately, there is no way to reliably predict socket timestamp ID value in case of error returned by sendmsg. For UDP sockets it's impossible because of lockless nature of UDP transmit, several threads may send packets in parallel. In case of RAW sockets MSG_MORE option makes things complicated. More details are in the conversation [1].
This patch adds new control message type to give user-space software an opportunity to control the mapping between packets and values by providing ID with each sendmsg for UDP sockets. The documentation is also added in this patch.

[1] https://lore.kernel.org/netdev/CALCETrU0jB+kg0mhV6A8mrHfTE1D1pr1SD_B9Eaa9aDPfgHdtA@mail.gmail.com/

Thanks,
Jason

author
#25 lore raw

Re: [PATCH bpf-next v7 11/13] net-timestamp: add a new callback in tcp_tx_timestamp()

Quoted 165 lines · Jason Xing
Quoted 109 lines · Jason Xing
Quoted 77 lines · Willem de Bruijn
Quoted 75 lines · Jason Xing
Quoted 8 lines · Martin KaFai Lau
Quoted 3 lines · Jason Xing

Introduce the callback to correlate tcp_sendmsg timestamp with other points, like SND/SW/ACK. We can let bpf trace the beginning of tcp_sendmsg_locked() and fetch the socket addr, so that in

Instead of "fetch the socket addr...", should be "store the sendmsg timestamp at the bpf_sk_storage ...".

I will revise it. Thanks.

Quoted 19 lines
Quoted 6 lines

tcp_tx_timestamp() we can correlate the tskey with the socket addr.

It is accurate since they are under the protect of socket lock. More details can be found in the selftest.

The selftest uses the bpf_sk_storage to store the sendmsg timestamp at fentry/tcp_sendmsg_locked and retrieves it back at tcp_tx_timestamp (i.e. BPF_SOCK_OPS_TS_SND_CB added in this patch).

Quoted 8 lines
---
  include/uapi/linux/bpf.h       | 7 +++++++
  net/ipv4/tcp.c                 | 1 +
  tools/include/uapi/linux/bpf.h | 7 +++++++
  3 files changed, 15 insertions(+)
Quoted 16 lines
Quoted 5 lines
include/uapi/linux/bpf.h +0 -0
index 800122a8abe5..accb3b314fff 100644

include/uapi/linux/bpf.h enum { · line 7060

Quoted 9 lines
include/uapi/linux/bpf.h +6 -0
@@ -7052,6 +7052,13 @@ enum {
                                       * when SK_BPF_CB_TX_TIMESTAMPING
                                       * feature is on.
                                       */
+     BPF_SOCK_OPS_TS_SND_CB,         /* Called when every sendmsg syscall
+                                      * is triggered. For TCP, it stays
+                                      * in the last send process to
+                                      * correlate with tcp_sendmsg timestamp
+                                      * with other timestamping callbacks,
+                                      * like SND/SW/ACK.

Do you have a chance to look at how this will work at UDP?

Sure, I feel like it could not be useful for UDP. Well, things get strange because I did write a long paragraph about this thing which apparently disappeared...

I manage to find what I wrote:
For UDP type, BPF_SOCK_OPS_TS_SND_CB may be not suitable because there are two sending process, 1) lockless path, 2) lock path, which should be handled carefully later. For the former, even though it's unlikely multiple threads access the socket to call sendmsg at the same time, I think we'd better not correlate it like what we do to the TCP case because of the lack of sock lock protection. Considering SND_CB is uapi flag, I think we don't need to forcely add the 'TCP_' prefix in case we need to use it someday.

And one more thing is I'd like to use the v5[1] method in the next round to introduce a new tskey_bpf which is good for UDP type. The new field will not conflict with the tskey in shared info which is generated by sk->sk_tskey in __ip_append_data(). It hardly works if both features (so_timestamping and its bpf extension) exists at the same time. Users could get confused because sometimes they fetch the tskey from skb, sometimes they don't, especially when we have cmsg feature to turn it on/ off per sendmsg. A standalone tskey for bpf extension will be needed. With this tskey_bpf, we can easily correlate the timestamp in sendmsg syscall with other tx points(SND/SW/ACK...).

[1]: https://lore.kernel.org/all/[email protected]/

Quoted 8 lines
Quoted 5 lines

If possible, we can leave this question until the UDP support series shows up. I will figure out a better solution :)

In conclusion, it probably won't be used by the UDP type. It's uAPI flag so I consider the compatibility reason.

I don't think this is acceptable. We should aim for an API that can easily be used across protocols, like SO_TIMESTAMPING.

After I revisit the UDP SO_TIMESTAMPING again, my thoughts are adjusted like below:

It's hard to provide an absolutely uniform interface or usage to users for TCP and UDP and even more protocols. Cases can be handled one by one. The main obstacle is how we can correlate the timestamp in sendmsg syscall with other sending timestamps. It's worth noticing that for SO_TIMESTAMPING the sendmsg timestamp is collected in the userspace. For instance, while skb enters the qdisc, we fail to know which skb belongs to which sendmsg.

An idea coming up is to introduce BPF_SOCK_OPS_TS_SND_CB to correlate the sendmsg timestamp with tskey (in tcp_tx_timestamp()) under the protection of socket lock + syscall as the current patch does. But for UDP, it can be lockless. IIUC, there is a very special case where even SO_TIMESTAMPING may get lost: if multiple threads accessing the same socket send UDP packets in parallel, then users could be confused which tskey matches which sendmsg. IIUC, I will not consider this unlikely case, then the UDP case is quite similar to the TCP case.

Quoted 11 lines

The scenario for the UDP case is:
1) using fentry bpf to hook the udp_sendmsg() to get the timestamp like TCP does in this series.
2) insert BPF_SOCK_OPS_TS_SND_CB into __ip_append_data() near the SO_TIMESTAMPING code snippets to let bpf program correlate the tskey with timestamp.
Note: tskey in UDP will be handled carefully in a different way because we should support both modes for socket timestamping at the same time.
It's really similar to TCP regardless of handling tskey.

To be more precise in case you don't have much time to read the above long paragraph, BPF_SOCK_OPS_TS_SND_CB is mainly used to correlate sendmsg timestamp with corresponding tskey.

1. For TCP, we can correlate it in tcp_tx_timestamp() like this patch does.

2. For UDP, we can correlate in __ip_append_data() along with those tskey initialization, assuming there are no multiple threads calling locklessly ip_make_skb(). Locked path (udp_sendmsg()->ip_append_data()) works like TCP under the socket lock protection, so it can be easily handled. Lockless path (udp_sendmsg()->ip_make_skb()) can be visited by multiple threads at the same time, which should be handled properly. I prefer to implement the bpf extension for IPCORK_TS_OPT_ID, which should be another topic, I think. This might be the only one corner case, IIUC?

Overall I think BPF_SOCK_OPS_TS_SND_CB can work across protocols to do the correlation job.

To be on the safe side, I can change the name BPF_SOCK_OPS_TS_SND_CB to BPF_SOCK_OPS_TS_TCP_SND_CB just in case this approach is not the best one. What do you think about this?

[1]
commit 4aecca4c76808f3736056d18ff510df80424bc9f

Author: Vadim Fedorenko <[email protected]>
Date:   Tue Oct 1 05:57:14 2024 -0700

net_tstamp: add SCM_TS_OPT_ID to provide OPT_ID in control message

SOF_TIMESTAMPING_OPT_ID socket option flag gives a way to correlate TX timestamps and packets sent via socket. Unfortunately, there is no way to reliably predict socket timestamp ID value in case of error returned by sendmsg. For UDP sockets it's impossible because of lockless nature of UDP transmit, several threads may send packets in parallel. In case of RAW sockets MSG_MORE option makes things complicated. More details are in the conversation [1].
This patch adds new control message type to give user-space software an opportunity to control the mapping between packets and values by providing ID with each sendmsg for UDP sockets. The documentation is also added in this patch.

Quoted 2 lines

[1] https://lore.kernel.org/netdev/CALCETrU0jB+kg0mhV6A8mrHfTE1D1pr1SD_B9Eaa9aDPfgHdtA@mail.gmail.com/

Oh, I came up with a feasible approach for UDP protocol:
1. introduce a field ts_opt_id_bpf which works like ts_opt_id to allow the bpf program to fully take control of the management of tskey.
2. use fentry hook udp_sendmsg(), and introduce a callback function like BPF_SOCK_OPS_TIMEOUT_INIT in kernel to initialize the ts_opt_id_bpf with tskey that bpf prog generates. We can directly use BPF_SOCK_OPS_TS_SND_CB.
3. modify the SCM_TS_OPT_ID logic to support bpf extension so that the newly added field ts_opt_id_bpf can be passed to the skb_shinfo(skb)->tskey in __ip_append_data().

In this way, this approach can also be extended for other protocols.

Thanks,
Jason

#26 lore raw

Re: [PATCH bpf-next v7 11/13] net-timestamp: add a new callback in tcp_tx_timestamp()

In reply to Jason Xing
Quoted 90 lines · Jason Xing
Quoted 89 lines · Jason Xing
Quoted 85 lines · Willem de Bruijn
Quoted 73 lines · Jason Xing
Quoted 8 lines · Martin KaFai Lau
Quoted 3 lines · Jason Xing

Introduce the callback to correlate tcp_sendmsg timestamp with other points, like SND/SW/ACK. We can let bpf trace the beginning of tcp_sendmsg_locked() and fetch the socket addr, so that in

Instead of "fetch the socket addr...", should be "store the sendmsg timestamp at the bpf_sk_storage ...".

I will revise it. Thanks.

Quoted 19 lines
Quoted 6 lines

tcp_tx_timestamp() we can correlate the tskey with the socket addr.

It is accurate since they are under the protect of socket lock. More details can be found in the selftest.

The selftest uses the bpf_sk_storage to store the sendmsg timestamp at fentry/tcp_sendmsg_locked and retrieves it back at tcp_tx_timestamp (i.e. BPF_SOCK_OPS_TS_SND_CB added in this patch).

Quoted 8 lines
---
  include/uapi/linux/bpf.h       | 7 +++++++
  net/ipv4/tcp.c                 | 1 +
  tools/include/uapi/linux/bpf.h | 7 +++++++
  3 files changed, 15 insertions(+)
Quoted 16 lines
Quoted 5 lines
include/uapi/linux/bpf.h +0 -0
index 800122a8abe5..accb3b314fff 100644

include/uapi/linux/bpf.h enum { · line 7060

Quoted 9 lines
include/uapi/linux/bpf.h +6 -0
@@ -7052,6 +7052,13 @@ enum {
                                       * when SK_BPF_CB_TX_TIMESTAMPING
                                       * feature is on.
                                       */
+     BPF_SOCK_OPS_TS_SND_CB,         /* Called when every sendmsg syscall
+                                      * is triggered. For TCP, it stays
+                                      * in the last send process to
+                                      * correlate with tcp_sendmsg timestamp
+                                      * with other timestamping callbacks,
+                                      * like SND/SW/ACK.

Do you have a chance to look at how this will work at UDP?

Sure, I feel like it could not be useful for UDP. Well, things get strange because I did write a long paragraph about this thing which apparently disappeared...

I manage to find what I wrote:
For UDP type, BPF_SOCK_OPS_TS_SND_CB may be not suitable because there are two sending process, 1) lockless path, 2) lock path, which should be handled carefully later. For the former, even though it's unlikely multiple threads access the socket to call sendmsg at the same time, I think we'd better not correlate it like what we do to the TCP case because of the lack of sock lock protection. Considering SND_CB is uapi flag, I think we don't need to forcely add the 'TCP_' prefix in case we need to use it someday.

And one more thing is I'd like to use the v5[1] method in the next round to introduce a new tskey_bpf which is good for UDP type. The new field will not conflict with the tskey in shared info which is generated by sk->sk_tskey in __ip_append_data(). It hardly works if both features (so_timestamping and its bpf extension) exists at the same time. Users could get confused because sometimes they fetch the tskey from skb, sometimes they don't, especially when we have cmsg feature to turn it on/ off per sendmsg. A standalone tskey for bpf extension will be needed. With this tskey_bpf, we can easily correlate the timestamp in sendmsg syscall with other tx points(SND/SW/ACK...).

Quoted 7 lines

[1]: https://lore.kernel.org/all/[email protected]/

If possible, we can leave this question until the UDP support series shows up. I will figure out a better solution :)

In conclusion, it probably won't be used by the UDP type. It's uAPI flag so I consider the compatibility reason.

I don't think this is acceptable. We should aim for an API that can easily be used across protocols, like SO_TIMESTAMPING.

Quoted 6 lines
Quoted 6 lines

After I revisit the UDP SO_TIMESTAMPING again, my thoughts are adjusted like below:

It's hard to provide an absolutely uniform interface or usage to users for TCP and UDP and even more protocols. Cases can be handled one by one.

We should try hard. SO_TIMESTAMPING is uniform across protocols. An interface that is not is just hard to use.

Quoted 6 lines
Quoted 6 lines

The main obstacle is how we can correlate the timestamp in sendmsg syscall with other sending timestamps. It's worth noticing that for SO_TIMESTAMPING the sendmsg timestamp is collected in the userspace. For instance, while skb enters the qdisc, we fail to know which skb belongs to which sendmsg.

Quoted 7 lines
Quoted 7 lines

An idea coming up is to introduce BPF_SOCK_OPS_TS_SND_CB to correlate the sendmsg timestamp with tskey (in tcp_tx_timestamp()) under the protection of socket lock + syscall as the current patch does. But for UDP, it can be lockless. IIUC, there is a very special case where even SO_TIMESTAMPING may get lost: if multiple threads accessing the same socket send UDP packets in parallel, then users could be confused which tskey matches which sendmsg.

This is a known issue for lockless datagram sockets.

With SO_TIMESTAMPING, but the use of timestamping and of concurrent sendmsg calls is under control of the process, so it only shoots itself in the foot.

With BPF timestamping, a process may confuse a third party admin, so the situation is slightly different.

Quoted 21 lines
Quoted 14 lines

IIUC, I will not consider this
unlikely case, then the UDP case is quite similar to the TCP case.

The scenario for the UDP case is:
1) using fentry bpf to hook the udp_sendmsg() to get the timestamp like TCP does in this series.
2) insert BPF_SOCK_OPS_TS_SND_CB into __ip_append_data() near the SO_TIMESTAMPING code snippets to let bpf program correlate the tskey with timestamp.
Note: tskey in UDP will be handled carefully in a different way because we should support both modes for socket timestamping at the same time.
It's really similar to TCP regardless of handling tskey.

To be more precise in case you don't have much time to read the above long paragraph, BPF_SOCK_OPS_TS_SND_CB is mainly used to correlate sendmsg timestamp with corresponding tskey.

1. For TCP, we can correlate it in tcp_tx_timestamp() like this patch does.

Quoted 7 lines

2. For UDP, we can correlate in __ip_append_data() along with those tskey initialization, assuming there are no multiple threads calling locklessly ip_make_skb(). Locked path (udp_sendmsg()->ip_append_data()) works like TCP under the socket lock protection, so it can be easily handled. Lockless path (udp_sendmsg()->ip_make_skb()) can be visited by multiple threads at the same time, which should be handled properly.

Different hook points is fine, as UDP (and RAW) uses __ip_append_data or more importantly ip_send_skb, while TCP uses ip_queue_xmit.

As long as the API is the same: the operation (BPF_SOCK_OPS_TS_SND_CB) and the behavior of that operation. Subject to the usual distinction between protocol behavior (bytestream vs datagram).

Quoted 3 lines

I prefer to implement
the bpf extension for IPCORK_TS_OPT_ID, which should be another topic, I think. This might be the only one corner case, IIUC?

This sounds like an entirely different topic? Not sure what this is.

Quoted 30 lines

Overall I think BPF_SOCK_OPS_TS_SND_CB can work across protocols to do the correlation job.

To be on the safe side, I can change the name BPF_SOCK_OPS_TS_SND_CB to BPF_SOCK_OPS_TS_TCP_SND_CB just in case this approach is not the best one. What do you think about this?

[1]
commit 4aecca4c76808f3736056d18ff510df80424bc9f

Author: Vadim Fedorenko <[email protected]>
Date:   Tue Oct 1 05:57:14 2024 -0700

net_tstamp: add SCM_TS_OPT_ID to provide OPT_ID in control message

SOF_TIMESTAMPING_OPT_ID socket option flag gives a way to correlate TX timestamps and packets sent via socket. Unfortunately, there is no way to reliably predict socket timestamp ID value in case of error returned by sendmsg. For UDP sockets it's impossible because of lockless nature of UDP transmit, several threads may send packets in parallel. In case of RAW sockets MSG_MORE option makes things complicated. More details are in the conversation [1].
This patch adds new control message type to give user-space software an opportunity to control the mapping between packets and values by providing ID with each sendmsg for UDP sockets. The documentation is also added in this patch.

[1] https://lore.kernel.org/netdev/CALCETrU0jB+kg0mhV6A8mrHfTE1D1pr1SD_B9Eaa9aDPfgHdtA@mail.gmail.com/

Thanks,
Jason

author
#27 lore raw

Re: [PATCH bpf-next v7 11/13] net-timestamp: add a new callback in tcp_tx_timestamp()

Quoted 116 lines · Willem de Bruijn
Quoted 90 lines · Jason Xing
Quoted 89 lines · Jason Xing
Quoted 85 lines · Willem de Bruijn
Quoted 73 lines · Jason Xing
Quoted 8 lines · Martin KaFai Lau
Quoted 3 lines · Jason Xing

Introduce the callback to correlate tcp_sendmsg timestamp with other points, like SND/SW/ACK. We can let bpf trace the beginning of tcp_sendmsg_locked() and fetch the socket addr, so that in

Instead of "fetch the socket addr...", should be "store the sendmsg timestamp at the bpf_sk_storage ...".

I will revise it. Thanks.

Quoted 19 lines
Quoted 6 lines

tcp_tx_timestamp() we can correlate the tskey with the socket addr.

It is accurate since they are under the protect of socket lock. More details can be found in the selftest.

The selftest uses the bpf_sk_storage to store the sendmsg timestamp at fentry/tcp_sendmsg_locked and retrieves it back at tcp_tx_timestamp (i.e. BPF_SOCK_OPS_TS_SND_CB added in this patch).

Quoted 8 lines
---
  include/uapi/linux/bpf.h       | 7 +++++++
  net/ipv4/tcp.c                 | 1 +
  tools/include/uapi/linux/bpf.h | 7 +++++++
  3 files changed, 15 insertions(+)
Quoted 16 lines
Quoted 5 lines
include/uapi/linux/bpf.h +0 -0
index 800122a8abe5..accb3b314fff 100644

include/uapi/linux/bpf.h enum { · line 7060

Quoted 9 lines
include/uapi/linux/bpf.h +6 -0
@@ -7052,6 +7052,13 @@ enum {
                                       * when SK_BPF_CB_TX_TIMESTAMPING
                                       * feature is on.
                                       */
+     BPF_SOCK_OPS_TS_SND_CB,         /* Called when every sendmsg syscall
+                                      * is triggered. For TCP, it stays
+                                      * in the last send process to
+                                      * correlate with tcp_sendmsg timestamp
+                                      * with other timestamping callbacks,
+                                      * like SND/SW/ACK.

Do you have a chance to look at how this will work at UDP?

Sure, I feel like it could not be useful for UDP. Well, things get strange because I did write a long paragraph about this thing which apparently disappeared...

I manage to find what I wrote:
For UDP type, BPF_SOCK_OPS_TS_SND_CB may be not suitable because there are two sending process, 1) lockless path, 2) lock path, which should be handled carefully later. For the former, even though it's unlikely multiple threads access the socket to call sendmsg at the same time, I think we'd better not correlate it like what we do to the TCP case because of the lack of sock lock protection. Considering SND_CB is uapi flag, I think we don't need to forcely add the 'TCP_' prefix in case we need to use it someday.

And one more thing is I'd like to use the v5[1] method in the next round to introduce a new tskey_bpf which is good for UDP type. The new field will not conflict with the tskey in shared info which is generated by sk->sk_tskey in __ip_append_data(). It hardly works if both features (so_timestamping and its bpf extension) exists at the same time. Users could get confused because sometimes they fetch the tskey from skb, sometimes they don't, especially when we have cmsg feature to turn it on/ off per sendmsg. A standalone tskey for bpf extension will be needed. With this tskey_bpf, we can easily correlate the timestamp in sendmsg syscall with other tx points(SND/SW/ACK...).

Quoted 7 lines

[1]: https://lore.kernel.org/all/[email protected]/

If possible, we can leave this question until the UDP support series shows up. I will figure out a better solution :)

In conclusion, it probably won't be used by the UDP type. It's uAPI flag so I consider the compatibility reason.

I don't think this is acceptable. We should aim for an API that can easily be used across protocols, like SO_TIMESTAMPING.

Quoted 6 lines
Quoted 6 lines

After I revisit the UDP SO_TIMESTAMPING again, my thoughts are adjusted like below:

It's hard to provide an absolutely uniform interface or usage to users for TCP and UDP and even more protocols. Cases can be handled one by one.

We should try hard. SO_TIMESTAMPING is uniform across protocols. An interface that is not is just hard to use.

Quoted 6 lines
Quoted 6 lines

The main obstacle is how we can correlate the timestamp in sendmsg syscall with other sending timestamps. It's worth noticing that for SO_TIMESTAMPING the sendmsg timestamp is collected in the userspace. For instance, while skb enters the qdisc, we fail to know which skb belongs to which sendmsg.

Quoted 7 lines
Quoted 7 lines

An idea coming up is to introduce BPF_SOCK_OPS_TS_SND_CB to correlate the sendmsg timestamp with tskey (in tcp_tx_timestamp()) under the protection of socket lock + syscall as the current patch does. But for UDP, it can be lockless. IIUC, there is a very special case where even SO_TIMESTAMPING may get lost: if multiple threads accessing the same socket send UDP packets in parallel, then users could be confused which tskey matches which sendmsg.

Quoted 8 lines

This is a known issue for lockless datagram sockets.

With SO_TIMESTAMPING, but the use of timestamping and of concurrent sendmsg calls is under control of the process, so it only shoots itself in the foot.

With BPF timestamping, a process may confuse a third party admin, so the situation is slightly different.

Agreed.

Quoted 30 lines
Quoted 22 lines
Quoted 4 lines

IIUC, I will not consider this
unlikely case, then the UDP case is quite similar to the TCP case.

Quoted 11 lines

The scenario for the UDP case is:
1) using fentry bpf to hook the udp_sendmsg() to get the timestamp like TCP does in this series.
2) insert BPF_SOCK_OPS_TS_SND_CB into __ip_append_data() near the SO_TIMESTAMPING code snippets to let bpf program correlate the tskey with timestamp.
Note: tskey in UDP will be handled carefully in a different way because we should support both modes for socket timestamping at the same time.
It's really similar to TCP regardless of handling tskey.

To be more precise in case you don't have much time to read the above long paragraph, BPF_SOCK_OPS_TS_SND_CB is mainly used to correlate sendmsg timestamp with corresponding tskey.

1. For TCP, we can correlate it in tcp_tx_timestamp() like this patch does.

Quoted 7 lines

2. For UDP, we can correlate in __ip_append_data() along with those tskey initialization, assuming there are no multiple threads calling locklessly ip_make_skb(). Locked path (udp_sendmsg()->ip_append_data()) works like TCP under the socket lock protection, so it can be easily handled. Lockless path (udp_sendmsg()->ip_make_skb()) can be visited by multiple threads at the same time, which should be handled properly.

Quoted 1 line

Different hook points is fine, as UDP (and RAW) uses __ip_append_data

Then this approach (introducing this new flag) is feasible. Sorry that last night I wrote such a long paragraph which buried something important. Because of that, I rephrase the whole idea about how to let UDP work with this kind of new flag in [patch v8 11/12]. Link is
https://lore.kernel.org/all/CAL+tcoCmXcDot-855XYU7PKCiGvJL=O3CQBGuOTRAs2_=Ys=gg@mail.gmail.com/

Quoted 1 line

or more importantly ip_send_skb, while TCP uses ip_queue_xmit.

For TCP, we use tcp_tx_timestamp to finish the map between sendmsg timestamp and tskey.

Quoted 4 lines

As long as the API is the same: the operation (BPF_SOCK_OPS_TS_SND_CB) and the behavior of that operation. Subject to the usual distinction between protocol behavior (bytestream vs datagram).

I see your point.

Quoted 6 lines
Quoted 4 lines

I prefer to implement
the bpf extension for IPCORK_TS_OPT_ID, which should be another topic, I think. This might be the only one corner case, IIUC?

This sounds like an entirely different topic? Not sure what this is.

Not really a different topic. I mean let bpf prog take the whole control of setting the tskey, then with this BPF_SOCK_OPS_TS_SND_CB flag we can correlate the sendmsg timestamp with tskey. So It has something to do with the usage of UDP. Please take a look at that link to patch 11/12. For TCP, we don't need to care about the value of tskey which has already been taken care of by SO_TIMESTAMPING. So it is slightly different. I'm not sure if this kind of usage is acceptable?

Thanks,
Jason

Quoted 33 lines
Quoted 31 lines

Overall I think BPF_SOCK_OPS_TS_SND_CB can work across protocols to do the correlation job.

To be on the safe side, I can change the name BPF_SOCK_OPS_TS_SND_CB to BPF_SOCK_OPS_TS_TCP_SND_CB just in case this approach is not the best one. What do you think about this?

[1]
commit 4aecca4c76808f3736056d18ff510df80424bc9f

Author: Vadim Fedorenko <[email protected]>
Date:   Tue Oct 1 05:57:14 2024 -0700

net_tstamp: add SCM_TS_OPT_ID to provide OPT_ID in control message

SOF_TIMESTAMPING_OPT_ID socket option flag gives a way to correlate TX timestamps and packets sent via socket. Unfortunately, there is no way to reliably predict socket timestamp ID value in case of error returned by sendmsg. For UDP sockets it's impossible because of lockless nature of UDP transmit, several threads may send packets in parallel. In case of RAW sockets MSG_MORE option makes things complicated. More details are in the conversation [1].
This patch adds new control message type to give user-space software an opportunity to control the mapping between packets and values by providing ID with each sendmsg for UDP sockets. The documentation is also added in this patch.

[1] https://lore.kernel.org/netdev/CALCETrU0jB+kg0mhV6A8mrHfTE1D1pr1SD_B9Eaa9aDPfgHdtA@mail.gmail.com/

Thanks,
Jason

#28 lore raw

Re: [PATCH bpf-next v7 11/13] net-timestamp: add a new callback in tcp_tx_timestamp()

Quoted 186 lines · Jason Xing
Quoted 116 lines · Willem de Bruijn
Quoted 90 lines · Jason Xing
Quoted 89 lines · Jason Xing
Quoted 85 lines · Willem de Bruijn
Quoted 73 lines · Jason Xing
Quoted 8 lines · Martin KaFai Lau
Quoted 3 lines · Jason Xing

Introduce the callback to correlate tcp_sendmsg timestamp with other points, like SND/SW/ACK. We can let bpf trace the beginning of tcp_sendmsg_locked() and fetch the socket addr, so that in

Instead of "fetch the socket addr...", should be "store the sendmsg timestamp at the bpf_sk_storage ...".

I will revise it. Thanks.

Quoted 19 lines
Quoted 6 lines

tcp_tx_timestamp() we can correlate the tskey with the socket addr.

It is accurate since they are under the protect of socket lock. More details can be found in the selftest.

The selftest uses the bpf_sk_storage to store the sendmsg timestamp at fentry/tcp_sendmsg_locked and retrieves it back at tcp_tx_timestamp (i.e. BPF_SOCK_OPS_TS_SND_CB added in this patch).

Quoted 8 lines
---
  include/uapi/linux/bpf.h       | 7 +++++++
  net/ipv4/tcp.c                 | 1 +
  tools/include/uapi/linux/bpf.h | 7 +++++++
  3 files changed, 15 insertions(+)
Quoted 16 lines
Quoted 5 lines
include/uapi/linux/bpf.h +0 -0
index 800122a8abe5..accb3b314fff 100644

include/uapi/linux/bpf.h enum { · line 7060

Quoted 9 lines
include/uapi/linux/bpf.h +6 -0
@@ -7052,6 +7052,13 @@ enum {
                                       * when SK_BPF_CB_TX_TIMESTAMPING
                                       * feature is on.
                                       */
+     BPF_SOCK_OPS_TS_SND_CB,         /* Called when every sendmsg syscall
+                                      * is triggered. For TCP, it stays
+                                      * in the last send process to
+                                      * correlate with tcp_sendmsg timestamp
+                                      * with other timestamping callbacks,
+                                      * like SND/SW/ACK.

Do you have a chance to look at how this will work at UDP?

Sure, I feel like it could not be useful for UDP. Well, things get strange because I did write a long paragraph about this thing which apparently disappeared...

I manage to find what I wrote:
For UDP type, BPF_SOCK_OPS_TS_SND_CB may be not suitable because there are two sending process, 1) lockless path, 2) lock path, which should be handled carefully later. For the former, even though it's unlikely multiple threads access the socket to call sendmsg at the same time, I think we'd better not correlate it like what we do to the TCP case because of the lack of sock lock protection. Considering SND_CB is uapi flag, I think we don't need to forcely add the 'TCP_' prefix in case we need to use it someday.

And one more thing is I'd like to use the v5[1] method in the next round to introduce a new tskey_bpf which is good for UDP type. The new field will not conflict with the tskey in shared info which is generated by sk->sk_tskey in __ip_append_data(). It hardly works if both features (so_timestamping and its bpf extension) exists at the same time. Users could get confused because sometimes they fetch the tskey from skb, sometimes they don't, especially when we have cmsg feature to turn it on/ off per sendmsg. A standalone tskey for bpf extension will be needed. With this tskey_bpf, we can easily correlate the timestamp in sendmsg syscall with other tx points(SND/SW/ACK...).

Quoted 7 lines

[1]: https://lore.kernel.org/all/[email protected]/

If possible, we can leave this question until the UDP support series shows up. I will figure out a better solution :)

In conclusion, it probably won't be used by the UDP type. It's uAPI flag so I consider the compatibility reason.

I don't think this is acceptable. We should aim for an API that can easily be used across protocols, like SO_TIMESTAMPING.

Quoted 6 lines
Quoted 6 lines

After I revisit the UDP SO_TIMESTAMPING again, my thoughts are adjusted like below:

It's hard to provide an absolutely uniform interface or usage to users for TCP and UDP and even more protocols. Cases can be handled one by one.

We should try hard. SO_TIMESTAMPING is uniform across protocols. An interface that is not is just hard to use.

Quoted 6 lines
Quoted 6 lines

The main obstacle is how we can correlate the timestamp in sendmsg syscall with other sending timestamps. It's worth noticing that for SO_TIMESTAMPING the sendmsg timestamp is collected in the userspace. For instance, while skb enters the qdisc, we fail to know which skb belongs to which sendmsg.

Quoted 7 lines
Quoted 7 lines

An idea coming up is to introduce BPF_SOCK_OPS_TS_SND_CB to correlate the sendmsg timestamp with tskey (in tcp_tx_timestamp()) under the protection of socket lock + syscall as the current patch does. But for UDP, it can be lockless. IIUC, there is a very special case where even SO_TIMESTAMPING may get lost: if multiple threads accessing the same socket send UDP packets in parallel, then users could be confused which tskey matches which sendmsg.

Quoted 8 lines

This is a known issue for lockless datagram sockets.

With SO_TIMESTAMPING, but the use of timestamping and of concurrent sendmsg calls is under control of the process, so it only shoots itself in the foot.

With BPF timestamping, a process may confuse a third party admin, so the situation is slightly different.

Agreed.

Quoted 30 lines
Quoted 22 lines
Quoted 4 lines

IIUC, I will not consider this
unlikely case, then the UDP case is quite similar to the TCP case.

Quoted 11 lines

The scenario for the UDP case is:
1) using fentry bpf to hook the udp_sendmsg() to get the timestamp like TCP does in this series.
2) insert BPF_SOCK_OPS_TS_SND_CB into __ip_append_data() near the SO_TIMESTAMPING code snippets to let bpf program correlate the tskey with timestamp.
Note: tskey in UDP will be handled carefully in a different way because we should support both modes for socket timestamping at the same time.
It's really similar to TCP regardless of handling tskey.

To be more precise in case you don't have much time to read the above long paragraph, BPF_SOCK_OPS_TS_SND_CB is mainly used to correlate sendmsg timestamp with corresponding tskey.

1. For TCP, we can correlate it in tcp_tx_timestamp() like this patch does.

Quoted 7 lines

2. For UDP, we can correlate in __ip_append_data() along with those tskey initialization, assuming there are no multiple threads calling locklessly ip_make_skb(). Locked path (udp_sendmsg()->ip_append_data()) works like TCP under the socket lock protection, so it can be easily handled. Lockless path (udp_sendmsg()->ip_make_skb()) can be visited by multiple threads at the same time, which should be handled properly.

Quoted 1 line

Different hook points is fine, as UDP (and RAW) uses __ip_append_data

Then this approach (introducing this new flag) is feasible. Sorry that last night I wrote such a long paragraph which buried something important. Because of that, I rephrase the whole idea about how to let UDP work with this kind of new flag in [patch v8 11/12]. Link is
https://lore.kernel.org/all/CAL+tcoCmXcDot-855XYU7PKCiGvJL=O3CQBGuOTRAs2_=Ys=gg@mail.gmail.com/

Quoted 1 line

or more importantly ip_send_skb, while TCP uses ip_queue_xmit.

For TCP, we use tcp_tx_timestamp to finish the map between sendmsg timestamp and tskey.

Quoted 4 lines

As long as the API is the same: the operation (BPF_SOCK_OPS_TS_SND_CB) and the behavior of that operation. Subject to the usual distinction between protocol behavior (bytestream vs datagram).

I see your point.

Quoted 6 lines
Quoted 4 lines

I prefer to implement
the bpf extension for IPCORK_TS_OPT_ID, which should be another topic, I think. This might be the only one corner case, IIUC?

This sounds like an entirely different topic? Not sure what this is.

Quoted 8 lines

Not really a different topic. I mean let bpf prog take the whole control of setting the tskey, then with this BPF_SOCK_OPS_TS_SND_CB flag we can correlate the sendmsg timestamp with tskey. So It has something to do with the usage of UDP. Please take a look at that link to patch 11/12. For TCP, we don't need to care about the value of tskey which has already been taken care of by SO_TIMESTAMPING. So it is slightly different. I'm not sure if this kind of usage is acceptable?

Why can TCP rely on SO_TIMESTAMPING to set tskey, but UDP cannot?

BPF will need to set the key for both protocol if SO_TIMESTAMPING is not enabled, right?

author
#29 lore raw

Re: [PATCH bpf-next v7 11/13] net-timestamp: add a new callback in tcp_tx_timestamp()

Quoted 197 lines · Willem de Bruijn
Quoted 186 lines · Jason Xing
Quoted 116 lines · Willem de Bruijn
Quoted 90 lines · Jason Xing
Quoted 89 lines · Jason Xing
Quoted 85 lines · Willem de Bruijn
Quoted 73 lines · Jason Xing
Quoted 8 lines · Martin KaFai Lau
Quoted 3 lines · Jason Xing

Introduce the callback to correlate tcp_sendmsg timestamp with other points, like SND/SW/ACK. We can let bpf trace the beginning of tcp_sendmsg_locked() and fetch the socket addr, so that in

Instead of "fetch the socket addr...", should be "store the sendmsg timestamp at the bpf_sk_storage ...".

I will revise it. Thanks.

Quoted 19 lines
Quoted 6 lines

tcp_tx_timestamp() we can correlate the tskey with the socket addr.

It is accurate since they are under the protect of socket lock. More details can be found in the selftest.

The selftest uses the bpf_sk_storage to store the sendmsg timestamp at fentry/tcp_sendmsg_locked and retrieves it back at tcp_tx_timestamp (i.e. BPF_SOCK_OPS_TS_SND_CB added in this patch).

Quoted 8 lines
---
  include/uapi/linux/bpf.h       | 7 +++++++
  net/ipv4/tcp.c                 | 1 +
  tools/include/uapi/linux/bpf.h | 7 +++++++
  3 files changed, 15 insertions(+)
Quoted 16 lines
Quoted 5 lines
include/uapi/linux/bpf.h +0 -0
index 800122a8abe5..accb3b314fff 100644

include/uapi/linux/bpf.h enum { · line 7060

Quoted 9 lines
include/uapi/linux/bpf.h +6 -0
@@ -7052,6 +7052,13 @@ enum {
                                       * when SK_BPF_CB_TX_TIMESTAMPING
                                       * feature is on.
                                       */
+     BPF_SOCK_OPS_TS_SND_CB,         /* Called when every sendmsg syscall
+                                      * is triggered. For TCP, it stays
+                                      * in the last send process to
+                                      * correlate with tcp_sendmsg timestamp
+                                      * with other timestamping callbacks,
+                                      * like SND/SW/ACK.

Do you have a chance to look at how this will work at UDP?

Sure, I feel like it could not be useful for UDP. Well, things get strange because I did write a long paragraph about this thing which apparently disappeared...

I manage to find what I wrote:
For UDP type, BPF_SOCK_OPS_TS_SND_CB may be not suitable because there are two sending process, 1) lockless path, 2) lock path, which should be handled carefully later. For the former, even though it's unlikely multiple threads access the socket to call sendmsg at the same time, I think we'd better not correlate it like what we do to the TCP case because of the lack of sock lock protection. Considering SND_CB is uapi flag, I think we don't need to forcely add the 'TCP_' prefix in case we need to use it someday.

And one more thing is I'd like to use the v5[1] method in the next round to introduce a new tskey_bpf which is good for UDP type. The new field will not conflict with the tskey in shared info which is generated by sk->sk_tskey in __ip_append_data(). It hardly works if both features (so_timestamping and its bpf extension) exists at the same time. Users could get confused because sometimes they fetch the tskey from skb, sometimes they don't, especially when we have cmsg feature to turn it on/ off per sendmsg. A standalone tskey for bpf extension will be needed. With this tskey_bpf, we can easily correlate the timestamp in sendmsg syscall with other tx points(SND/SW/ACK...).

Quoted 7 lines

[1]: https://lore.kernel.org/all/[email protected]/

If possible, we can leave this question until the UDP support series shows up. I will figure out a better solution :)

In conclusion, it probably won't be used by the UDP type. It's uAPI flag so I consider the compatibility reason.

I don't think this is acceptable. We should aim for an API that can easily be used across protocols, like SO_TIMESTAMPING.

Quoted 6 lines
Quoted 6 lines

After I revisit the UDP SO_TIMESTAMPING again, my thoughts are adjusted like below:

It's hard to provide an absolutely uniform interface or usage to users for TCP and UDP and even more protocols. Cases can be handled one by one.

We should try hard. SO_TIMESTAMPING is uniform across protocols. An interface that is not is just hard to use.

Quoted 6 lines
Quoted 6 lines

The main obstacle is how we can correlate the timestamp in sendmsg syscall with other sending timestamps. It's worth noticing that for SO_TIMESTAMPING the sendmsg timestamp is collected in the userspace. For instance, while skb enters the qdisc, we fail to know which skb belongs to which sendmsg.

Quoted 7 lines
Quoted 7 lines

An idea coming up is to introduce BPF_SOCK_OPS_TS_SND_CB to correlate the sendmsg timestamp with tskey (in tcp_tx_timestamp()) under the protection of socket lock + syscall as the current patch does. But for UDP, it can be lockless. IIUC, there is a very special case where even SO_TIMESTAMPING may get lost: if multiple threads accessing the same socket send UDP packets in parallel, then users could be confused which tskey matches which sendmsg.

Quoted 8 lines

This is a known issue for lockless datagram sockets.

With SO_TIMESTAMPING, but the use of timestamping and of concurrent sendmsg calls is under control of the process, so it only shoots itself in the foot.

With BPF timestamping, a process may confuse a third party admin, so the situation is slightly different.

Agreed.

Quoted 30 lines
Quoted 22 lines
Quoted 4 lines

IIUC, I will not consider this
unlikely case, then the UDP case is quite similar to the TCP case.

Quoted 11 lines

The scenario for the UDP case is:
1) using fentry bpf to hook the udp_sendmsg() to get the timestamp like TCP does in this series.
2) insert BPF_SOCK_OPS_TS_SND_CB into __ip_append_data() near the SO_TIMESTAMPING code snippets to let bpf program correlate the tskey with timestamp.
Note: tskey in UDP will be handled carefully in a different way because we should support both modes for socket timestamping at the same time.
It's really similar to TCP regardless of handling tskey.

To be more precise in case you don't have much time to read the above long paragraph, BPF_SOCK_OPS_TS_SND_CB is mainly used to correlate sendmsg timestamp with corresponding tskey.

1. For TCP, we can correlate it in tcp_tx_timestamp() like this patch does.

Quoted 7 lines

2. For UDP, we can correlate in __ip_append_data() along with those tskey initialization, assuming there are no multiple threads calling locklessly ip_make_skb(). Locked path (udp_sendmsg()->ip_append_data()) works like TCP under the socket lock protection, so it can be easily handled. Lockless path (udp_sendmsg()->ip_make_skb()) can be visited by multiple threads at the same time, which should be handled properly.

Quoted 1 line

Different hook points is fine, as UDP (and RAW) uses __ip_append_data

Then this approach (introducing this new flag) is feasible. Sorry that last night I wrote such a long paragraph which buried something important. Because of that, I rephrase the whole idea about how to let UDP work with this kind of new flag in [patch v8 11/12]. Link is
https://lore.kernel.org/all/CAL+tcoCmXcDot-855XYU7PKCiGvJL=O3CQBGuOTRAs2_=Ys=gg@mail.gmail.com/

Quoted 1 line

or more importantly ip_send_skb, while TCP uses ip_queue_xmit.

For TCP, we use tcp_tx_timestamp to finish the map between sendmsg timestamp and tskey.

Quoted 4 lines

As long as the API is the same: the operation (BPF_SOCK_OPS_TS_SND_CB) and the behavior of that operation. Subject to the usual distinction between protocol behavior (bytestream vs datagram).

I see your point.

Quoted 6 lines
Quoted 4 lines

I prefer to implement
the bpf extension for IPCORK_TS_OPT_ID, which should be another topic, I think. This might be the only one corner case, IIUC?

This sounds like an entirely different topic? Not sure what this is.

Quoted 8 lines

Not really a different topic. I mean let bpf prog take the whole control of setting the tskey, then with this BPF_SOCK_OPS_TS_SND_CB flag we can correlate the sendmsg timestamp with tskey. So It has something to do with the usage of UDP. Please take a look at that link to patch 11/12. For TCP, we don't need to care about the value of tskey which has already been taken care of by SO_TIMESTAMPING. So it is slightly different. I'm not sure if this kind of usage is acceptable?

Quoted 1 line

Why can TCP rely on SO_TIMESTAMPING to set tskey, but UDP cannot?

Because for TCP the shared info tskey is calculated by seqno (in tcp_tx_timestamp()), so it works for so_timestamping and its bpf extension and they are the same. However, for UDP, the shared info tskey can be different, depending on when to call __ip_append_data() and what the sk->sk_tskey is. It can cause conflicts when two modes work at the same time. More than that, lockless UDP case is a tough one since we cannot correlate the sendmsg timestamp in udp_sendmsg() with the tskey generated in __ip_append_data(), which is a long gap without any lock. So reuse the IPCORK_TS_OPT_ID logic for bpf extension here can work.

It's worth to highlight that 1) for TCP the time to generate a sendmsg timestamp and generate a tskey are under the same lock, 2) for non-lockless UDP, it works like TCP. But in order to deal with the lockless part, I choose to implement the IPCORK_TS_OPT_ID idea. This is somehow another topic, but the relevant part is that I tried to prove BPF_SOCK_OPS_TS_SND_CB can be also used in UDP (but in a different position compared to TCP protocol).

Quoted 3 lines

BPF will need to set the key for both protocol if SO_TIMESTAMPING is not enabled, right?

For TCP, we will rely on 'shinfo->tskey = TCP_SKB_CB(skb)->seq + skb->len - 1;' in tcp_tx_timestamp() regardless of the status of SO_TIMESTAMPING, so don't bother to manage the tskey from bpf prog. While for UDP, we have to manage the tskey. Of course, it's another topic.

Thanks,
Jason

#30 lore raw

Re: [PATCH bpf-next v7 11/13] net-timestamp: add a new callback in tcp_tx_timestamp()

Quoted 199 lines · Jason Xing
Quoted 197 lines · Willem de Bruijn
Quoted 194 lines · Jason Xing
Quoted 116 lines · Willem de Bruijn
Quoted 90 lines · Jason Xing
Quoted 89 lines · Jason Xing
Quoted 85 lines · Willem de Bruijn
Quoted 73 lines · Jason Xing
Quoted 8 lines · Martin KaFai Lau
Quoted 3 lines · Jason Xing

Introduce the callback to correlate tcp_sendmsg timestamp with other points, like SND/SW/ACK. We can let bpf trace the beginning of tcp_sendmsg_locked() and fetch the socket addr, so that in

Instead of "fetch the socket addr...", should be "store the sendmsg timestamp at the bpf_sk_storage ...".

I will revise it. Thanks.

Quoted 19 lines
Quoted 6 lines

tcp_tx_timestamp() we can correlate the tskey with the socket addr.

It is accurate since they are under the protect of socket lock. More details can be found in the selftest.

The selftest uses the bpf_sk_storage to store the sendmsg timestamp at fentry/tcp_sendmsg_locked and retrieves it back at tcp_tx_timestamp (i.e. BPF_SOCK_OPS_TS_SND_CB added in this patch).

Quoted 8 lines
---
  include/uapi/linux/bpf.h       | 7 +++++++
  net/ipv4/tcp.c                 | 1 +
  tools/include/uapi/linux/bpf.h | 7 +++++++
  3 files changed, 15 insertions(+)
Quoted 16 lines
Quoted 5 lines
include/uapi/linux/bpf.h +0 -0
index 800122a8abe5..accb3b314fff 100644

include/uapi/linux/bpf.h enum { · line 7060

Quoted 9 lines
include/uapi/linux/bpf.h +6 -0
@@ -7052,6 +7052,13 @@ enum {
                                       * when SK_BPF_CB_TX_TIMESTAMPING
                                       * feature is on.
                                       */
+     BPF_SOCK_OPS_TS_SND_CB,         /* Called when every sendmsg syscall
+                                      * is triggered. For TCP, it stays
+                                      * in the last send process to
+                                      * correlate with tcp_sendmsg timestamp
+                                      * with other timestamping callbacks,
+                                      * like SND/SW/ACK.

Do you have a chance to look at how this will work at UDP?

Sure, I feel like it could not be useful for UDP. Well, things get strange because I did write a long paragraph about this thing which apparently disappeared...

I manage to find what I wrote:
For UDP type, BPF_SOCK_OPS_TS_SND_CB may be not suitable because there are two sending process, 1) lockless path, 2) lock path, which should be handled carefully later. For the former, even though it's unlikely multiple threads access the socket to call sendmsg at the same time, I think we'd better not correlate it like what we do to the TCP case because of the lack of sock lock protection. Considering SND_CB is uapi flag, I think we don't need to forcely add the 'TCP_' prefix in case we need to use it someday.

And one more thing is I'd like to use the v5[1] method in the next round to introduce a new tskey_bpf which is good for UDP type. The new field will not conflict with the tskey in shared info which is generated by sk->sk_tskey in __ip_append_data(). It hardly works if both features (so_timestamping and its bpf extension) exists at the same time. Users could get confused because sometimes they fetch the tskey from skb, sometimes they don't, especially when we have cmsg feature to turn it on/ off per sendmsg. A standalone tskey for bpf extension will be needed. With this tskey_bpf, we can easily correlate the timestamp in sendmsg syscall with other tx points(SND/SW/ACK...).

Quoted 7 lines

[1]: https://lore.kernel.org/all/[email protected]/

If possible, we can leave this question until the UDP support series shows up. I will figure out a better solution :)

In conclusion, it probably won't be used by the UDP type. It's uAPI flag so I consider the compatibility reason.

I don't think this is acceptable. We should aim for an API that can easily be used across protocols, like SO_TIMESTAMPING.

Quoted 6 lines
Quoted 6 lines

After I revisit the UDP SO_TIMESTAMPING again, my thoughts are adjusted like below:

It's hard to provide an absolutely uniform interface or usage to users for TCP and UDP and even more protocols. Cases can be handled one by one.

We should try hard. SO_TIMESTAMPING is uniform across protocols. An interface that is not is just hard to use.

Quoted 6 lines
Quoted 6 lines

The main obstacle is how we can correlate the timestamp in sendmsg syscall with other sending timestamps. It's worth noticing that for SO_TIMESTAMPING the sendmsg timestamp is collected in the userspace. For instance, while skb enters the qdisc, we fail to know which skb belongs to which sendmsg.

Quoted 7 lines
Quoted 7 lines

An idea coming up is to introduce BPF_SOCK_OPS_TS_SND_CB to correlate the sendmsg timestamp with tskey (in tcp_tx_timestamp()) under the protection of socket lock + syscall as the current patch does. But for UDP, it can be lockless. IIUC, there is a very special case where even SO_TIMESTAMPING may get lost: if multiple threads accessing the same socket send UDP packets in parallel, then users could be confused which tskey matches which sendmsg.

Quoted 8 lines

This is a known issue for lockless datagram sockets.

With SO_TIMESTAMPING, but the use of timestamping and of concurrent sendmsg calls is under control of the process, so it only shoots itself in the foot.

With BPF timestamping, a process may confuse a third party admin, so the situation is slightly different.

Agreed.

Quoted 30 lines
Quoted 22 lines
Quoted 4 lines

IIUC, I will not consider this
unlikely case, then the UDP case is quite similar to the TCP case.

Quoted 11 lines

The scenario for the UDP case is:
1) using fentry bpf to hook the udp_sendmsg() to get the timestamp like TCP does in this series.
2) insert BPF_SOCK_OPS_TS_SND_CB into __ip_append_data() near the SO_TIMESTAMPING code snippets to let bpf program correlate the tskey with timestamp.
Note: tskey in UDP will be handled carefully in a different way because we should support both modes for socket timestamping at the same time.
It's really similar to TCP regardless of handling tskey.

To be more precise in case you don't have much time to read the above long paragraph, BPF_SOCK_OPS_TS_SND_CB is mainly used to correlate sendmsg timestamp with corresponding tskey.

1. For TCP, we can correlate it in tcp_tx_timestamp() like this patch does.

Quoted 7 lines

2. For UDP, we can correlate in __ip_append_data() along with those tskey initialization, assuming there are no multiple threads calling locklessly ip_make_skb(). Locked path (udp_sendmsg()->ip_append_data()) works like TCP under the socket lock protection, so it can be easily handled. Lockless path (udp_sendmsg()->ip_make_skb()) can be visited by multiple threads at the same time, which should be handled properly.

Quoted 1 line

Different hook points is fine, as UDP (and RAW) uses __ip_append_data

Then this approach (introducing this new flag) is feasible. Sorry that last night I wrote such a long paragraph which buried something important. Because of that, I rephrase the whole idea about how to let UDP work with this kind of new flag in [patch v8 11/12]. Link is
https://lore.kernel.org/all/CAL+tcoCmXcDot-855XYU7PKCiGvJL=O3CQBGuOTRAs2_=Ys=gg@mail.gmail.com/

Quoted 1 line

or more importantly ip_send_skb, while TCP uses ip_queue_xmit.

For TCP, we use tcp_tx_timestamp to finish the map between sendmsg timestamp and tskey.

Quoted 4 lines

As long as the API is the same: the operation (BPF_SOCK_OPS_TS_SND_CB) and the behavior of that operation. Subject to the usual distinction between protocol behavior (bytestream vs datagram).

I see your point.

Quoted 6 lines
Quoted 4 lines

I prefer to implement
the bpf extension for IPCORK_TS_OPT_ID, which should be another topic, I think. This might be the only one corner case, IIUC?

This sounds like an entirely different topic? Not sure what this is.

Not really a different topic. I mean let bpf prog take the whole control of setting the tskey, then with this BPF_SOCK_OPS_TS_SND_CB flag we can correlate the sendmsg timestamp with tskey. So It has something to do with the usage of UDP. Please take a look at that link to patch 11/12. For TCP, we don't need to care about the value of tskey which has already been taken care of by SO_TIMESTAMPING. So it is slightly different. I'm not sure if this kind of usage is acceptable?

Quoted 8 lines
Quoted 1 line

Why can TCP rely on SO_TIMESTAMPING to set tskey, but UDP cannot?

Because for TCP the shared info tskey is calculated by seqno (in tcp_tx_timestamp()), so it works for so_timestamping and its bpf extension and they are the same. However, for UDP, the shared info tskey can be different, depending on when to call __ip_append_data() and what the sk->sk_tskey is. It can cause conflicts when two modes work at the same time.

lockless and locked cannot conflict. (if up->pending then the only option is to append to that.)

Quoted 3 lines

More than that, lockless UDP case is a tough
one since we cannot correlate the sendmsg timestamp in udp_sendmsg() with the tskey generated in __ip_append_data(),

With SO_TIMESTAMPING we do not have this distinction between TCP and UDP, so we don't need it here.

It is true that multiple lockless sendmsg calls can race and in that case that correlation is ambiguous. That is also the case for SO_TIMESTAMPING and a known issue.

This is unlikely in most workloads in practice.

Quoted 3 lines

which is a long gap
without any lock. So reuse the IPCORK_TS_OPT_ID logic for bpf extension here can work.

It is fine to add a solution to work around the ambiguity. But not to make it a precondition and so diverge the API for TCP and UDP.

The same argument to choose a key from BPF can be made for TCP to a certain extent.

Quoted 20 lines

It's worth to highlight that 1) for TCP the time to generate a sendmsg timestamp and generate a tskey are under the same lock, 2) for non-lockless UDP, it works like TCP. But in order to deal with the lockless part, I choose to implement the IPCORK_TS_OPT_ID idea. This is somehow another topic, but the relevant part is that I tried to prove BPF_SOCK_OPS_TS_SND_CB can be also used in UDP (but in a different position compared to TCP protocol).

Quoted 3 lines

BPF will need to set the key for both protocol if SO_TIMESTAMPING is not enabled, right?

For TCP, we will rely on 'shinfo->tskey = TCP_SKB_CB(skb)->seq + skb->len - 1;' in tcp_tx_timestamp() regardless of the status of SO_TIMESTAMPING, so don't bother to manage the tskey from bpf prog. While for UDP, we have to manage the tskey. Of course, it's another topic.

Thanks,
Jason

author
#31 lore raw

Re: [PATCH bpf-next v7 11/13] net-timestamp: add a new callback in tcp_tx_timestamp()

Quoted 210 lines · Willem de Bruijn
Quoted 199 lines · Jason Xing
Quoted 197 lines · Willem de Bruijn
Quoted 194 lines · Jason Xing
Quoted 116 lines · Willem de Bruijn
Quoted 90 lines · Jason Xing
Quoted 89 lines · Jason Xing
Quoted 85 lines · Willem de Bruijn
Quoted 73 lines · Jason Xing
Quoted 8 lines · Martin KaFai Lau
Quoted 3 lines · Jason Xing

Introduce the callback to correlate tcp_sendmsg timestamp with other points, like SND/SW/ACK. We can let bpf trace the beginning of tcp_sendmsg_locked() and fetch the socket addr, so that in

Instead of "fetch the socket addr...", should be "store the sendmsg timestamp at the bpf_sk_storage ...".

I will revise it. Thanks.

Quoted 19 lines
Quoted 6 lines

tcp_tx_timestamp() we can correlate the tskey with the socket addr.

It is accurate since they are under the protect of socket lock. More details can be found in the selftest.

The selftest uses the bpf_sk_storage to store the sendmsg timestamp at fentry/tcp_sendmsg_locked and retrieves it back at tcp_tx_timestamp (i.e. BPF_SOCK_OPS_TS_SND_CB added in this patch).

Quoted 8 lines
---
  include/uapi/linux/bpf.h       | 7 +++++++
  net/ipv4/tcp.c                 | 1 +
  tools/include/uapi/linux/bpf.h | 7 +++++++
  3 files changed, 15 insertions(+)
Quoted 16 lines
Quoted 5 lines
include/uapi/linux/bpf.h +0 -0
index 800122a8abe5..accb3b314fff 100644

include/uapi/linux/bpf.h enum { · line 7060

Quoted 9 lines
include/uapi/linux/bpf.h +6 -0
@@ -7052,6 +7052,13 @@ enum {
                                       * when SK_BPF_CB_TX_TIMESTAMPING
                                       * feature is on.
                                       */
+     BPF_SOCK_OPS_TS_SND_CB,         /* Called when every sendmsg syscall
+                                      * is triggered. For TCP, it stays
+                                      * in the last send process to
+                                      * correlate with tcp_sendmsg timestamp
+                                      * with other timestamping callbacks,
+                                      * like SND/SW/ACK.

Do you have a chance to look at how this will work at UDP?

Sure, I feel like it could not be useful for UDP. Well, things get strange because I did write a long paragraph about this thing which apparently disappeared...

I manage to find what I wrote:
For UDP type, BPF_SOCK_OPS_TS_SND_CB may be not suitable because there are two sending process, 1) lockless path, 2) lock path, which should be handled carefully later. For the former, even though it's unlikely multiple threads access the socket to call sendmsg at the same time, I think we'd better not correlate it like what we do to the TCP case because of the lack of sock lock protection. Considering SND_CB is uapi flag, I think we don't need to forcely add the 'TCP_' prefix in case we need to use it someday.

And one more thing is I'd like to use the v5[1] method in the next round to introduce a new tskey_bpf which is good for UDP type. The new field will not conflict with the tskey in shared info which is generated by sk->sk_tskey in __ip_append_data(). It hardly works if both features (so_timestamping and its bpf extension) exists at the same time. Users could get confused because sometimes they fetch the tskey from skb, sometimes they don't, especially when we have cmsg feature to turn it on/ off per sendmsg. A standalone tskey for bpf extension will be needed. With this tskey_bpf, we can easily correlate the timestamp in sendmsg syscall with other tx points(SND/SW/ACK...).

Quoted 7 lines

[1]: https://lore.kernel.org/all/[email protected]/

If possible, we can leave this question until the UDP support series shows up. I will figure out a better solution :)

In conclusion, it probably won't be used by the UDP type. It's uAPI flag so I consider the compatibility reason.

I don't think this is acceptable. We should aim for an API that can easily be used across protocols, like SO_TIMESTAMPING.

Quoted 6 lines
Quoted 6 lines

After I revisit the UDP SO_TIMESTAMPING again, my thoughts are adjusted like below:

It's hard to provide an absolutely uniform interface or usage to users for TCP and UDP and even more protocols. Cases can be handled one by one.

We should try hard. SO_TIMESTAMPING is uniform across protocols. An interface that is not is just hard to use.

Quoted 6 lines
Quoted 6 lines

The main obstacle is how we can correlate the timestamp in sendmsg syscall with other sending timestamps. It's worth noticing that for SO_TIMESTAMPING the sendmsg timestamp is collected in the userspace. For instance, while skb enters the qdisc, we fail to know which skb belongs to which sendmsg.

Quoted 7 lines
Quoted 7 lines

An idea coming up is to introduce BPF_SOCK_OPS_TS_SND_CB to correlate the sendmsg timestamp with tskey (in tcp_tx_timestamp()) under the protection of socket lock + syscall as the current patch does. But for UDP, it can be lockless. IIUC, there is a very special case where even SO_TIMESTAMPING may get lost: if multiple threads accessing the same socket send UDP packets in parallel, then users could be confused which tskey matches which sendmsg.

Quoted 8 lines

This is a known issue for lockless datagram sockets.

With SO_TIMESTAMPING, but the use of timestamping and of concurrent sendmsg calls is under control of the process, so it only shoots itself in the foot.

With BPF timestamping, a process may confuse a third party admin, so the situation is slightly different.

Agreed.

Quoted 30 lines
Quoted 22 lines
Quoted 4 lines

IIUC, I will not consider this
unlikely case, then the UDP case is quite similar to the TCP case.

Quoted 11 lines

The scenario for the UDP case is:
1) using fentry bpf to hook the udp_sendmsg() to get the timestamp like TCP does in this series.
2) insert BPF_SOCK_OPS_TS_SND_CB into __ip_append_data() near the SO_TIMESTAMPING code snippets to let bpf program correlate the tskey with timestamp.
Note: tskey in UDP will be handled carefully in a different way because we should support both modes for socket timestamping at the same time.
It's really similar to TCP regardless of handling tskey.

To be more precise in case you don't have much time to read the above long paragraph, BPF_SOCK_OPS_TS_SND_CB is mainly used to correlate sendmsg timestamp with corresponding tskey.

1. For TCP, we can correlate it in tcp_tx_timestamp() like this patch does.

Quoted 7 lines

2. For UDP, we can correlate in __ip_append_data() along with those tskey initialization, assuming there are no multiple threads calling locklessly ip_make_skb(). Locked path (udp_sendmsg()->ip_append_data()) works like TCP under the socket lock protection, so it can be easily handled. Lockless path (udp_sendmsg()->ip_make_skb()) can be visited by multiple threads at the same time, which should be handled properly.

Quoted 1 line

Different hook points is fine, as UDP (and RAW) uses __ip_append_data

Then this approach (introducing this new flag) is feasible. Sorry that last night I wrote such a long paragraph which buried something important. Because of that, I rephrase the whole idea about how to let UDP work with this kind of new flag in [patch v8 11/12]. Link is
https://lore.kernel.org/all/CAL+tcoCmXcDot-855XYU7PKCiGvJL=O3CQBGuOTRAs2_=Ys=gg@mail.gmail.com/

Quoted 1 line

or more importantly ip_send_skb, while TCP uses ip_queue_xmit.

For TCP, we use tcp_tx_timestamp to finish the map between sendmsg timestamp and tskey.

Quoted 4 lines

As long as the API is the same: the operation (BPF_SOCK_OPS_TS_SND_CB) and the behavior of that operation. Subject to the usual distinction between protocol behavior (bytestream vs datagram).

I see your point.

Quoted 6 lines
Quoted 4 lines

I prefer to implement
the bpf extension for IPCORK_TS_OPT_ID, which should be another topic, I think. This might be the only one corner case, IIUC?

This sounds like an entirely different topic? Not sure what this is.

Not really a different topic. I mean let bpf prog take the whole control of setting the tskey, then with this BPF_SOCK_OPS_TS_SND_CB flag we can correlate the sendmsg timestamp with tskey. So It has something to do with the usage of UDP. Please take a look at that link to patch 11/12. For TCP, we don't need to care about the value of tskey which has already been taken care of by SO_TIMESTAMPING. So it is slightly different. I'm not sure if this kind of usage is acceptable?

Quoted 8 lines
Quoted 1 line

Why can TCP rely on SO_TIMESTAMPING to set tskey, but UDP cannot?

Because for TCP the shared info tskey is calculated by seqno (in tcp_tx_timestamp()), so it works for so_timestamping and its bpf extension and they are the same. However, for UDP, the shared info tskey can be different, depending on when to call __ip_append_data() and what the sk->sk_tskey is. It can cause conflicts when two modes work at the same time.

Quoted 2 lines

lockless and locked cannot conflict. (if up->pending then the only option is to append to that.)

Sorry, I should have described more about this point. I was trying to say that if two modes (bpf extension and application timestamping) work at the same time, will the tskey get messed up? Because we have to check if the application mode is turned on. If on, we fetch the existing key generated by the application mode, or else we generate one by modifying the sk->sk_tskey or the tskey of skb?

Quoted 5 lines
Quoted 4 lines

More than that, lockless UDP case is a tough
one since we cannot correlate the sendmsg timestamp in udp_sendmsg() with the tskey generated in __ip_append_data(),

Quoted 8 lines

With SO_TIMESTAMPING we do not have this distinction between TCP and UDP, so we don't need it here.

It is true that multiple lockless sendmsg calls can race and in that case that correlation is ambiguous. That is also the case for SO_TIMESTAMPING and a known issue.

This is unlikely in most workloads in practice.

Oh, I see.

Quoted 6 lines
Quoted 3 lines

which is a long gap
without any lock. So reuse the IPCORK_TS_OPT_ID logic for bpf extension here can work.

It is fine to add a solution to work around the ambiguity. But not to make it a precondition and so diverge the API for TCP and UDP.

There is no divergence in API. BPF always uses SND flag to finish the correlation. The only difference is UDP needs to manage the tskey pool (allocating which tskey to which sendmsg).

Quoted 3 lines

The same argument to choose a key from BPF can be made for TCP to a certain extent.

Well...right, but we don't bother to do this for TCP. The TCP case is simpler.

It seems that you object to the idea that let bpf prog control the allocating tskey for UDP _as default_.

I'm not sure if I follow you. Sorry for repeating in case that I miss something:
1) For UDP, do not allocate the tskey from the bpf program, unless it's an alternative/workaround to handle the lockless case. So it's a backup choice.
2) Use the exact same way like TCP to finish the correlation on the basis of socket lock protection _as default_. Because the lockless method is seldomly used then we may provide 1) method?
?

Thanks,
Jason

Quoted 23 lines
Quoted 21 lines

It's worth to highlight that 1) for TCP the time to generate a sendmsg timestamp and generate a tskey are under the same lock, 2) for non-lockless UDP, it works like TCP. But in order to deal with the lockless part, I choose to implement the IPCORK_TS_OPT_ID idea. This is somehow another topic, but the relevant part is that I tried to prove BPF_SOCK_OPS_TS_SND_CB can be also used in UDP (but in a different position compared to TCP protocol).

Quoted 3 lines

BPF will need to set the key for both protocol if SO_TIMESTAMPING is not enabled, right?

For TCP, we will rely on 'shinfo->tskey = TCP_SKB_CB(skb)->seq + skb->len - 1;' in tcp_tx_timestamp() regardless of the status of SO_TIMESTAMPING, so don't bother to manage the tskey from bpf prog. While for UDP, we have to manage the tskey. Of course, it's another topic.

Thanks,
Jason

#32 lore raw

Re: [PATCH bpf-next v7 11/13] net-timestamp: add a new callback in tcp_tx_timestamp()

Quoted 215 lines · Jason Xing
Quoted 210 lines · Willem de Bruijn
Quoted 199 lines · Jason Xing
Quoted 197 lines · Willem de Bruijn
Quoted 194 lines · Jason Xing
Quoted 116 lines · Willem de Bruijn
Quoted 90 lines · Jason Xing
Quoted 89 lines · Jason Xing
Quoted 85 lines · Willem de Bruijn
Quoted 73 lines · Jason Xing
Quoted 8 lines · Martin KaFai Lau
Quoted 3 lines · Jason Xing

Introduce the callback to correlate tcp_sendmsg timestamp with other points, like SND/SW/ACK. We can let bpf trace the beginning of tcp_sendmsg_locked() and fetch the socket addr, so that in

Instead of "fetch the socket addr...", should be "store the sendmsg timestamp at the bpf_sk_storage ...".

I will revise it. Thanks.

Quoted 19 lines
Quoted 6 lines

tcp_tx_timestamp() we can correlate the tskey with the socket addr.

It is accurate since they are under the protect of socket lock. More details can be found in the selftest.

The selftest uses the bpf_sk_storage to store the sendmsg timestamp at fentry/tcp_sendmsg_locked and retrieves it back at tcp_tx_timestamp (i.e. BPF_SOCK_OPS_TS_SND_CB added in this patch).

Quoted 8 lines
---
  include/uapi/linux/bpf.h       | 7 +++++++
  net/ipv4/tcp.c                 | 1 +
  tools/include/uapi/linux/bpf.h | 7 +++++++
  3 files changed, 15 insertions(+)
Quoted 16 lines
Quoted 5 lines
include/uapi/linux/bpf.h +0 -0
index 800122a8abe5..accb3b314fff 100644

include/uapi/linux/bpf.h enum { · line 7060

Quoted 9 lines
include/uapi/linux/bpf.h +6 -0
@@ -7052,6 +7052,13 @@ enum {
                                       * when SK_BPF_CB_TX_TIMESTAMPING
                                       * feature is on.
                                       */
+     BPF_SOCK_OPS_TS_SND_CB,         /* Called when every sendmsg syscall
+                                      * is triggered. For TCP, it stays
+                                      * in the last send process to
+                                      * correlate with tcp_sendmsg timestamp
+                                      * with other timestamping callbacks,
+                                      * like SND/SW/ACK.

Do you have a chance to look at how this will work at UDP?

Sure, I feel like it could not be useful for UDP. Well, things get strange because I did write a long paragraph about this thing which apparently disappeared...

I manage to find what I wrote:
For UDP type, BPF_SOCK_OPS_TS_SND_CB may be not suitable because there are two sending process, 1) lockless path, 2) lock path, which should be handled carefully later. For the former, even though it's unlikely multiple threads access the socket to call sendmsg at the same time, I think we'd better not correlate it like what we do to the TCP case because of the lack of sock lock protection. Considering SND_CB is uapi flag, I think we don't need to forcely add the 'TCP_' prefix in case we need to use it someday.

And one more thing is I'd like to use the v5[1] method in the next round to introduce a new tskey_bpf which is good for UDP type. The new field will not conflict with the tskey in shared info which is generated by sk->sk_tskey in __ip_append_data(). It hardly works if both features (so_timestamping and its bpf extension) exists at the same time. Users could get confused because sometimes they fetch the tskey from skb, sometimes they don't, especially when we have cmsg feature to turn it on/ off per sendmsg. A standalone tskey for bpf extension will be needed. With this tskey_bpf, we can easily correlate the timestamp in sendmsg syscall with other tx points(SND/SW/ACK...).

Quoted 7 lines

[1]: https://lore.kernel.org/all/[email protected]/

If possible, we can leave this question until the UDP support series shows up. I will figure out a better solution :)

In conclusion, it probably won't be used by the UDP type. It's uAPI flag so I consider the compatibility reason.

I don't think this is acceptable. We should aim for an API that can easily be used across protocols, like SO_TIMESTAMPING.

Quoted 6 lines
Quoted 6 lines

After I revisit the UDP SO_TIMESTAMPING again, my thoughts are adjusted like below:

It's hard to provide an absolutely uniform interface or usage to users for TCP and UDP and even more protocols. Cases can be handled one by one.

We should try hard. SO_TIMESTAMPING is uniform across protocols. An interface that is not is just hard to use.

Quoted 6 lines
Quoted 6 lines

The main obstacle is how we can correlate the timestamp in sendmsg syscall with other sending timestamps. It's worth noticing that for SO_TIMESTAMPING the sendmsg timestamp is collected in the userspace. For instance, while skb enters the qdisc, we fail to know which skb belongs to which sendmsg.

Quoted 7 lines
Quoted 7 lines

An idea coming up is to introduce BPF_SOCK_OPS_TS_SND_CB to correlate the sendmsg timestamp with tskey (in tcp_tx_timestamp()) under the protection of socket lock + syscall as the current patch does. But for UDP, it can be lockless. IIUC, there is a very special case where even SO_TIMESTAMPING may get lost: if multiple threads accessing the same socket send UDP packets in parallel, then users could be confused which tskey matches which sendmsg.

Quoted 8 lines

This is a known issue for lockless datagram sockets.

With SO_TIMESTAMPING, but the use of timestamping and of concurrent sendmsg calls is under control of the process, so it only shoots itself in the foot.

With BPF timestamping, a process may confuse a third party admin, so the situation is slightly different.

Agreed.

Quoted 30 lines
Quoted 22 lines
Quoted 4 lines

IIUC, I will not consider this
unlikely case, then the UDP case is quite similar to the TCP case.

Quoted 11 lines

The scenario for the UDP case is:
1) using fentry bpf to hook the udp_sendmsg() to get the timestamp like TCP does in this series.
2) insert BPF_SOCK_OPS_TS_SND_CB into __ip_append_data() near the SO_TIMESTAMPING code snippets to let bpf program correlate the tskey with timestamp.
Note: tskey in UDP will be handled carefully in a different way because we should support both modes for socket timestamping at the same time.
It's really similar to TCP regardless of handling tskey.

To be more precise in case you don't have much time to read the above long paragraph, BPF_SOCK_OPS_TS_SND_CB is mainly used to correlate sendmsg timestamp with corresponding tskey.

1. For TCP, we can correlate it in tcp_tx_timestamp() like this patch does.

Quoted 7 lines

2. For UDP, we can correlate in __ip_append_data() along with those tskey initialization, assuming there are no multiple threads calling locklessly ip_make_skb(). Locked path (udp_sendmsg()->ip_append_data()) works like TCP under the socket lock protection, so it can be easily handled. Lockless path (udp_sendmsg()->ip_make_skb()) can be visited by multiple threads at the same time, which should be handled properly.

Quoted 1 line

Different hook points is fine, as UDP (and RAW) uses __ip_append_data

Then this approach (introducing this new flag) is feasible. Sorry that last night I wrote such a long paragraph which buried something important. Because of that, I rephrase the whole idea about how to let UDP work with this kind of new flag in [patch v8 11/12]. Link is
https://lore.kernel.org/all/CAL+tcoCmXcDot-855XYU7PKCiGvJL=O3CQBGuOTRAs2_=Ys=gg@mail.gmail.com/

Quoted 1 line

or more importantly ip_send_skb, while TCP uses ip_queue_xmit.

For TCP, we use tcp_tx_timestamp to finish the map between sendmsg timestamp and tskey.

Quoted 4 lines

As long as the API is the same: the operation (BPF_SOCK_OPS_TS_SND_CB) and the behavior of that operation. Subject to the usual distinction between protocol behavior (bytestream vs datagram).

I see your point.

Quoted 6 lines
Quoted 4 lines

I prefer to implement
the bpf extension for IPCORK_TS_OPT_ID, which should be another topic, I think. This might be the only one corner case, IIUC?

This sounds like an entirely different topic? Not sure what this is.

Not really a different topic. I mean let bpf prog take the whole control of setting the tskey, then with this BPF_SOCK_OPS_TS_SND_CB flag we can correlate the sendmsg timestamp with tskey. So It has something to do with the usage of UDP. Please take a look at that link to patch 11/12. For TCP, we don't need to care about the value of tskey which has already been taken care of by SO_TIMESTAMPING. So it is slightly different. I'm not sure if this kind of usage is acceptable?

Quoted 8 lines
Quoted 1 line

Why can TCP rely on SO_TIMESTAMPING to set tskey, but UDP cannot?

Because for TCP the shared info tskey is calculated by seqno (in tcp_tx_timestamp()), so it works for so_timestamping and its bpf extension and they are the same. However, for UDP, the shared info tskey can be different, depending on when to call __ip_append_data() and what the sk->sk_tskey is. It can cause conflicts when two modes work at the same time.

Quoted 2 lines

lockless and locked cannot conflict. (if up->pending then the only option is to append to that.)

Quoted 6 lines

Sorry, I should have described more about this point. I was trying to say that if two modes (bpf extension and application timestamping) work at the same time, will the tskey get messed up? Because we have to check if the application mode is turned on. If on, we fetch the existing key generated by the application mode, or else we generate one by modifying the sk->sk_tskey or the tskey of skb?

This applies to all protocols that implement both. TCP, UDP, eventually RAW and maybe others like L2TP, CAN, MPTCP, ..

Which is why it should be handled the same uniformly.

Quoted 20 lines
Quoted 5 lines
Quoted 4 lines

More than that, lockless UDP case is a tough
one since we cannot correlate the sendmsg timestamp in udp_sendmsg() with the tskey generated in __ip_append_data(),

Quoted 8 lines

With SO_TIMESTAMPING we do not have this distinction between TCP and UDP, so we don't need it here.

It is true that multiple lockless sendmsg calls can race and in that case that correlation is ambiguous. That is also the case for SO_TIMESTAMPING and a known issue.

This is unlikely in most workloads in practice.

Oh, I see.

Quoted 4 lines
Quoted 3 lines

which is a long gap
without any lock. So reuse the IPCORK_TS_OPT_ID logic for bpf extension here can work.

Quoted 6 lines
Quoted 2 lines

It is fine to add a solution to work around the ambiguity. But not to make it a precondition and so diverge the API for TCP and UDP.

There is no divergence in API. BPF always uses SND flag to finish the correlation. The only difference is UDP needs to manage the tskey pool (allocating which tskey to which sendmsg).

Again, I don't see how this is UDP specific.

Quoted 9 lines
Quoted 4 lines

The same argument to choose a key from BPF can be made for TCP to a certain extent.

Well...right, but we don't bother to do this for TCP. The TCP case is simpler.

It seems that you object to the idea that let bpf prog control the allocating tskey for UDP _as default_.

Correct. All protocols should have the same sensible default behavior.

Quoted 4 lines

I'm not sure if I follow you. Sorry for repeating in case that I miss something:
1) For UDP, do not allocate the tskey from the bpf program, unless it's an alternative/workaround to handle the lockless case. So it's a backup choice.

And an alterative API should apply equally to all protocols too.

Quoted 4 lines

2) Use the exact same way like TCP to finish the correlation on the basis of socket lock protection _as default_. Because the lockless method is seldomly used then we may provide 1) method?
?

The opposite: the default for UDP is the lockless fast path.

Quoted 26 lines

Thanks,
Jason

Quoted 23 lines
Quoted 21 lines

It's worth to highlight that 1) for TCP the time to generate a sendmsg timestamp and generate a tskey are under the same lock, 2) for non-lockless UDP, it works like TCP. But in order to deal with the lockless part, I choose to implement the IPCORK_TS_OPT_ID idea. This is somehow another topic, but the relevant part is that I tried to prove BPF_SOCK_OPS_TS_SND_CB can be also used in UDP (but in a different position compared to TCP protocol).

Quoted 3 lines

BPF will need to set the key for both protocol if SO_TIMESTAMPING is not enabled, right?

For TCP, we will rely on 'shinfo->tskey = TCP_SKB_CB(skb)->seq + skb->len - 1;' in tcp_tx_timestamp() regardless of the status of SO_TIMESTAMPING, so don't bother to manage the tskey from bpf prog. While for UDP, we have to manage the tskey. Of course, it's another topic.

Thanks,
Jason

author
#33 lore raw

Re: [PATCH bpf-next v7 11/13] net-timestamp: add a new callback in tcp_tx_timestamp()

Quoted 224 lines · Willem de Bruijn
Quoted 215 lines · Jason Xing
Quoted 210 lines · Willem de Bruijn
Quoted 199 lines · Jason Xing
Quoted 197 lines · Willem de Bruijn
Quoted 194 lines · Jason Xing
Quoted 116 lines · Willem de Bruijn
Quoted 90 lines · Jason Xing
Quoted 89 lines · Jason Xing
Quoted 85 lines · Willem de Bruijn
Quoted 73 lines · Jason Xing
Quoted 8 lines · Martin KaFai Lau
Quoted 3 lines · Jason Xing

Introduce the callback to correlate tcp_sendmsg timestamp with other
points, like SND/SW/ACK. We can let bpf trace the beginning of
tcp_sendmsg_locked() and fetch the socket addr, so that in

Instead of "fetch the socket addr...", should be "store the sendmsg timestamp at the bpf_sk_storage ...".

I will revise it. Thanks.

Quoted 19 lines
Quoted 6 lines

tcp_tx_timestamp() we can correlate the tskey with the socket addr.

It is accurate since they are under the protect of socket lock.
More details can be found in the selftest.

The selftest uses the bpf_sk_storage to store the sendmsg timestamp at fentry/tcp_sendmsg_locked and retrieves it back at tcp_tx_timestamp (i.e. BPF_SOCK_OPS_TS_SND_CB added in this patch).

Quoted 8 lines

Signed-off-by: Jason Xing <[email protected]>
---
include/uapi/linux/bpf.h | 7 +++++++
net/ipv4/tcp.c | 1 +
tools/include/uapi/linux/bpf.h | 7 +++++++
3 files changed, 15 insertions(+)

Quoted 16 lines
Quoted 14 lines

diff --git a/include/uapi/linux/bpf.h b/include/uapi/linux/bpf.h
index 800122a8abe5..accb3b314fff 100644
--- a/include/uapi/linux/bpf.h
+++ b/include/uapi/linux/bpf.h
@@ -7052,6 +7052,13 @@ enum {
* when SK_BPF_CB_TX_TIMESTAMPING
* feature is on.
*/
+ BPF_SOCK_OPS_TS_SND_CB, /* Called when every sendmsg syscall
+ * is triggered. For TCP, it stays
+ * in the last send process to
+ * correlate with tcp_sendmsg timestamp
+ * with other timestamping callbacks,
+ * like SND/SW/ACK.

Do you have a chance to look at how this will work at UDP?

Sure, I feel like it could not be useful for UDP. Well, things get strange because I did write a long paragraph about this thing which apparently disappeared...

I manage to find what I wrote:
For UDP type, BPF_SOCK_OPS_TS_SND_CB may be not suitable because there are two sending process, 1) lockless path, 2) lock path, which should be handled carefully later. For the former, even though it's unlikely multiple threads access the socket to call sendmsg at the same time, I think we'd better not correlate it like what we do to the TCP case because of the lack of sock lock protection. Considering SND_CB is uapi flag, I think we don't need to forcely add the 'TCP_' prefix in case we need to use it someday.

And one more thing is I'd like to use the v5[1] method in the next round to introduce a new tskey_bpf which is good for UDP type. The new field will not conflict with the tskey in shared info which is generated by sk->sk_tskey in __ip_append_data(). It hardly works if both features (so_timestamping and its bpf extension) exists at the same time. Users could get confused because sometimes they fetch the tskey from skb, sometimes they don't, especially when we have cmsg feature to turn it on/ off per sendmsg. A standalone tskey for bpf extension will be needed. With this tskey_bpf, we can easily correlate the timestamp in sendmsg syscall with other tx points(SND/SW/ACK...).

Quoted 7 lines

[1]: https://lore.kernel.org/all/[email protected]/

If possible, we can leave this question until the UDP support series shows up. I will figure out a better solution :)

In conclusion, it probably won't be used by the UDP type. It's uAPI flag so I consider the compatibility reason.

I don't think this is acceptable. We should aim for an API that can easily be used across protocols, like SO_TIMESTAMPING.

Quoted 6 lines
Quoted 6 lines

After I revisit the UDP SO_TIMESTAMPING again, my thoughts are adjusted like below:

It's hard to provide an absolutely uniform interface or usage to users for TCP and UDP and even more protocols. Cases can be handled one by one.

We should try hard. SO_TIMESTAMPING is uniform across protocols. An interface that is not is just hard to use.

Quoted 6 lines
Quoted 6 lines

The main obstacle is how we can correlate the timestamp in sendmsg syscall with other sending timestamps. It's worth noticing that for SO_TIMESTAMPING the sendmsg timestamp is collected in the userspace. For instance, while skb enters the qdisc, we fail to know which skb belongs to which sendmsg.

Quoted 7 lines
Quoted 7 lines

An idea coming up is to introduce BPF_SOCK_OPS_TS_SND_CB to correlate the sendmsg timestamp with tskey (in tcp_tx_timestamp()) under the protection of socket lock + syscall as the current patch does. But for UDP, it can be lockless. IIUC, there is a very special case where even SO_TIMESTAMPING may get lost: if multiple threads accessing the same socket send UDP packets in parallel, then users could be confused which tskey matches which sendmsg.

Quoted 8 lines

This is a known issue for lockless datagram sockets.

With SO_TIMESTAMPING, but the use of timestamping and of concurrent sendmsg calls is under control of the process, so it only shoots itself in the foot.

With BPF timestamping, a process may confuse a third party admin, so the situation is slightly different.

Agreed.

Quoted 30 lines
Quoted 22 lines
Quoted 4 lines

IIUC, I will not consider this
unlikely case, then the UDP case is quite similar to the TCP case.

Quoted 11 lines

The scenario for the UDP case is:
1) using fentry bpf to hook the udp_sendmsg() to get the timestamp like TCP does in this series.
2) insert BPF_SOCK_OPS_TS_SND_CB into __ip_append_data() near the SO_TIMESTAMPING code snippets to let bpf program correlate the tskey with timestamp.
Note: tskey in UDP will be handled carefully in a different way because we should support both modes for socket timestamping at the same time.
It's really similar to TCP regardless of handling tskey.

To be more precise in case you don't have much time to read the above long paragraph, BPF_SOCK_OPS_TS_SND_CB is mainly used to correlate sendmsg timestamp with corresponding tskey.

1. For TCP, we can correlate it in tcp_tx_timestamp() like this patch does.

Quoted 7 lines

2. For UDP, we can correlate in __ip_append_data() along with those tskey initialization, assuming there are no multiple threads calling locklessly ip_make_skb(). Locked path (udp_sendmsg()->ip_append_data()) works like TCP under the socket lock protection, so it can be easily handled. Lockless path (udp_sendmsg()->ip_make_skb()) can be visited by multiple threads at the same time, which should be handled properly.

Quoted 1 line

Different hook points is fine, as UDP (and RAW) uses __ip_append_data

Then this approach (introducing this new flag) is feasible. Sorry that last night I wrote such a long paragraph which buried something important. Because of that, I rephrase the whole idea about how to let UDP work with this kind of new flag in [patch v8 11/12]. Link is
https://lore.kernel.org/all/CAL+tcoCmXcDot-855XYU7PKCiGvJL=O3CQBGuOTRAs2_=Ys=gg@mail.gmail.com/

Quoted 1 line

or more importantly ip_send_skb, while TCP uses ip_queue_xmit.

For TCP, we use tcp_tx_timestamp to finish the map between sendmsg timestamp and tskey.

Quoted 4 lines

As long as the API is the same: the operation (BPF_SOCK_OPS_TS_SND_CB) and the behavior of that operation. Subject to the usual distinction between protocol behavior (bytestream vs datagram).

I see your point.

Quoted 6 lines
Quoted 4 lines

I prefer to implement
the bpf extension for IPCORK_TS_OPT_ID, which should be another topic, I think. This might be the only one corner case, IIUC?

This sounds like an entirely different topic? Not sure what this is.

Not really a different topic. I mean let bpf prog take the whole control of setting the tskey, then with this BPF_SOCK_OPS_TS_SND_CB flag we can correlate the sendmsg timestamp with tskey. So It has something to do with the usage of UDP. Please take a look at that link to patch 11/12. For TCP, we don't need to care about the value of tskey which has already been taken care of by SO_TIMESTAMPING. So it is slightly different. I'm not sure if this kind of usage is acceptable?

Quoted 8 lines
Quoted 1 line

Why can TCP rely on SO_TIMESTAMPING to set tskey, but UDP cannot?

Because for TCP the shared info tskey is calculated by seqno (in tcp_tx_timestamp()), so it works for so_timestamping and its bpf extension and they are the same. However, for UDP, the shared info tskey can be different, depending on when to call __ip_append_data() and what the sk->sk_tskey is. It can cause conflicts when two modes work at the same time.

Quoted 2 lines

lockless and locked cannot conflict. (if up->pending then the only option is to append to that.)

Quoted 6 lines

Sorry, I should have described more about this point. I was trying to say that if two modes (bpf extension and application timestamping) work at the same time, will the tskey get messed up? Because we have to check if the application mode is turned on. If on, we fetch the existing key generated by the application mode, or else we generate one by modifying the sk->sk_tskey or the tskey of skb?

Quoted 4 lines

This applies to all protocols that implement both. TCP, UDP, eventually RAW and maybe others like L2TP, CAN, MPTCP, ..

Which is why it should be handled the same uniformly.

Got it.

Quoted 35 lines
Quoted 21 lines
Quoted 6 lines
Quoted 5 lines

More than that, lockless UDP case is a tough
one since we cannot correlate the sendmsg timestamp in udp_sendmsg() with the tskey generated in __ip_append_data(),

Quoted 8 lines

With SO_TIMESTAMPING we do not have this distinction between TCP and UDP, so we don't need it here.

It is true that multiple lockless sendmsg calls can race and in that case that correlation is ambiguous. That is also the case for SO_TIMESTAMPING and a known issue.

This is unlikely in most workloads in practice.

Oh, I see.

Quoted 4 lines
Quoted 3 lines

which is a long gap
without any lock. So reuse the IPCORK_TS_OPT_ID logic for bpf extension here can work.

Quoted 6 lines
Quoted 2 lines

It is fine to add a solution to work around the ambiguity. But not to make it a precondition and so diverge the API for TCP and UDP.

There is no divergence in API. BPF always uses SND flag to finish the correlation. The only difference is UDP needs to manage the tskey pool (allocating which tskey to which sendmsg).

Again, I don't see how this is UDP specific.

Quoted 5 lines
Quoted 4 lines

The same argument to choose a key from BPF can be made for TCP to a certain extent.

Quoted 6 lines
Quoted 4 lines

Well...right, but we don't bother to do this for TCP. The TCP case is simpler.

It seems that you object to the idea that let bpf prog control the allocating tskey for UDP _as default_.

Correct. All protocols should have the same sensible default behavior.

I see.

Quoted 7 lines
Quoted 5 lines

I'm not sure if I follow you. Sorry for repeating in case that I miss something:
1) For UDP, do not allocate the tskey from the bpf program, unless it's an alternative/workaround to handle the lockless case. So it's a backup choice.

And an alterative API should apply equally to all protocols too.

It seems feasible.

Quoted 7 lines
Quoted 5 lines

2) Use the exact same way like TCP to finish the correlation on the basis of socket lock protection _as default_. Because the lockless method is seldomly used then we may provide 1) method?
?

The opposite: the default for UDP is the lockless fast path.

Then using SND flag to correlate for UDP is safe (by putting the SND flag in the __ip_append_data() is suitable for at least UDP case).

Now I have a clear vision and goal after so many rounds of discussion. Thanks.

Thanks,
Jason