Skip to contentSkip to conversation

multi release pacing for UDP GSO

RFC PATCH In review Willem de Bruijn 2 participants 9 messages Jun 9, 2020

Subsystems (6)
lore
Share & tools
lore

Willem de Bruijn

author maintainer
#0 lore raw

cover multi release pacing for UDP GSO

From: Willem de Bruijn <[email protected]>

UDP segmentation offload with UDP_SEGMENT can significantly reduce the transmission cycle cost per byte for protocols like QUIC.

Pacing offload with SO_TXTIME can improve accuracy and cycle cost of pacing for such userspace protocols further.

But the maximum GSO size built is limited by the pacing rate. As msec pacing interval, for many Internet clients results in at most a few segments per datagram.

The pros and cons were captured in a recent CloudFlare article, specifically mentioning

"But it does not yet support specifying different times for each packet when GSO is used, as there is no way to define multiple timestamps for packets that need to be segmented (each segmented packet essentially ends up being sent at the same time anyway)."

https://blog.cloudflare.com/accelerating-udp-packet-transmission-for-quic/

We have been evaluating such a mechanism for multiple release times per UDP GSO packets. Since it sounds like it may of interest to others, too, it may be a while before we have all the data I'd like and it's more quiet on the list now that the merge window is open, sharing a WIP version.

The basic approach is to specify

1. initial early release time (in nsec)
2. interval between subsequent release times (in msec)
3. number of segments to release at each release time

One implementation concern is where to store the additional two fields in the skb. Given that msec granularity is the Internet pacing speed, for now repurpose the two lowest 4B nibbles in skb->tstamp to hold the interval and segment count. I'm aware that this does not win a prize for elegance.

Patch 1 adds the socket option and basic segmentation function to
adjust the skb->tstamp of the individual segments.

Patch 2 extends this with support for build GSO segs. Build one GSO
segment per interval if the hardware can offload (USO) and thus we are segmenting only to maintain pacing rate.

Patch 3 wires the segmentation up to the FQ qdisc on enqueue, so that
segments will be scheduled for delivery at their adjusted time.

Patch 4..6 extend existing tests to experiment with the feature

Patch 4 allows testing so_txtime across hardware (for USO) Patch 5 extends the so_txtime test with support for gso and mr-pacing Patch 6 extends the udpgso bench to support pacing and mr-pacing

Some known limitations:

- the aforementioned storage in skb->tstamp.

- exposing this constraint through the SO_TXTIME interface. it is cleaner to add new fields to the cmsg, at nsec resolution.

- the fq_enqueue path adds a branch to the hot path. a static branch would avoid that.

- a few udp specific assumptions in a net/core datapath. notably the hw_features. this can be derived from gso_type.

Willem de Bruijn (6):
net: multiple release time SO_TXTIME
net: build gso segs in multi release time SO_TXTIME
net_sched: sch_fq: multiple release time support selftests/net: so_txtime: support txonly/rxonly modes selftests/net: so_txtime: add gso and multi release pacing selftests/net: upgso bench: add pacing with SO_TXTIME

 include/linux/netdevice.h                     |   1 +
 include/net/sock.h                            |   3 +-
 include/uapi/linux/net_tstamp.h               |   3 +-
 net/core/dev.c                                |  71 +++++++++
 net/core/sock.c                               |   4 +
 net/sched/sch_fq.c                            |  33 ++++-
 tools/testing/selftests/net/so_txtime.c       | 136 ++++++++++++++----
 tools/testing/selftests/net/so_txtime.sh      |   7 +
 .../testing/selftests/net/so_txtime_multi.sh  |  68 +++++++++
 .../selftests/net/udpgso_bench_multi.sh       |  65 +++++++++
 tools/testing/selftests/net/udpgso_bench_tx.c |  72 +++++++++-
 11 files changed, 431 insertions(+), 32 deletions(-)
 create mode 100755 tools/testing/selftests/net/so_txtime_multi.sh
 create mode 100755 tools/testing/selftests/net/udpgso_bench_multi.sh
Signature · 1 line
2.27.0.278.ge193c7cf3a9-goog
author maintainer
#1 lore raw

RFC PATCH 1/6 net: multiple release time SO_TXTIME

From: Willem de Bruijn <[email protected]>

Pace transmission of segments in a UDP GSO datagram.

Batching datagram protocol stack traversals with UDP_SEGMENT saves significant cycles for large data transfers.

But GSO packets are sent at once. Pacing traffic to internet clients often requires sending just a few MSS per msec pacing interval.

SO_TXTIME allows delivery of packets at a later time. Extend it to allow pacing the segments in a UDP GSO packet, to be able to build larger GSO datagrams.

Add SO_TXTIME flag SOF_TXTIME_MULTI_RELEASE. This reinterprets the lower 8 bits of the 64-bit release timestamp as

- bits 4..7: release time interval in usec
- bits 0..3: number of segments sent per period

So a timestamp of 0x148 means

- 0x100 initial timestamp in Qdisc selected clocksource
- every 4 usec release N MSS
- N is 8

A subsequent qdisc change will pace the individual segments.

Packet transmission can race with the socket option. This is safe. For predictable behavior, it is up to the caller to not toggle the feature while packets on a socket are in flight.

---
 include/linux/netdevice.h       |  1 +
 include/net/sock.h              |  3 ++-
 include/uapi/linux/net_tstamp.h |  3 ++-
 net/core/dev.c                  | 44 +++++++++++++++++++++++++++++++++
 net/core/sock.c                 |  4 +++
 5 files changed, 53 insertions(+), 2 deletions(-)
include/linux/netdevice.h +1 -0
index 1a96e9c4ec36..15ea976dd446 100644
@@ -4528,6 +4528,7 @@ struct sk_buff *__skb_gso_segment(struct sk_buff *skb,
 				  netdev_features_t features, bool tx_path);
 struct sk_buff *skb_mac_gso_segment(struct sk_buff *skb,
 				    netdev_features_t features);
+struct sk_buff *skb_gso_segment_txtime(struct sk_buff *skb);
 
 struct netdev_bonding_info {
 	ifslave	slave;
include/net/sock.h +2 -1
index c53cc42b5ab9..491e389b3570 100644
@@ -493,7 +493,8 @@ struct sock {
 	u8			sk_clockid;
 	u8			sk_txtime_deadline_mode : 1,
 				sk_txtime_report_errors : 1,
-				sk_txtime_unused : 6;
+				sk_txtime_multi_release : 1,
+				sk_txtime_unused : 5;
 
 	struct socket		*sk_socket;
 	void			*sk_user_data;
include/uapi/linux/net_tstamp.h +2 -1
index 7ed0b3d1c00a..ca1ae3b6f601 100644
@@ -162,8 +162,9 @@ struct scm_ts_pktinfo {
 enum txtime_flags {
 	SOF_TXTIME_DEADLINE_MODE = (1 << 0),
 	SOF_TXTIME_REPORT_ERRORS = (1 << 1),
+	SOF_TXTIME_MULTI_RELEASE = (1 << 2),
 
-	SOF_TXTIME_FLAGS_LAST = SOF_TXTIME_REPORT_ERRORS,
+	SOF_TXTIME_FLAGS_LAST = SOF_TXTIME_MULTI_RELEASE,
 	SOF_TXTIME_FLAGS_MASK = (SOF_TXTIME_FLAGS_LAST - 1) |
 				 SOF_TXTIME_FLAGS_LAST
 };
net/core/dev.c +44 -0

50 lines not on this page · raw message

net/core/sock.c +4 -0
index 6c4acf1f0220..7036b8855154 100644
@@ -1258,6 +1258,8 @@ int sock_setsockopt(struct socket *sock, int level, int optname,
 			!!(sk_txtime.flags & SOF_TXTIME_DEADLINE_MODE);
 		sk->sk_txtime_report_errors =
 			!!(sk_txtime.flags & SOF_TXTIME_REPORT_ERRORS);
+		sk->sk_txtime_multi_release =
+			!!(sk_txtime.flags & SOF_TXTIME_MULTI_RELEASE);
 		break;
 
 	case SO_BINDTOIFINDEX:
@@ -1608,6 +1610,8 @@ int sock_getsockopt(struct socket *sock, int level, int optname,
 				  SOF_TXTIME_DEADLINE_MODE : 0;
 		v.txtime.flags |= sk->sk_txtime_report_errors ?
 				  SOF_TXTIME_REPORT_ERRORS : 0;
+		v.txtime.flags |= sk->sk_txtime_multi_release ?
+				  SOF_TXTIME_MULTI_RELEASE : 0;
 		break;
 
 	case SO_BINDTOIFINDEX:
Signature · 1 line
2.27.0.278.ge193c7cf3a9-goog
author maintainer
#2 lore raw

RFC PATCH 2/6 net: build gso segs in multi release time SO_TXTIME

In reply to Willem de Bruijn

From: Willem de Bruijn <[email protected]>

When sending multiple segments per interval and the device supports hardware segmentation, build one GSO segment per interval.

---
 net/core/dev.c | 29 ++++++++++++++++++++++++++++-
 1 file changed, 28 insertions(+), 1 deletion(-)
net/core/dev.c +28 -1
index 5058083375fb..05f538f0f631 100644
@@ -3379,7 +3379,9 @@ EXPORT_SYMBOL(__skb_gso_segment);
 
 struct sk_buff *skb_gso_segment_txtime(struct sk_buff *skb)
 {
+	const netdev_features_t hw_features = NETIF_F_GSO_UDP_L4;
 	int mss_per_ival, mss_in_cur_ival;
+	u16 gso_size_orig, gso_segs_orig;
 	struct sk_buff *segs, *seg;
 	struct skb_shared_info *sh;
 	u64 step_ns, tstamp;
@@ -3401,13 +3403,27 @@ struct sk_buff *skb_gso_segment_txtime(struct sk_buff *skb)
 	if (sh->gso_segs <= mss_per_ival)
 		return NULL;
 
+	/* update gso size and segs to build 1 GSO packet per ival */
+	gso_size_orig = sh->gso_size;
+	gso_segs_orig = sh->gso_segs;
+	if (mss_per_ival > 1 && skb->dev->features & hw_features) {
+		sh->gso_size *= mss_per_ival;
+		sh->gso_segs = DIV_ROUND_UP(sh->gso_segs, mss_per_ival);
+		mss_per_ival = 1;
+	}
+
 	segs = skb_gso_segment(skb, NETIF_F_SG | NETIF_F_HW_CSUM);
-	if (IS_ERR_OR_NULL(segs))
+	if (IS_ERR_OR_NULL(segs)) {
+		sh->gso_size = gso_size_orig;
+		sh->gso_segs = gso_segs_orig;
 		return segs;
+	}
 
 	mss_in_cur_ival = 0;
 
 	for (seg = segs; seg; seg = seg->next) {
+		unsigned int data_len, data_off;
+
 		seg->tstamp = tstamp & ~0xFF;
 
 		mss_in_cur_ival++;
@@ -3415,6 +3431,17 @@ struct sk_buff *skb_gso_segment_txtime(struct sk_buff *skb)
 			tstamp += step_ns;
 			mss_in_cur_ival = 0;
 		}
+
+		data_off = skb_checksum_start_offset(skb) +
+			   skb->csum_offset + sizeof(__sum16);
+		data_len = seg->len - data_off;
+
+		if (data_len > gso_size_orig) {
+			sh = skb_shinfo(seg);
+			sh->gso_type = skb_shinfo(skb)->gso_type;
+			sh->gso_size = gso_size_orig;
+			sh->gso_segs = DIV_ROUND_UP(data_len, gso_size_orig);
+		}
 	}
 
 	return segs;
Signature · 1 line
2.27.0.278.ge193c7cf3a9-goog
author maintainer
#3 lore raw

RFC PATCH 3/6 net_sched: sch_fq: multiple release time support

In reply to Willem de Bruijn

From: Willem de Bruijn <[email protected]>

Optionally segment skbs on FQ enqueue, to later send segments at their individual delivery time.

Segmentation on enqueue is new for FQ, but already happens in TBF, CAKE and netem.

This slow patch should probably be behind a static_branch.

---
 net/sched/sch_fq.c | 33 +++++++++++++++++++++++++++++++--
 1 file changed, 31 insertions(+), 2 deletions(-)
net/sched/sch_fq.c +31 -2
index 8f06a808c59a..a5e2c35bb557 100644
@@ -439,8 +439,8 @@ static bool fq_packet_beyond_horizon(const struct sk_buff *skb,
 	return unlikely((s64)skb->tstamp > (s64)(q->ktime_cache + q->horizon));
 }
 
-static int fq_enqueue(struct sk_buff *skb, struct Qdisc *sch,
-		      struct sk_buff **to_free)
+static int __fq_enqueue(struct sk_buff *skb, struct Qdisc *sch,
+			struct sk_buff **to_free)
 {
 	struct fq_sched_data *q = qdisc_priv(sch);
 	struct fq_flow *f;
@@ -496,6 +496,35 @@ static int fq_enqueue(struct sk_buff *skb, struct Qdisc *sch,
 	return NET_XMIT_SUCCESS;
 }
 
+static int fq_enqueue(struct sk_buff *skb, struct Qdisc *sch,
+		      struct sk_buff **to_free)
+{
+	struct sk_buff *segs, *next;
+	int ret;
+
+	if (likely(!skb_is_gso(skb) || !skb->sk ||
+		   !skb->sk->sk_txtime_multi_release))
+		return __fq_enqueue(skb, sch, to_free);
+
+	segs = skb_gso_segment_txtime(skb);
+	if (IS_ERR(segs))
+		return qdisc_drop(skb, sch, to_free);
+	if (!segs)
+		return __fq_enqueue(skb, sch, to_free);
+
+	consume_skb(skb);
+
+	ret = NET_XMIT_DROP;
+	skb_list_walk_safe(segs, segs, next) {
+		skb_mark_not_on_list(segs);
+		qdisc_skb_cb(segs)->pkt_len = segs->len;
+		if (__fq_enqueue(segs, sch, to_free) == NET_XMIT_SUCCESS)
+			ret = NET_XMIT_SUCCESS;
+	}
+
+	return ret;
+}
+
 static void fq_check_throttled(struct fq_sched_data *q, u64 now)
 {
 	unsigned long sample;
Signature · 1 line
2.27.0.278.ge193c7cf3a9-goog
#4 lore raw

Re: [PATCH RFC net-next 3/6] net_sched: sch_fq: multiple release time support

Quoted 31 lines · Willem de Bruijn

From: Willem de Bruijn <[email protected]>

Optionally segment skbs on FQ enqueue, to later send segments at their individual delivery time.

Segmentation on enqueue is new for FQ, but already happens in TBF, CAKE and netem.

This slow patch should probably be behind a static_branch.

---
 net/sched/sch_fq.c | 33 +++++++++++++++++++++++++++++++--
 1 file changed, 31 insertions(+), 2 deletions(-)
net/sched/sch_fq.c +2 -2
index 8f06a808c59a..a5e2c35bb557 100644
@@ -439,8 +439,8 @@ static bool fq_packet_beyond_horizon(const struct sk_buff *skb,
 	return unlikely((s64)skb->tstamp > (s64)(q->ktime_cache + q->horizon));
 }
 
-static int fq_enqueue(struct sk_buff *skb, struct Qdisc *sch,
-		      struct sk_buff **to_free)
+static int __fq_enqueue(struct sk_buff *skb, struct Qdisc *sch,
+			struct sk_buff **to_free)
 {
 	struct fq_sched_data *q = qdisc_priv(sch);
 	struct fq_flow *f;

net/sched/sch_fq.c static int fq_enqueue(struct sk_buff *skb, struct Qdisc *sch, · line 505

Quoted 10 lines
net/sched/sch_fq.c +7 -0
@@ -496,6 +496,35 @@ static int fq_enqueue(struct sk_buff *skb, struct Qdisc *sch,
 	return NET_XMIT_SUCCESS;
 }
 
+static int fq_enqueue(struct sk_buff *skb, struct Qdisc *sch,
+		      struct sk_buff **to_free)
+{
+	struct sk_buff *segs, *next;
+	int ret;
+
+	if (likely(!skb_is_gso(skb) || !skb->sk ||

You also need to check sk_fullsock(skb->sk), otherwise KMSAN might be unhappy.

net/sched/sch_fq.c static int fq_enqueue(struct sk_buff *skb, struct Qdisc *sch, · line 515

Quoted 10 lines
net/sched/sch_fq.c +10 -0
+		   !skb->sk->sk_txtime_multi_release))
+		return __fq_enqueue(skb, sch, to_free);
+
+	segs = skb_gso_segment_txtime(skb);
+	if (IS_ERR(segs))
+		return qdisc_drop(skb, sch, to_free);
+	if (!segs)
+		return __fq_enqueue(skb, sch, to_free);
+
+	consume_skb(skb);

This needs to be qdisc_drop(skb, sch, to_free) if queue is full, see below.

net/sched/sch_fq.c static int fq_enqueue(struct sk_buff *skb, struct Qdisc *sch, · line 520

Quoted 5 lines
net/sched/sch_fq.c +5 -0
+
+	ret = NET_XMIT_DROP;
+	skb_list_walk_safe(segs, segs, next) {
+		skb_mark_not_on_list(segs);
+		qdisc_skb_cb(segs)->pkt_len = segs->len;

This seems to under-estimate bytes sent. See qdisc_pkt_len_init() for details.

net/sched/sch_fq.c static int fq_enqueue(struct sk_buff *skb, struct Qdisc *sch, · line 523

Quoted 3 lines
net/sched/sch_fq.c +3 -0
+		if (__fq_enqueue(segs, sch, to_free) == NET_XMIT_SUCCESS)
+			ret = NET_XMIT_SUCCESS;
+	}
        if (unlikely(ret == NET_XMIT_DROP))
            qdisc_drop(skb, sch, to_free);
        else
            consume_skb(skb);
Quoted 8 lines
net/sched/sch_fq.c +4 -0
+
+	return ret;
+}
+
 static void fq_check_throttled(struct fq_sched_data *q, u64 now)
 {
 	unsigned long sample;
#5 lore raw

Re: [PATCH RFC net-next 3/6] net_sched: sch_fq: multiple release time support

Quoted 68 lines · Eric Dumazet
Quoted 31 lines · Willem de Bruijn

From: Willem de Bruijn <[email protected]>

Optionally segment skbs on FQ enqueue, to later send segments at their individual delivery time.

Segmentation on enqueue is new for FQ, but already happens in TBF, CAKE and netem.

This slow patch should probably be behind a static_branch.

---
 net/sched/sch_fq.c | 33 +++++++++++++++++++++++++++++++--
 1 file changed, 31 insertions(+), 2 deletions(-)
net/sched/sch_fq.c +2 -2
index 8f06a808c59a..a5e2c35bb557 100644
@@ -439,8 +439,8 @@ static bool fq_packet_beyond_horizon(const struct sk_buff *skb,
 	return unlikely((s64)skb->tstamp > (s64)(q->ktime_cache + q->horizon));
 }
 
-static int fq_enqueue(struct sk_buff *skb, struct Qdisc *sch,
-		      struct sk_buff **to_free)
+static int __fq_enqueue(struct sk_buff *skb, struct Qdisc *sch,
+			struct sk_buff **to_free)
 {
 	struct fq_sched_data *q = qdisc_priv(sch);
 	struct fq_flow *f;

net/sched/sch_fq.c static int fq_enqueue(struct sk_buff *skb, struct Qdisc *sch, · line 505

Quoted 10 lines
net/sched/sch_fq.c +7 -0
@@ -496,6 +496,35 @@ static int fq_enqueue(struct sk_buff *skb, struct Qdisc *sch,
 	return NET_XMIT_SUCCESS;
 }
 
+static int fq_enqueue(struct sk_buff *skb, struct Qdisc *sch,
+		      struct sk_buff **to_free)
+{
+	struct sk_buff *segs, *next;
+	int ret;
+
+	if (likely(!skb_is_gso(skb) || !skb->sk ||

You also need to check sk_fullsock(skb->sk), otherwise KMSAN might be unhappy.

net/sched/sch_fq.c static int fq_enqueue(struct sk_buff *skb, struct Qdisc *sch, · line 515

Quoted 10 lines
net/sched/sch_fq.c +10 -0
+		   !skb->sk->sk_txtime_multi_release))
+		return __fq_enqueue(skb, sch, to_free);
+
+	segs = skb_gso_segment_txtime(skb);
+	if (IS_ERR(segs))
+		return qdisc_drop(skb, sch, to_free);
+	if (!segs)
+		return __fq_enqueue(skb, sch, to_free);
+
+	consume_skb(skb);

This needs to be qdisc_drop(skb, sch, to_free) if queue is full, see below.

net/sched/sch_fq.c static int fq_enqueue(struct sk_buff *skb, struct Qdisc *sch, · line 520

Quoted 5 lines
net/sched/sch_fq.c +5 -0
+
+	ret = NET_XMIT_DROP;
+	skb_list_walk_safe(segs, segs, next) {
+		skb_mark_not_on_list(segs);
+		qdisc_skb_cb(segs)->pkt_len = segs->len;

This seems to under-estimate bytes sent. See qdisc_pkt_len_init() for details.

Quoted 6 lines

net/sched/sch_fq.c static int fq_enqueue(struct sk_buff *skb, struct Qdisc *sch, · line 523

Quoted 3 lines
net/sched/sch_fq.c +3 -0
+		if (__fq_enqueue(segs, sch, to_free) == NET_XMIT_SUCCESS)
+			ret = NET_XMIT_SUCCESS;
+	}
        if (unlikely(ret == NET_XMIT_DROP))
            qdisc_drop(skb, sch, to_free);

Maybe not qdisc_drop() (qdisc drop counters are already updated) but at least kfree_skb() so that drop monitor is not fooled.

Quoted 13 lines
        else
            consume_skb(skb);
Quoted 8 lines
net/sched/sch_fq.c +4 -0
+
+	return ret;
+}
+
 static void fq_check_throttled(struct fq_sched_data *q, u64 now)
 {
 	unsigned long sample;
author maintainer
#6 lore raw

RFC PATCH 4/6 selftests/net: so_txtime: support txonly/rxonly modes

In reply to Willem de Bruijn

From: Willem de Bruijn <[email protected]>

Allow running the test across two machines, to test nic hw offload.

Add options
-A: receiver address
-r: receive only
-t: transmit only
-T: SO_RCVTIMEO value

---
 tools/testing/selftests/net/so_txtime.c | 60 ++++++++++++++++++++-----
 1 file changed, 48 insertions(+), 12 deletions(-)
tools/testing/selftests/net/so_txtime.c +48 -12
index 383bac05ac32..fa748e4209c0 100644
@@ -28,9 +28,13 @@
 #include <time.h>
 #include <unistd.h>
 
+static const char *cfg_addr;
 static int	cfg_clockid	= CLOCK_TAI;
 static bool	cfg_do_ipv4;
 static bool	cfg_do_ipv6;
+static bool	cfg_rxonly;
+static int	cfg_timeout_sec;
+static bool	cfg_txonly;
 static uint16_t	cfg_port	= 8000;
 static int	cfg_variance_us	= 4000;
 
@@ -238,8 +242,12 @@ static int setup_rx(struct sockaddr *addr, socklen_t alen)
 	if (fd == -1)
 		error(1, errno, "socket r");
 
-	if (bind(fd, addr, alen))
-		error(1, errno, "bind");
+	if (!cfg_txonly)
+		if (bind(fd, addr, alen))
+			error(1, errno, "bind");
+
+	if (cfg_timeout_sec)
+		tv.tv_sec = cfg_timeout_sec;
 
 	if (setsockopt(fd, SOL_SOCKET, SO_RCVTIMEO, &tv, sizeof(tv)))
 		error(1, errno, "setsockopt rcv timeout");
@@ -260,13 +268,18 @@ static void do_test(struct sockaddr *addr, socklen_t alen)
 
 	glob_tstart = gettime_ns();
 
-	for (i = 0; i < cfg_num_pkt; i++)
-		do_send_one(fdt, &cfg_in[i]);
-	for (i = 0; i < cfg_num_pkt; i++)
-		if (do_recv_one(fdr, &cfg_out[i]))
-			do_recv_errqueue_timeout(fdt);
+	if (!cfg_rxonly) {
+		for (i = 0; i < cfg_num_pkt; i++)
+			do_send_one(fdt, &cfg_in[i]);
+	}
 
-	do_recv_verify_empty(fdr);
+	if (!cfg_txonly) {
+		for (i = 0; i < cfg_num_pkt; i++)
+			if (do_recv_one(fdr, &cfg_out[i]))
+				do_recv_errqueue_timeout(fdt);
+
+		do_recv_verify_empty(fdr);
+	}
 
 	if (close(fdr))
 		error(1, errno, "close r");
@@ -308,7 +321,7 @@ static void parse_opts(int argc, char **argv)
 {
 	int c, ilen, olen;
 
-	while ((c = getopt(argc, argv, "46c:")) != -1) {
+	while ((c = getopt(argc, argv, "46A:c:rtT:")) != -1) {
 		switch (c) {
 		case '4':
 			cfg_do_ipv4 = true;
@@ -316,6 +329,9 @@ static void parse_opts(int argc, char **argv)
 		case '6':
 			cfg_do_ipv6 = true;
 			break;
+		case 'A':
+			cfg_addr = optarg;
+			break;
 		case 'c':
 			if (!strcmp(optarg, "tai"))
 				cfg_clockid = CLOCK_TAI;
@@ -325,13 +341,27 @@ static void parse_opts(int argc, char **argv)
 			else
 				error(1, 0, "unknown clock id %s", optarg);
 			break;
+		case 'r':
+			cfg_rxonly = true;
+			break;
+		case 't':
+			cfg_txonly = true;
+			break;
+		case 'T':
+			cfg_timeout_sec = strtol(optarg, NULL, 0);
+			break;
 		default:
 			error(1, 0, "parse error at %d", optind);
 		}
 	}
 
 	if (argc - optind != 2)
-		error(1, 0, "Usage: %s [-46] -c <clock> <in> <out>", argv[0]);
+		error(1, 0, "Usage: %s [-46rt] [-A addr] [-c clock] [-T timeout] <in> <out>", argv[0]);
+
+	if (cfg_rxonly && cfg_txonly)
+		error(1, 0, "Select rx-only or tx-only, not both");
+	if (cfg_addr && cfg_do_ipv4 && cfg_do_ipv6)
+		error(1, 0, "Cannot run both IPv4 and IPv6 when passing address");
 
 	ilen = parse_io(argv[optind], cfg_in);
 	olen = parse_io(argv[optind + 1], cfg_out);
@@ -349,7 +379,10 @@ int main(int argc, char **argv)
 
 		addr6.sin6_family = AF_INET6;
 		addr6.sin6_port = htons(cfg_port);
-		addr6.sin6_addr = in6addr_loopback;
+		if (!cfg_addr)
+			addr6.sin6_addr = in6addr_loopback;
+		else if (inet_pton(AF_INET6, cfg_addr, &addr6.sin6_addr) != 1)
+			error(1, 0, "ipv6 parse error: %s", cfg_addr);
 
 		cfg_errq_level = SOL_IPV6;
 		cfg_errq_type = IPV6_RECVERR;
@@ -362,7 +395,10 @@ int main(int argc, char **argv)
 
 		addr4.sin_family = AF_INET;
 		addr4.sin_port = htons(cfg_port);
-		addr4.sin_addr.s_addr = htonl(INADDR_LOOPBACK);
+		if (!cfg_addr)
+			addr4.sin_addr.s_addr = htonl(INADDR_LOOPBACK);
+		else if (inet_pton(AF_INET, cfg_addr, &addr4.sin_addr) != 1)
+			error(1, 0, "ipv4 parse error: %s", cfg_addr);
 
 		cfg_errq_level = SOL_IP;
 		cfg_errq_type = IP_RECVERR;
Signature · 1 line
2.27.0.278.ge193c7cf3a9-goog
author maintainer
#7 lore raw

RFC PATCH 5/6 selftests/net: so_txtime: add gso and multi release pacing

In reply to Willem de Bruijn

From: Willem de Bruijn <[email protected]>

Support sending more than 1B payload and passing segment size.

- add option '-s' to send more than 1B payload
- add option '-m' to configure mss

If size exceeds mss, enable UDP_SEGMENT.

Optionally also allow configuring multi release time.

- add option '-M' to set release interval (msec)
- add option '-N' to set release segment count

Both options have to be specified, or neither.

Add a testcase to so_txtime.sh over loopback.
Add a testscript so_txtime_multi.sh that operates over veth using txonly/rxonly mode.

Also fix a small sock_extended_err parsing bug, mixing up ee_code and ee_errno

---
 tools/testing/selftests/net/so_txtime.c       | 80 +++++++++++++++----
 tools/testing/selftests/net/so_txtime.sh      |  7 ++
 .../testing/selftests/net/so_txtime_multi.sh  | 68 ++++++++++++++++
 3 files changed, 141 insertions(+), 14 deletions(-)
 create mode 100755 tools/testing/selftests/net/so_txtime_multi.sh
tools/testing/selftests/net/so_txtime.c +66 -14
index fa748e4209c0..da4ef411693c 100644
@@ -17,6 +17,7 @@
 #include <linux/errqueue.h>
 #include <linux/ipv6.h>
 #include <linux/tcp.h>
+#include <netinet/udp.h>
 #include <stdbool.h>
 #include <stdlib.h>
 #include <stdio.h>
@@ -32,7 +33,11 @@ static const char *cfg_addr;
 static int	cfg_clockid	= CLOCK_TAI;
 static bool	cfg_do_ipv4;
 static bool	cfg_do_ipv6;
+static uint8_t	cfg_mr_num;
+static uint8_t	cfg_mr_ival;
+static int	cfg_mss		= 1400;
 static bool	cfg_rxonly;
+static uint16_t	cfg_size	= 1;
 static int	cfg_timeout_sec;
 static bool	cfg_txonly;
 static uint16_t	cfg_port	= 8000;
@@ -66,6 +71,7 @@ static uint64_t gettime_ns(void)
 
 static void do_send_one(int fdt, struct timed_send *ts)
 {
+	static char buf[1 << 16];
 	char control[CMSG_SPACE(sizeof(uint64_t))];
 	struct msghdr msg = {0};
 	struct iovec iov = {0};
@@ -73,8 +79,10 @@ static void do_send_one(int fdt, struct timed_send *ts)
 	uint64_t tdeliver;
 	int ret;
 
-	iov.iov_base = &ts->data;
-	iov.iov_len = 1;
+	memset(buf, ts->data, cfg_size);
+
+	iov.iov_base = buf;
+	iov.iov_len = cfg_size;
 
 	msg.msg_iov = &iov;
 	msg.msg_iovlen = 1;
@@ -85,6 +93,11 @@ static void do_send_one(int fdt, struct timed_send *ts)
 		msg.msg_controllen = sizeof(control);
 
 		tdeliver = glob_tstart + ts->delay_us * 1000;
+		if (cfg_mr_ival) {
+			tdeliver &= ~0xFF;
+			tdeliver |= cfg_mr_ival << 4;
+			tdeliver |= cfg_mr_num;
+		}
 
 		cm = CMSG_FIRSTHDR(&msg);
 		cm->cmsg_level = SOL_SOCKET;
@@ -104,30 +117,41 @@ static void do_send_one(int fdt, struct timed_send *ts)
 static bool do_recv_one(int fdr, struct timed_send *ts)
 {
 	int64_t tstop, texpect;
+	int total = 0;
 	char rbuf[2];
 	int ret;
 
-	ret = recv(fdr, rbuf, sizeof(rbuf), 0);
+read_again:
+	ret = recv(fdr, rbuf, sizeof(rbuf), MSG_TRUNC);
 	if (ret == -1 && errno == EAGAIN)
-		return true;
+		goto timedout;
 	if (ret == -1)
 		error(1, errno, "read");
-	if (ret != 1)
-		error(1, 0, "read: %dB", ret);
 
 	tstop = (gettime_ns() - glob_tstart) / 1000;
 	texpect = ts->delay_us >= 0 ? ts->delay_us : 0;
 
-	fprintf(stderr, "payload:%c delay:%lld expected:%lld (us)\n",
-			rbuf[0], (long long)tstop, (long long)texpect);
+	fprintf(stderr, "payload:%c delay:%lld expected:%lld (us) -- read=%d,len=%d,total=%d\n",
+			rbuf[0], (long long)tstop, (long long)texpect,
+			total, ret, cfg_size);
 
 	if (rbuf[0] != ts->data)
 		error(1, 0, "payload mismatch. expected %c", ts->data);
 
-	if (labs(tstop - texpect) > cfg_variance_us)
+	total += ret;
+	if (total < cfg_size)
+		goto read_again;
+
+	/* measure latency if all data arrives in a single datagram (not GSO) */
+	if (ret == cfg_size && labs(tstop - texpect) > cfg_variance_us)
 		error(1, 0, "exceeds variance (%d us)", cfg_variance_us);
 
 	return false;
+
+timedout:
+	if (total != 0 && total != cfg_size)
+		error(1, 0, "timeout mid-read");
+	return true;
 }
 
 static void do_recv_verify_empty(int fdr)
@@ -168,7 +192,9 @@ static void do_recv_errqueue_timeout(int fdt)
 			break;
 		if (ret == -1)
 			error(1, errno, "errqueue");
-		if (msg.msg_flags != MSG_ERRQUEUE)
+		if (ret != sizeof(data))
+			error(1, errno, "insufficient data");
+		if (msg.msg_flags & ~(MSG_ERRQUEUE | MSG_TRUNC))
 			error(1, 0, "errqueue: flags 0x%x\n", msg.msg_flags);
 
 		cm = CMSG_FIRSTHDR(&msg);
@@ -180,7 +206,9 @@ static void do_recv_errqueue_timeout(int fdt)
 		err = (struct sock_extended_err *)CMSG_DATA(cm);
 		if (err->ee_origin != SO_EE_ORIGIN_TXTIME)
 			error(1, 0, "errqueue: origin 0x%x\n", err->ee_origin);
-		if (err->ee_code != ECANCELED)
+		if (err->ee_errno != ECANCELED)
+			error(1, 0, "errqueue: errno 0x%x\n", err->ee_errno);
+		if (err->ee_code != SO_EE_CODE_TXTIME_MISSED)
 			error(1, 0, "errqueue: code 0x%x\n", err->ee_code);
 
 		tstamp = ((int64_t) err->ee_data) << 32 | err->ee_info;
@@ -202,7 +230,7 @@ static void setsockopt_txtime(int fd)
 	struct sock_txtime so_txtime_val_read = { 0 };
 	socklen_t vallen = sizeof(so_txtime_val);
 
-	so_txtime_val.flags = SOF_TXTIME_REPORT_ERRORS;
+	so_txtime_val.flags = SOF_TXTIME_REPORT_ERRORS | SOF_TXTIME_MULTI_RELEASE;
 
 	if (setsockopt(fd, SOL_SOCKET, SO_TXTIME,
 		       &so_txtime_val, sizeof(so_txtime_val)))
@@ -230,6 +258,12 @@ static int setup_tx(struct sockaddr *addr, socklen_t alen)
 
 	setsockopt_txtime(fd);
 
+	if (cfg_size > cfg_mss) {
+		if (setsockopt(fd, SOL_UDP, UDP_SEGMENT,
+			       &cfg_mss, sizeof(cfg_mss)))
+			error(1, errno, "setsockopt udp segment");
+	}
+
 	return fd;
 }
 
@@ -321,7 +355,7 @@ static void parse_opts(int argc, char **argv)
 {
 	int c, ilen, olen;
 
-	while ((c = getopt(argc, argv, "46A:c:rtT:")) != -1) {
+	while ((c = getopt(argc, argv, "46A:c:m:M:N:rs:tT:")) != -1) {
 		switch (c) {
 		case '4':
 			cfg_do_ipv4 = true;
@@ -341,9 +375,25 @@ static void parse_opts(int argc, char **argv)
 			else
 				error(1, 0, "unknown clock id %s", optarg);
 			break;
+		case 'm':
+			cfg_mss = strtol(optarg, NULL, 0);
+			break;
+		case 'M':
+			cfg_mr_ival = atoi(optarg);
+			if (cfg_mr_ival > 0xF)
+				error(1, 0, "multi release ival exceeds max");
+			break;
+		case 'N':
+			cfg_mr_num = atoi(optarg);
+			if (cfg_mr_num > 0xF)
+				error(1, 0, "multi release count exceeds max");
+			break;
 		case 'r':
 			cfg_rxonly = true;
 			break;
+		case 's':
+			cfg_size = atoi(optarg);
+			break;
 		case 't':
 			cfg_txonly = true;
 			break;
@@ -356,12 +406,14 @@ static void parse_opts(int argc, char **argv)
 	}
 
 	if (argc - optind != 2)
-		error(1, 0, "Usage: %s [-46rt] [-A addr] [-c clock] [-T timeout] <in> <out>", argv[0]);
+		error(1, 0, "Usage: %s [-46rt] [-A addr] [-c clock] [-m mtu] [-M ival] [-N num] [-s size] [-T timeout] <in> <out>", argv[0]);
 
 	if (cfg_rxonly && cfg_txonly)
 		error(1, 0, "Select rx-only or tx-only, not both");
 	if (cfg_addr && cfg_do_ipv4 && cfg_do_ipv6)
 		error(1, 0, "Cannot run both IPv4 and IPv6 when passing address");
+	if (!!cfg_mr_ival ^ !!cfg_mr_num)
+		error(1, 0, "Multi release pacing requires both -M and -N");
 
 	ilen = parse_io(argv[optind], cfg_in);
 	olen = parse_io(argv[optind + 1], cfg_out);
tools/testing/selftests/net/so_txtime.sh +7 -0
index 3f7800eaecb1..7c60c11717e4 100755
@@ -16,13 +16,20 @@ fi
 
 set -e
 
+ip link set dev lo mtu 1500
 tc qdisc add dev lo root fq
+
 ./so_txtime -4 -6 -c mono a,-1 a,-1
 ./so_txtime -4 -6 -c mono a,0 a,0
 ./so_txtime -4 -6 -c mono a,10 a,10
 ./so_txtime -4 -6 -c mono a,10,b,20 a,10,b,20
 ./so_txtime -4 -6 -c mono a,20,b,10 b,20,a,20
 
+# test gso
+./so_txtime -4 -6 -m 1000 -s 3500 -c mono a,50,b,100 a,50,b,100
+./so_txtime -4 -6 -m 1000 -s 3500 -M 5 -N 1 -c mono a,50,b,100 a,50,b,100
+./so_txtime -4 -6 -m 1000 -s 3500 -M 5 -N 2 -c mono a,50,b,100 a,50,b,100
+
 if tc qdisc replace dev lo root etf clockid CLOCK_TAI delta 400000; then
 	! ./so_txtime -4 -6 -c tai a,-1 a,-1
 	! ./so_txtime -4 -6 -c tai a,0 a,0
tools/testing/selftests/net/so_txtime_multi.sh new file +68 -0

68 lines not on this page · raw message

Signature · 1 line
2.27.0.278.ge193c7cf3a9-goog
author maintainer
#8 lore raw

RFC PATCH 6/6 selftests/net: upgso bench: add pacing with SO_TXTIME

In reply to Willem de Bruijn

From: Willem de Bruijn <[email protected]>

Enable passing an SCM_TXTIME ('-x'), optionally configured to use multi release (SOF_TXTIME_MULTI_RELEASE) ('-X').

With multi release, the segments that make up a single udp gso packet can be released over time, as opposed to burst at once.

Repurpose the lower 8 bits of the 64 bit timestamp for this purpose.
- bits 4..7: delay between transmit periods in msec
- bits 0..3: number of segments sent per period

Also add an optional delay in usec between sendmsg calls ('-d'). To reduce throughput sufficiently to observe pacing delay introduced.

Added udpgso_bench_multi.sh, derived from so_txtime_multi.sh, to run tests over veth.

Also fix up a minor issue where sizeof the wrong field was used (mss, instead of gso_size). They happen to both be uint16_t.

Tested:

    ./udpgso_bench_multi.sh

    or manually ran across two hosts:

    # sender
    #
    # -S 6000 -s 1400:  6000B buffer encoding 1400B datagrams
    # -d 200000:        200 msec delay between sendmsg
    # -x 0x989611:      ~1000000 nsec first delay + 1 MSS every 1 msec

    ./udpgso_bench_tx -6 -D fd00::1 \
        -l 1 -s 6000 -S 1400 -v
        -d 200000 -x 0x989611 -X

    # receiver

    tcpdump -n -i eth1 -c 100 udp and port 8000 &
    sleep 0.2
    ./udpgso_bench_rx

    16:29:45.146855 IP6 host1.40803 > host2.8000: UDP, length 1400
    16:29:45.147798 IP6 host1.40803 > host2.8000: UDP, length 1400
    16:29:45.148797 IP6 host1.40803 > host2.8000: UDP, length 1400
    16:29:45.149797 IP6 host1.40803 > host2.8000: UDP, length 1400
    16:29:45.150796 IP6 host1.40803 > host2.8000: UDP, length 400
    16:29:45.347056 IP6 host1.40803 > host2.8000: UDP, length 1400
    16:29:45.348000 IP6 host1.40803 > host2.8000: UDP, length 1400
    16:29:45.349000 IP6 host1.40803 > host2.8000: UDP, length 1400
    16:29:45.349999 IP6 host1.40803 > host2.8000: UDP, length 1400
    16:29:45.350999 IP6 host1.40803 > host2.8000: UDP, length 400
---
 .../selftests/net/udpgso_bench_multi.sh       | 65 +++++++++++++++++
 tools/testing/selftests/net/udpgso_bench_tx.c | 72 +++++++++++++++++--
 2 files changed, 133 insertions(+), 4 deletions(-)
 create mode 100755 tools/testing/selftests/net/udpgso_bench_multi.sh
tools/testing/selftests/net/udpgso_bench_multi.sh new file +65 -0
new file mode 100755
index 000000000000..c29f75aec759
@@ -0,0 +1,65 @@
+#!/bin/bash
+# SPDX-License-Identifier: GPL-2.0
+#
+# Regression tests for the SO_TXTIME interface
+
+readonly ns_prefix="ns-sotxtime-"
+readonly ns1="${ns_prefix}1"
+readonly ns2="${ns_prefix}2"
+
+readonly ns1_v4=192.168.1.1
+readonly ns2_v4=192.168.1.2
+readonly ns1_v6=fd::1
+readonly ns2_v6=fd::2
+
+set -eu
+
+cleanup() {
+	ip netns del "${ns2}"
+	ip netns del "${ns1}"
+}
+
+setup() {
+	ip netns add "${ns1}"
+	ip netns add "${ns2}"
+
+	ip link add dev veth1 mtu 1500 netns "${ns1}" type veth \
+	      peer name veth2 mtu 1500 netns "${ns2}"
+
+	ip -netns "${ns1}" link set veth1 up
+	ip -netns "${ns2}" link set veth2 up
+
+	ip -netns "${ns1}" -4 addr add "${ns1_v4}/24" dev veth1
+	ip -netns "${ns2}" -4 addr add "${ns2_v4}/24" dev veth2
+	ip -netns "${ns1}" -6 addr add "${ns1_v6}/64" dev veth1 nodad
+	ip -netns "${ns2}" -6 addr add "${ns2_v6}/64" dev veth2 nodad
+
+	ip netns exec "${ns1}" tc qdisc add dev veth1 root fq
+}
+
+run_test() {
+	ip netns exec "${ns2}" tcpdump -q -n -i veth2 udp &
+	ip netns exec "${ns2}" ./udpgso_bench_rx &
+	sleep 0.1
+	ip netns exec "${ns1}" ./udpgso_bench_tx $@
+	pkill -P $$
+}
+
+run_test_46() {
+	run_test -4 -D "${ns2_v4}" $@
+	run_test -6 -D "${ns2_v6}" $@
+}
+
+trap cleanup EXIT
+setup
+
+echo "gso + pacing"
+TEST_ARGS="-l 1 -s 3500 -S 1000 -v -d 200000 -x 1000000"
+run_test_46 ${TEST_ARGS} -x 1000000
+
+echo "gso + multi release pacing"
+run_test_46 ${TEST_ARGS} -X -x 0x989611
+run_test_46 ${TEST_ARGS} -X -x 0x9896A2
+
+# Does not validate pacing delay yet. Check manually.
+echo "Ok. Executed tests."
tools/testing/selftests/net/udpgso_bench_tx.c +68 -4

155 lines not on this page · raw message

Signature · 1 line
2.27.0.278.ge193c7cf3a9-goog