tcp: support non-GSO jumbograms
Subsystems (5)
cover tcp: support non-GSO jumbograms
This series adds support for sending TCP jumbograms over MTUs above 65535, and adds support for setting such MTUs on veth devices.
The TCP stack is already capable of receiving jumbograms and (up until recently) sending GSO-jumbograms that get segmented before exiting the host. However, the TCP stack doesn't support sending regular jumbograms over high MTUs as described in RFC2675. Specifically, the TCP stack doesn't support an MSS greater than 65535.
No device currently supports setting an MTU greater than 65535 but there are some devices that could benefit from higher MTUs and that can theoretically support it. For example, IPoIB in connected-mode can support MTUs up to 2^31. Virtual devices such as veth have no physical limitations and therefore could also support such MTUs.
This series only adds support for setting high MTUs on veth devices, with support for other devices being delegated to future patch series. Allowing jumbograms to pass through veth devices is useful for testing jumbogram functionality, and also increases throughput when compared to BIG TCP (see the benchmark numbers below).
In addition to removing the upper limit on veth MTUs, This series adds the missing pieces that allow the TCP stack to send TCP jumbograms. This includes a bit of code removed recently by Alice's patch series [1]. Most of the patches in this series are rather trivial. The main problem addressed by this series is that the TCP stack conflates the TSO segment length with the MSS, but the TSO segment length is a 16-bit integer, which doesn't allow for MSS > 65535. This series decouples them.
Running iperf over veth on an Intel Core i9-12900K CPU shows a 5% improvement in throughput.
Commands:
iperf3 -s
iperf3 -c -l 500KiB ${SERVER_IP}%veth0
MTU=524280, gso disabled:
[ ID] Interval Transfer Bitrate [ 5] 0.00-1.00 sec 24.8 GBytes 213 Gbits/sec [ 5] 1.00-2.00 sec 25.5 GBytes 219 Gbits/sec [ 5] 2.00-3.00 sec 26.0 GBytes 223 Gbits/sec [ 5] 3.00-4.00 sec 25.8 GBytes 222 Gbits/sec [ 5] 4.00-5.00 sec 25.8 GBytes 221 Gbits/sec [ 5] 5.00-6.00 sec 25.4 GBytes 218 Gbits/sec [ 5] 6.00-7.00 sec 25.3 GBytes 217 Gbits/sec [ 5] 7.00-8.00 sec 26.1 GBytes 224 Gbits/sec [ 5] 8.00-9.00 sec 25.9 GBytes 223 Gbits/sec [ 5] 9.00-10.00 sec 25.6 GBytes 220 Gbits/sec [ 5] 10.00-10.00 sec 28.3 MBytes 154 Gbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bitrate
[ 5] 0.00-10.00 sec 256 GBytes 220 Gbits/sec
MTU=1500, gso_max_size=524280:
[ ID] Interval Transfer Bitrate [ 5] 0.00-1.00 sec 24.1 GBytes 207 Gbits/sec [ 5] 1.00-2.00 sec 24.1 GBytes 207 Gbits/sec [ 5] 2.00-3.00 sec 24.6 GBytes 211 Gbits/sec [ 5] 3.00-4.00 sec 24.4 GBytes 209 Gbits/sec [ 5] 4.00-5.00 sec 24.5 GBytes 210 Gbits/sec [ 5] 5.00-6.00 sec 24.3 GBytes 209 Gbits/sec [ 5] 6.00-7.00 sec 24.6 GBytes 211 Gbits/sec [ 5] 7.00-8.00 sec 24.0 GBytes 206 Gbits/sec [ 5] 8.00-9.00 sec 23.9 GBytes 205 Gbits/sec [ 5] 9.00-10.00 sec 24.1 GBytes 207 Gbits/sec [ 5] 10.00-10.00 sec 7.81 MBytes 120 Gbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bitrate
[ 5] 0.00-10.00 sec 243 GBytes 208 Gbits/sec
[1] [email protected] (full link was blocked by gmail)
Mariusz Klimek (10):
ipv6: do not fragment packets into jumbograms
ipv6: allow route exceptions with MTUs above 65535
ipv6: add jumbo payload option to non-gso jumbograms
tcp: decouple TSO segment length from MSS
tcp: split jumbograms with urgent pointer correctly
tcp: set MSS correctly for PMTU above 65535
veth: raise the max MTU above 65535
selftests/net: test sending TCP jumbograms over veth selftests/net: add test cases with MTU above 65535 to big_tcp.sh selftests/net: add jumbogram test case to msg_zerocopy.sh
drivers/net/veth.c | 8 +- include/linux/ipv6.h | 6 + include/net/ip6_route.h | 5 +- include/net/ipv6.h | 11 + include/net/tcp.h | 12 +- net/ipv4/tcp.c | 12 +- net/ipv4/tcp_output.c | 84 +++-- net/ipv4/tcp_timer.c | 4 +- net/ipv6/ip6_output.c | 24 +- net/ipv6/route.c | 2 +- tools/testing/selftests/net/Makefile | 3 + tools/testing/selftests/net/big_tcp.sh | 24 +- tools/testing/selftests/net/jumbogram.bpf.c | 36 ++ tools/testing/selftests/net/jumbogram.sh | 380 ++++++++++++++++++++ tools/testing/selftests/net/jumbogram_rx.c | 199 ++++++++++ tools/testing/selftests/net/jumbogram_tx.c | 139 +++++++ tools/testing/selftests/net/msg_zerocopy.c | 3 +- tools/testing/selftests/net/msg_zerocopy.sh | 3 +- 18 files changed, 897 insertions(+), 58 deletions(-) create mode 100644 tools/testing/selftests/net/jumbogram.bpf.c create mode 100755 tools/testing/selftests/net/jumbogram.sh create mode 100644 tools/testing/selftests/net/jumbogram_rx.c create mode 100644 tools/testing/selftests/net/jumbogram_tx.c
Signature · 1 line
2.47.3
PATCH 1/10 ipv6: do not fragment packets into jumbograms
This patch prevents packets from being fragmented into jumbograms by capping the effective MTU to IP6_MAX_MTU in ip6_fragment, as RFC2675 prohibits a Jumbo Payload option from being carried in a packet that carries a fragment header.
--- net/ipv6/ip6_output.c | 2 ++ 1 file changed, 2 insertions(+)
net/ipv6/ip6_output.c +2 -0
@@ -935,6 +935,8 @@ int ip6_fragment(struct net *net, struct sock *sk, struct sk_buff *skb, }if (mtu < hlen +sizeof (struct frag_hdr ) +8 ) goto fail_toobig; +else if (unlikely (mtu >IP6_MAX_MTU )) + mtu =IP6_MAX_MTU ; mtu -= hlen +sizeof (struct frag_hdr ); frag_id = ipv6_select_ident(net, &ipv6_hdr(skb)->daddr,
Signature · 1 line
2.47.3
PATCH 2/10 ipv6: allow route exceptions with MTUs above 65535
In reply to Mariusz KlimekThis patch allows route exceptions to specify an MTU above 65535 so that PMTU discovery can settle on an PMTU above 65535 (if the MTUs along the path allow for it).
IP6_MAX_JUMBOGRAM_MTU is set to INT_MAX rather than UINT_MAX because MTU values and packet lengths are sometimes stored as signed integers.
--- include/linux/ipv6.h | 6 ++++++ include/net/ip6_route.h | 5 +++-- net/ipv6/route.c | 2 +- 3 files changed, 10 insertions(+), 3 deletions(-)
include/linux/ipv6.h +6 -0
@@ -196,6 +196,12 @@ static inline bool ipv6_l3mdev_skb(__u16 flags)#define IP6CB (skb) ((struct inet6_skb_parm*)((skb)->cb))#define IP6CBMTU (skb) ((struct ip6_mtuinfo *)((skb)->cb)) +/* IPv6 jumbogram payload length is stored into a 32bit field, but cap the max +* MTU to INT_MAX because MTUs and packet lengths are sometimes represented as +* signed ints. +*/ +#define IP6_MAX_JUMBOGRAM_MTU INT_MAX +static inline int inet6_iif (const struct sk_buff *skb) {bool l3_slave =ipv6_l3mdev_skb (IP6CB (skb)->flags);
include/net/ip6_route.h +3 -2
@@ -38,8 +38,9 @@ struct route_info {#define RT6_LOOKUP_F_IGNORE_LINKSTATE 0x00000040#define RT6_LOOKUP_F_DST_NOREF 0x00000080 -/* We do not (yet ?) support IPv6 jumbograms (RFC 2675) -* Unlike IPv4, hdr->seg_len doesn't include the IPv6 header +/* We do not (yet ?) fully support IPv6 jumbograms (RFC 2675) for all protocols. +* Where jumbograms are supported, IP6_MAX_JUMBOGRAM_MTU should be used instead. +* Unlike IPv4, hdr->seg_len doesn't include the IPv6 header. */ #define IP6_MAX_MTU (0xFFFF + sizeof(struct ipv6hdr))
net/ipv6/route.c +1 -1
@@ -1656,7 +1656,7 @@ static unsigned int fib6_mtu(const struct fib6_result *res)rcu_read_unlock (); } - mtu = min_t(unsigned int, mtu, IP6_MAX_MTU); + mtu = min_t(unsigned int, mtu, IP6_MAX_JUMBOGRAM_MTU);return mtu -lwtunnel_headroom (nh->fib_nh_lws, mtu); }
Signature · 1 line
2.47.3
PATCH 3/10 ipv6: add jumbo payload option to non-gso jumbograms
In reply to Mariusz KlimekThis path reintroduces code recently removed by Alice in commit 741d069aa488 ("net/ipv6: Drop HBH for BIG TCP on TX side") and commit 35f66ce90037 ("net/ipv6: Remove HBH helpers") that adds a hop-by-hop header containing a jumbo-payload option to packets larger than 65535 bytes. The hop-by-hop header is only added to jumbograms that aren't BIG TCP packets since BIG TCP packets no longer need to contain such hop-by-hop headers.
This fixes sending jumbograms to a loopback device with MTU > 65535.
--- include/net/ipv6.h | 11 +++++++++++ net/ipv6/ip6_output.c | 22 ++++++++++++++++++++-- 2 files changed, 31 insertions(+), 2 deletions(-)
include/net/ipv6.h +11 -0
@@ -152,6 +152,17 @@ struct frag_hdr {__be32 identification; }; +/* +* Jumbo payload option, as described in RFC 2675 2. +*/ +struct hop_jumbo_hdr { +u8 nexthdr; +u8 hdrlen; +u8 tlv_type;/* IPV6_TLV_JUMBO, 0xC2 */ +u8 tlv_len;/* 4 */ +__be32 jumbo_payload_len; +}; +#define IP6_MF 0x0001#define IP6_OFFSET 0xFFF8
net/ipv6/ip6_output.c +20 -2
@@ -293,6 +293,8 @@ int ip6_xmit(const struct sock *sk, struct sk_buff *skb, struct flowi6 *fl6,struct in6_addr *first_hop = &fl6->daddr;struct dst_entry *dst =skb_dst (skb);struct inet6_dev *idev =ip6_dst_idev (dst); +struct hop_jumbo_hdr *hop_jumbo; +int hoplen =sizeof (*hop_jumbo);struct net *net =sock_net (sk);unsigned int head_room;struct net_device *dev;
@@ -305,7 +307,7 @@ int ip6_xmit(const struct sock *sk, struct sk_buff *skb, struct flowi6 *fl6,rcu_read_lock (); dev =dst_dev_rcu (dst); - head_room =sizeof (struct ipv6hdr ) +LL_RESERVED_SPACE (dev); + head_room =sizeof (struct ipv6hdr ) + hoplen +LL_RESERVED_SPACE (dev);if (opt) head_room += opt->opt_nflen + opt->opt_flen;
@@ -331,8 +333,24 @@ int ip6_xmit(const struct sock *sk, struct sk_buff *skb, struct flowi6 *fl6, &fl6->saddr); } -if (unlikely (seg_len >IPV6_MAXPLEN )) +if (unlikely (seg_len >IPV6_MAXPLEN )) { +/* Only "real" jumbograms require a jumbo-payload option. +* BIG TCP packets just rely on skb->len. +*/ +if (!skb_is_gso (skb)) { + hop_jumbo =__skb_push (skb, hoplen); + + hop_jumbo->nexthdr = proto; + hop_jumbo->hdrlen =0 ; + hop_jumbo->tlv_type =IPV6_TLV_JUMBO ; + hop_jumbo->tlv_len =4 ; + hop_jumbo->jumbo_payload_len =htonl (seg_len + hoplen); + + proto =IPPROTO_HOPOPTS ; + } + seg_len =0 ; + }__skb_push (skb,sizeof (struct ipv6hdr ));skb_reset_network_header (skb);
Signature · 1 line
2.47.3
PATCH 4/10 tcp: decouple TSO segment length from MSS
In reply to Mariusz KlimekThis patch decouples the TSO segment length from the MSS to allow for MSS > 65535 despite the TSO segment length being capped to 16 bits. Ideally TSO/GSO would support jumbogram segments so that the decoupling isn't necessary, but that would require a much bigger change.
Add a new helper function tcp_tso_seglen that returns the segment length for a given MSS, capped at 65535 - MAX_TCP_HEADER, and use it where the MSS is treated as the segment length. This leaves enough room for TCP/IPv6 headers, including TCP options and extension headers.
Change the signatures of some functions to accept max_len instead of segs where segs is only used to calculate the maximum length of a TSO packet.
--- include/net/tcp.h | 12 ++++++-- net/ipv4/tcp.c | 10 ++++--- net/ipv4/tcp_output.c | 67 +++++++++++++++++++++++++------------------ net/ipv4/tcp_timer.c | 4 +-- 4 files changed, 57 insertions(+), 36 deletions(-)
include/net/tcp.h +10 -2
@@ -747,8 +747,8 @@ void tcp_skb_entail(struct sock *sk, struct sk_buff *skb);void tcp_mark_push (struct tcp_sock *tp,struct sk_buff *skb);void __tcp_push_pending_frames (struct sock *sk,unsigned int cur_mss,int nonagle); -int __tcp_retransmit_skb (struct sock *sk,struct sk_buff *skb,int segs); -int tcp_retransmit_skb (struct sock *sk,struct sk_buff *skb,int segs); +int __tcp_retransmit_skb (struct sock *sk,struct sk_buff *skb,int max_len); +int tcp_retransmit_skb (struct sock *sk,struct sk_buff *skb,int max_len);void tcp_retransmit_timer (struct sock *sk);void tcp_xmit_retransmit_queue (struct sock *);void tcp_simple_retransmit (struct sock *);
@@ -1219,6 +1219,14 @@ static inline void tcp_skb_pcount_add(struct sk_buff *skb, int segs)TCP_SKB_CB (skb)->tcp_gso_segs += segs; } +/* Return the segment length we want for the given MSS. We cap the segment +* length to prevent the segments from becoming jumbograms. +*/ +static inline u16 tcp_tso_seglen (u32 mss_now) +{ +return min_t (u32,GSO_BY_FRAGS -MAX_TCP_HEADER , mss_now); +} +/* This is valid iff skb is in write queue and tcp_skb_pcount() > 1. */ static inline int tcp_skb_mss (const struct sk_buff *skb) {
net/ipv4/tcp.c +6 -4
@@ -960,6 +960,7 @@ static unsigned int tcp_xmit_size_goal(struct sock *sk, u32 mss_now, {struct tcp_sock *tp =tcp_sk (sk);u32 new_size_goal, size_goal; +u16 gso_size;if (!large_allowed)return mss_now;
@@ -968,12 +969,13 @@ static unsigned int tcp_xmit_size_goal(struct sock *sk, u32 mss_now, new_size_goal =tcp_bound_to_half_wnd (tp, sk->sk_gso_max_size);/* We try hard to avoid divides here */ - size_goal = tp->gso_segs * mss_now; + gso_size =tcp_tso_seglen (mss_now); + size_goal = tp->gso_segs * gso_size;if (unlikely (new_size_goal < size_goal || - new_size_goal >= size_goal + mss_now)) { - tp->gso_segs =min_t (u16, new_size_goal / mss_now, + new_size_goal >= size_goal + gso_size)) { + tp->gso_segs =min_t (u16, new_size_goal / gso_size, sk->sk_gso_max_segs); - size_goal = tp->gso_segs * mss_now; + size_goal = tp->gso_segs * gso_size; }return max (size_goal, mss_now);
net/ipv4/tcp_output.c +39 -28
234 lines not on this page · raw message
net/ipv4/tcp_timer.c +2 -2
@@ -595,7 +595,7 @@ void tcp_retransmit_timer(struct sock *sk) goto out; }tcp_enter_loss (sk); -tcp_retransmit_skb (sk, skb,1 ); +tcp_retransmit_skb (sk, skb,tcp_current_mss (sk));__sk_dst_reset (sk); goto out_reset_timer; }
@@ -628,7 +628,7 @@ void tcp_retransmit_timer(struct sock *sk)tcp_enter_loss (sk);tcp_update_rto_stats (sk); -if (tcp_retransmit_skb (sk,tcp_rtx_queue_head (sk),1 ) >0 ) { +if (tcp_retransmit_skb (sk,tcp_rtx_queue_head (sk),tcp_current_mss (sk)) >0 ) {/* Retransmission failed because of local congestion, * Let senders fight for local resources conservatively. */
Signature · 1 line
2.47.3
PATCH 5/10 tcp: split jumbograms with urgent pointer correctly
In reply to Mariusz KlimekThis patch fixes urgent-pointer handling for TCP jumbograms. According to RFC2674, if the urgent pointer offset from the seq number is greater than 65535 but less than the length of the TCP data, the packet should be split into at least two pieces, with a new packet starting at the urgent offset.
Though the optimal solution is to split the packet into exactly two packets, not sending jumbograms in the first place is a much simpler implementation which only requires capping the size returned from tcp_xmit_size_goal when large_allowed is false. Considering that urgent pointer is rarely used with jumbograms, this solution should suffice.
--- net/ipv4/tcp.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
net/ipv4/tcp.c +1 -1
@@ -963,7 +963,7 @@ static unsigned int tcp_xmit_size_goal(struct sock *sk, u32 mss_now,u16 gso_size;if (!large_allowed) -return mss_now; +return min (IPV6_MAXPLEN , mss_now);/* Note : tcp_tso_autosize() will eventually split this later */ new_size_goal =tcp_bound_to_half_wnd (tp, sk->sk_gso_max_size);
Signature · 1 line
2.47.3
PATCH 6/10 tcp: set MSS correctly for PMTU above 65535
In reply to Mariusz KlimekThis patch fixes __tcp_mtu_to_mss to support MTUs above 65535. According to RFC2675 and RFC6691, an advertised MSS of 65535 means "any MSS, rely only on PMTU discovery". ip6_default_advmss already adheres to this behavior, but __tcp_mtu_to_mss doesn't. __tcp_mtu_to_mss instead clamps the MSS to 65535.
Note that MTU probing also doesn't currently support PMTU > 65535 because icsk_mtup.search_high is set to 65535. This commit doesn't add support for PMTU > 65535 when MTU probing is enabled because it is not obvious what icsk_mtup.search_high should be. A possible solution is to only set icsk_mtup.search_low and to exponentially raise it until a probe becomes too big (and then set icsk_mtup.search_high) but since this is a non-trivial change, delegate it to a separate patch series.
--- net/ipv4/tcp_output.c | 17 ++++++++++++++--- 1 file changed, 14 insertions(+), 3 deletions(-)
net/ipv4/tcp_output.c +14 -3
@@ -1990,19 +1990,30 @@ static inline int __tcp_mtu_to_mss(struct sock *sk, int pmtu) {const struct tcp_sock *tp =tcp_sk (sk);const struct inet_connection_sock *icsk =inet_csk (sk); +unsigned int ext_hdr_len;int mss_now; + ext_hdr_len = icsk->icsk_ext_hdr_len; + +/* Take into account added jumbogram HBH header. */ +if (unlikely (pmtu -sizeof (struct ipv6hdr ) >IPV6_MAXPLEN )) + ext_hdr_len +=sizeof (struct hop_jumbo_hdr ); +/* Calculate base mss without TCP options: It is MMS_S - sizeof(tcphdr) of rfc1122 */ mss_now = pmtu - icsk->icsk_af_ops->net_header_len -sizeof (struct tcphdr ); -/* Clamp it (mss_clamp does not include tcp options) */ -if (mss_now > tp->rx_opt.mss_clamp) +/* Clamp it (mss_clamp does not include tcp options). +* An mss of 65535 means we should rely entirely on PMTU discovery +* (RFC2675, RFC6691). +*/ +if (mss_now > tp->rx_opt.mss_clamp && +likely (tp->rx_opt.mss_clamp <IPV6_MAXPLEN )) mss_now = tp->rx_opt.mss_clamp;/* Now subtract optional transport overhead */ - mss_now -= icsk->icsk_ext_hdr_len; + mss_now -= ext_hdr_len;/* Then reserve room for full set of TCP options and 8 bytes of data */ mss_now = max(mss_now,
Signature · 1 line
2.47.3
PATCH 7/10 veth: raise the max MTU above 65535
In reply to Mariusz KlimekThis patch raises the maximum allowed MTU above 65535 to allow IPv6 jumbograms to pass through veth pairs. Raising the MTU above 65535 can significantly improve throughput between connected namespaces. This is particularly useful for docker containers, which are connected to the host through veth pairs. This also serves as a way to test jumbogram handling in the kernel.
--- drivers/net/veth.c | 8 +++++--- 1 file changed, 5 insertions(+), 3 deletions(-)
drivers/net/veth.c +5 -3
@@ -1425,9 +1425,11 @@ static int veth_close(struct net_device *dev)return 0 ; } +#define MAX_MTU IP6_MAX_JUMBOGRAM_MTU +static int is_valid_veth_mtu (int mtu) { -return mtu >=ETH_MIN_MTU && mtu <=ETH_MAX_MTU ; +return mtu >=ETH_MIN_MTU && mtu <=MAX_MTU ; }static int veth_alloc_queues (struct net_device *dev)
@@ -1628,7 +1630,7 @@ static int veth_xdp_set(struct net_device *dev, struct bpf_prog *prog,if (peer) { peer->hw_features |=NETIF_F_GSO_SOFTWARE ; - peer->max_mtu =ETH_MAX_MTU ; + peer->max_mtu =MAX_MTU ; } }bpf_prog_put (old_prog);
@@ -1754,7 +1756,7 @@ static void veth_setup(struct net_device *dev) dev->needs_free_netdev = true; dev->priv_destructor = veth_dev_free; dev->pcpu_stat_type =NETDEV_PCPU_STAT_TSTATS ; - dev->max_mtu =ETH_MAX_MTU ; + dev->max_mtu =MAX_MTU ; dev->hw_features =VETH_FEATURES ; dev->hw_enc_features =VETH_FEATURES ;
Signature · 1 line
2.47.3
PATCH 8/10 selftests/net: test sending TCP jumbograms over veth
In reply to Mariusz KlimekThis patch adds a selftest that tests TCP connections over large MTUs. The test uses three network namespaces called NS_CLIENT, NS_ROUTER and NS_SERVER. Messages are sent from NS_CLIENT to NS_SERVER through NS_ROUTER, which are connected through veth pairs. The test verifies that the messages are properly sent and received and that jumbograms are received if the MTUs allow it.
The jumbogram_tx and jumbogram_rx helper programs are used to send and receive large TCP messages. The programs are loosely based on udpgso_bench_tx and udpgso_bench_rx.
The jumbogram.bpf.c bpf program is used to keep track of the largest packet size received during the connection. If jumbograms are expected to be delivered, the largest observed packet size is checked to be above the expected size. We only check that at least one packet was a jumbogram because packets received early in the connection may have been smaller due to the small initial window size.
This test is loosely based on big_tcp.sh. The jumbogram.sh test is separate from big_tcp.sh because they are technically different features, and big_tcp.sh would require heavy modification anyway to properly test non-GSO jumbograms.
--- tools/testing/selftests/net/Makefile | 3 + tools/testing/selftests/net/jumbogram.bpf.c | 36 ++ tools/testing/selftests/net/jumbogram.sh | 380 ++++++++++++++++++++ tools/testing/selftests/net/jumbogram_rx.c | 199 ++++++++++ tools/testing/selftests/net/jumbogram_tx.c | 139 +++++++ 5 files changed, 757 insertions(+) create mode 100644 tools/testing/selftests/net/jumbogram.bpf.c create mode 100755 tools/testing/selftests/net/jumbogram.sh create mode 100644 tools/testing/selftests/net/jumbogram_rx.c create mode 100644 tools/testing/selftests/net/jumbogram_tx.c
tools/testing/selftests/net/Makefile +3 -0
@@ -53,6 +53,7 @@ TEST_PROGS := \ ipv6_force_forwarding.sh \ ipv6_route_update_soft_lockup.sh \ ipvtap_test.sh \ + jumbogram.sh \ l2_tos_ttl_inherit.sh \ l2tp.sh \ link_netns.py \
@@ -146,6 +147,8 @@ TEST_GEN_FILES := \ ipsec \ ipv6_flowlabel \ ipv6_flowlabel_mgr \ + jumbogram_rx \ + jumbogram_tx \ msg_zerocopy \ nettest \ psock_fanout \
tools/testing/selftests/net/jumbogram.bpf.c new file +36 -0
@@ -0,0 +1,36 @@ +// SPDX-License-Identifier: GPL-2.0-only + +#include <linux/bpf.h> +#include <linux/pkt_cls.h> +#include <bpf/bpf_helpers.h> + +struct { + __uint(type, BPF_MAP_TYPE_ARRAY); + __uint(max_entries, 1); + __type(key, __u32); + __type(value, __u32); +} max_packet_size SEC(".maps"); + +SEC("ingress") +int track_max_size (struct __sk_buff *skb) +{ +__u32 *max_size_ptr, *count; +__u32 max_size; +__u32 key =0 ; + + max_size_ptr =bpf_map_lookup_elem (&max_packet_size, &key); +if (max_size_ptr) + max_size = *max_size_ptr; +else + max_size =0 ; + +if (skb->len >= max_size) { + max_size = skb->len; +bpf_map_update_elem (&max_packet_size, &key, &max_size, +BPF_ANY ); + } + +return TC_ACT_OK ; +} + +char _license[] SEC("license") = ("GPL");
tools/testing/selftests/net/jumbogram.sh new file +380 -0
380 lines not on this page · raw message
tools/testing/selftests/net/jumbogram_rx.c new file +199 -0
199 lines not on this page · raw message
tools/testing/selftests/net/jumbogram_tx.c new file +139 -0
139 lines not on this page · raw message
Signature · 1 line
2.47.3
PATCH 9/10 selftests/net: add test cases with MTU above 65535 to big_tcp.sh
In reply to Mariusz KlimekThis patch adds test cases to big_tcp.sh that test BIG TCP over MTUs above 65535. Two test cases are added: gso_max_size > mtu > 65535 and mtu > gso_max_size > 65535.
--- tools/testing/selftests/net/big_tcp.sh | 24 ++++++++++++++++-------- 1 file changed, 16 insertions(+), 8 deletions(-)
tools/testing/selftests/net/big_tcp.sh +16 -8
@@ -132,8 +132,14 @@ do_test() { local gw_gro=$2 local gw_tso=$3 local ser_gro=$4 + local mtu=$5 local ret="PASS" + ip -n $CLIENT_NS link set link0 mtu "$mtu" + ip -n $ROUTER_NS link set link1 mtu "$mtu" + ip -n $ROUTER_NS link set link2 mtu "$mtu" + ip -n $SERVER_NS link set link3 mtu "$mtu" + ip net exec $CLIENT_NS ethtool -K link0 tso $cli_tso ip net exec $ROUTER_NS ethtool -K link1 gro $gw_gro ip net exec $ROUTER_NS ethtool -K link2 tso $gw_tso
@@ -151,18 +157,20 @@ do_test() { stop_counter link1 $ROUTER_NS stop_counter link3 $SERVER_NS - printf "%-9s %-8s %-8s %-8s: [%s]\n" \ - $cli_tso $gw_gro $gw_tso $ser_gro $ret + printf "%-9s %-8s %-8s %-9s %-7s: [%s]\n" \ + $cli_tso $gw_gro $gw_tso $ser_gro $mtu $ret test $ret = "PASS" } testup() { - echo "CLI GSO | GW GRO | GW GSO | SER GRO" && \ - do_test "on" "on" "on" "on" && \ - do_test "on" "off" "on" "off" && \ - do_test "off" "on" "on" "on" && \ - do_test "on" "on" "off" "on" && \ - do_test "off" "on" "off" "on" + echo "CLI GSO | GW GRO | GW GSO | SER GRO | MTU" && \ + do_test "on" "on" "on" "on" 1500 && \ + do_test "on" "off" "on" "off" 1500 && \ + do_test "off" "on" "on" "on" 1500 && \ + do_test "on" "on" "off" "on" 1500 && \ + do_test "off" "on" "off" "on" 1500 && \ + do_test "on" "off" "on" "off" 66000 && \ + do_test "on" "off" "on" "off" 200000 } if ! netperf -V &> /dev/null; then
Signature · 1 line
2.47.3
PATCH 10/10 selftests/net: add jumbogram test case to msg_zerocopy.sh
In reply to Mariusz KlimekThis patch adds a test case to msg_zerocopy.sh that tests whether zerocopy works with TCP jumbograms.
--- tools/testing/selftests/net/msg_zerocopy.c | 3 ++- tools/testing/selftests/net/msg_zerocopy.sh | 3 ++- 2 files changed, 4 insertions(+), 2 deletions(-)
tools/testing/selftests/net/msg_zerocopy.c +2 -1
@@ -789,7 +789,8 @@ static void parse_opts(int argc, char **argv)setup_sockaddr (cfg_family, daddr, &cfg_dst_addr);setup_sockaddr (cfg_family, saddr, &cfg_src_addr); -if (cfg_payload_len > max_payload_len) +if (cfg_payload_len > max_payload_len && + (strcmp (cfg_test,"tcp" ) !=0 || cfg_family !=PF_INET6 ))error (1 ,0 ,"-s: payload exceeds max (%d)" , max_payload_len);if (cfg_cork_mixed && (!cfg_zerocopy || !cfg_cork))error (1 ,0 ,"-m: cork_mixed requires corking and zerocopy" );
tools/testing/selftests/net/msg_zerocopy.sh +2 -1
@@ -7,7 +7,7 @@ set -e readonly DEV="veth0" readonly DUMMY_DEV="dummy0" -readonly DEV_MTU=65535 +readonly DEV_MTU=200000 readonly BIN="./msg_zerocopy" readonly RAND="$(mktemp -u XXXXXX)"
@@ -29,6 +29,7 @@ if [[ "$#" -eq "0" ]]; then $0 4 tcp -t 1 || ret=1 $0 6 tcp -t 1 || ret=1 + $0 6 tcp -t 1 -s 100000 || ret=1 $0 4 udp -t 1 || ret=1 $0 6 udp -t 1 || ret=1
Signature · 1 line
2.47.3
Quoted 30 lines · Mariusz Klimek
This series adds support for sending TCP jumbograms over MTUs above 65535, and adds support for setting such MTUs on veth devices.
The TCP stack is already capable of receiving jumbograms and (up until recently) sending GSO-jumbograms that get segmented before exiting the host. However, the TCP stack doesn't support sending regular jumbograms over high MTUs as described in RFC2675. Specifically, the TCP stack doesn't support an MSS greater than 65535.
No device currently supports setting an MTU greater than 65535 but there are some devices that could benefit from higher MTUs and that can theoretically support it. For example, IPoIB in connected-mode can support MTUs up to 2^31. Virtual devices such as veth have no physical limitations and therefore could also support such MTUs.
This series only adds support for setting high MTUs on veth devices, with support for other devices being delegated to future patch series. Allowing jumbograms to pass through veth devices is useful for testing jumbogram functionality, and also increases throughput when compared to BIG TCP (see the benchmark numbers below).
In addition to removing the upper limit on veth MTUs, This series adds the missing pieces that allow the TCP stack to send TCP jumbograms. This includes a bit of code removed recently by Alice's patch series [1]. Most of the patches in this series are rather trivial. The main problem addressed by this series is that the TCP stack conflates the TSO segment length with the MSS, but the TSO segment length is a 16-bit integer, which doesn't allow for MSS > 65535. This series decouples them.
Quoted 6 lines
Running iperf over veth on an Intel Core i9-12900K CPU shows a 5% improvement in throughput.
Commands:
iperf3 -s
iperf3 -c -l 500KiB ${SERVER_IP}%veth0
You are adding code to TCP fast path, for a narrow (and possibly dangerous) use case.
I am sorry, I will not review this patch series coming from a first-timer.
Quoted 34 lines · Eric Dumazet
Quoted 33 lines · Mariusz Klimek
This series adds support for sending TCP jumbograms over MTUs above 65535, and adds support for setting such MTUs on veth devices.
The TCP stack is already capable of receiving jumbograms and (up until recently) sending GSO-jumbograms that get segmented before exiting the host. However, the TCP stack doesn't support sending regular jumbograms over high MTUs as described in RFC2675. Specifically, the TCP stack doesn't support an MSS greater than 65535.
No device currently supports setting an MTU greater than 65535 but there are some devices that could benefit from higher MTUs and that can theoretically support it. For example, IPoIB in connected-mode can support MTUs up to 2^31. Virtual devices such as veth have no physical limitations and therefore could also support such MTUs.
This series only adds support for setting high MTUs on veth devices, with support for other devices being delegated to future patch series. Allowing jumbograms to pass through veth devices is useful for testing jumbogram functionality, and also increases throughput when compared to BIG TCP (see the benchmark numbers below).
In addition to removing the upper limit on veth MTUs, This series adds the missing pieces that allow the TCP stack to send TCP jumbograms. This includes a bit of code removed recently by Alice's patch series [1]. Most of the patches in this series are rather trivial. The main problem addressed by this series is that the TCP stack conflates the TSO segment length with the MSS, but the TSO segment length is a 16-bit integer, which doesn't allow for MSS > 65535. This series decouples them.
Running iperf over veth on an Intel Core i9-12900K CPU shows a 5% improvement in throughput.
Quoted 8 lines
Quoted 3 lines
Commands:
iperf3 -s
iperf3 -c -l 500KiB ${SERVER_IP}%veth0You are adding code to TCP fast path, for a narrow (and possibly dangerous) use case.
I am sorry, I will not review this patch series coming from a first-timer.
I understand your concerns and they're reasonable. I would still like to proceed with this series, though. What can I do for you to consider this series? Would it help to submit a few less-significant patches first before resubmitting this patch series? Should I resubmit it as an RFC?
This feature is opt-in, so even though I'm adding code to the TCP fastpath, it should be relatively straightforward to check that it doesn't introduce any bugs in the regular MTU < 65535 flow.
Signature · 1 line
Mariusz K.
Jakub Kicinski
maintainerQuoted 4 lines · On Tue, 9 Jun 2026 19:01:02 +0200 Mariusz Klimek wrote:
I understand your concerns and they're reasonable. I would still like to proceed with this series, though. What can I do for you to consider this series? Would it help to submit a few less-significant patches first before resubmitting this patch series? Should I resubmit it as an RFC?
AI tools changed how much we value code, but they did not change much about how one joins a community.
Quoted 8 lines · Jakub Kicinski
Quoted 4 lines · On Tue, 9 Jun 2026 19:01:02 +0200 Mariusz Klimek wrote:
I understand your concerns and they're reasonable. I would still like to proceed with this series, though. What can I do for you to consider this series? Would it help to submit a few less-significant patches first before resubmitting this patch series? Should I resubmit it as an RFC?
AI tools changed how much we value code, but they did not change much about how one joins a community.
OK, I'll start with smaller patches to establish myself more in the community. I am still interested in eventually getting this series through as it can speed up workloads relevant to our company. Do you think this patch could potentially be considered down the line?
Signature · 1 line
Mariusz K.
Quoted 8 lines · Mariusz Klimek
Quoted 6 lines · Jakub Kicinski
Quoted 4 lines · On Tue, 9 Jun 2026 19:01:02 +0200 Mariusz Klimek wrote:
I understand your concerns and they're reasonable. I would still like to proceed with this series, though. What can I do for you to consider this series? Would it help to submit a few less-significant patches first before resubmitting this patch series? Should I resubmit it as an RFC?
Quoted 7 lines
Quoted 2 lines
AI tools changed how much we value code, but they did not change much about how one joins a community.
OK, I'll start with smaller patches to establish myself more in the community. I am still interested in eventually getting this series through as it can speed up workloads relevant to our company. Do you think this patch could potentially be considered down the line?
Too many hidden assumptions in our code about MSS <= 65535.
I have more urgent work on my plate than making sure nothing breaks.
The opt-in argument will not fly, we will have many security reports because of CLONE_NEWUSER | CLONE_NEWNS abuses.
Paolo Abeni
maintainerQuoted 7 lines · Mariusz Klimek
Quoted 6 lines · Jakub Kicinski
Quoted 4 lines · On Tue, 9 Jun 2026 19:01:02 +0200 Mariusz Klimek wrote:
I understand your concerns and they're reasonable. I would still like to proceed with this series, though. What can I do for you to consider this series? Would it help to submit a few less-significant patches first before resubmitting this patch series? Should I resubmit it as an RFC?
Quoted 7 lines
Quoted 2 lines
AI tools changed how much we value code, but they did not change much about how one joins a community.
OK, I'll start with smaller patches to establish myself more in the community. I am still interested in eventually getting this series through as it can speed up workloads relevant to our company. Do you think this patch could potentially be considered down the line?
If you are strongly motivated to do progresses in this area, one thing you could look at is the root cause of the performance difference between the veth jumbogram and the same size veth big tcp test: I'm quite surprised some delta is visible there.
Possibly perf can outline some bottle-neck in the big tcp path that could be ironed-out.
Also it could be interesting if some devices could do H/W GRO based big TCP.
/P
Quoted 18 lines · Paolo Abeni
Quoted 6 lines · Mariusz Klimek
Quoted 5 lines · Jakub Kicinski
Quoted 4 lines · On Tue, 9 Jun 2026 19:01:02 +0200 Mariusz Klimek wrote:
I understand your concerns and they're reasonable. I would still like to proceed with this series, though. What can I do for you to consider this series? Would it help to submit a few less-significant patches first before resubmitting this patch series? Should I resubmit it as an RFC?
Quoted 6 lines
Quoted 2 lines
AI tools changed how much we value code, but they did not change much about how one joins a community.
OK, I'll start with smaller patches to establish myself more in the community. I am still interested in eventually getting this series through as it can speed up workloads relevant to our company. Do you think this patch could potentially be considered down the line?
If you are strongly motivated to do progresses in this area, one thing you could look at is the root cause of the performance difference between the veth jumbogram and the same size veth big tcp test: I'm quite surprised some delta is visible there.
Quoted 7 lines
Possibly perf can outline some bottle-neck in the big tcp path that could be ironed-out.
Also it could be interesting if some devices could do H/W GRO based big TCP.
/P
Hi Paolo, it's been a long time.
After some research I found out why jumbograms outperform BIG TCP in the iperf3 over veth test: it's because Nagle splits BIG TCP packets but not jumbograms.
BIG TCP assumes packets will eventually be sent over the network, and so applies the Nagle test over the real MSS of 1500 bytes. If a send occurs before the last ACK is received, Nagle splits the BIG TCP packet into two packets: one whose size is a multiple of the MSS, and one that contains the remainder (tcp_mss_split_point and tso_fragment). The remainder packet is queued and is only sent once the ACK is received.
In the jumbogram case, the jumbogram's size is already smaller than or equal to the MSS size, so Nagle doesn't split the packet. The ~5% performance gain comes from avoiding the packet being split in two and having one packet traverse the network stack instead of two.
A simple way to prove the above is disabling Nagle with the TCP_NODELAY flag and seeing that the performance gap resolves. I also ran iperf3 on a kernel with the limit >= max_segs * tp->mss_cache, limit >= chunk, and limit > tcp_max_tso_deferred_mss(tp) tp->mss_cache checks removed from tcp_tso_should_defer, and it also resolved the performance gap. Perhaps tcp_tso_should_defer should be fixed to so that these BIG TCP packets aren't split?
While debugging this issue I tried restricting the iperf3 client and server processes to one CPU. That reduced the number of packets that were split to around 10% (from 60%), but surprisingly the throughput also increased significantly and was much larger than the other tests. It appears that restricting the iperf3 client and server to the same CPU fixes something else and provides a 12% improvement over the base case and a ~7% improvement over jumbograms. I can't explain this improvement yet, but the reduction in packet splitting is likely due to less racy execution that makes the Nagle case happen less frequently.
Measurements (output of a python wrapper around iperf3):
[ROUNDS=84 MTU=1500 GSO_SIZE=524280]
IPERF3 BITRATE: 241.35 [95.0% CI 239.81 - 242.91] ±7.17 Gbits/sec
SERVER RX PACKETS SZ=862: 406030.32 [95.0% CI 374158.02 - 437902.63] ±144141.51 SERVER RX PACKETS SZ=511310: 401084.13 [95.0% CI 368103.20 - 434065.07] ±150101.57 SERVER RX PACKETS SZ=512086: 197816.40 [95.0% CI 162980.59 - 232652.22] ±160523.86
CLIENT RX PACKETS SZ=86: 647034.43 [95.0% CI 636767.16 - 657301.70] ±47311.70
[ROUNDS=284 MTU=524280 GSO_OFF]
IPERF3 BITRATE: 252.56 [95.0% CI 251.67 - 253.44] ±7.55 Gbits/sec
SERVER RX PACKETS SZ=512094: 616549.29 [95.0% CI 614399.99 - 618698.58] ±18401.15
CLIENT RX PACKETS SZ=86: 616754.45 [95.0% CI 614599.64 - 618909.26] ±18448.43
[ROUNDS=84 MTU=1500 GSO_SIZE=524280 NODELAY]
IPERF3 BITRATE: 252.96 [95.0% CI 251.36 - 254.55] ±7.36 Gbits/sec
SERVER RX PACKETS SZ=512086: 617684.17 [95.0% CI 613782.72 - 621585.61] ±17977.91
CLIENT RX PACKETS SZ=86: 617721.74 [95.0% CI 613817.65 - 621625.83] ±17990.11
[ROUNDS=84 MTU=524280 GSO_OFF NODELAY]
IPERF3 BITRATE: 252.21 [95.0% CI 250.54 - 253.88] ±7.69 Gbits/sec
SERVER RX PACKETS SZ=512094: 615726.77 [95.0% CI 611620.41 - 619833.14] ±18922.17
CLIENT RX PACKETS SZ=86: 615912.33 [95.0% CI 611836.51 - 619988.15] ±18781.42
[ROUNDS=125 MTU=1500 GSO_SIZE=524280 NCPUS=1]
IPERF3 BITRATE: 272.80 [95.0% CI 272.37 - 273.23] ±2.42 Gbits/sec
SERVER RX PACKETS SZ=511310: 6074.89 [95.0% CI 5988.63 - 6161.14] ±487.24 SERVER RX PACKETS SZ=512086: 652866.16 [95.0% CI 651750.82 - 653981.50] ±6300.23 SERVER RX PACKETS SZ=512738: 6454.82 [95.0% CI 6355.94 - 6553.71] ±558.58
CLIENT RX PACKETS SZ=86: 655050.50 [95.0% CI 653940.67 - 656160.32] ±6269.06
[ROUNDS=64 MTU=1500 GSO_SIZE=524280 DELAY_PACKETS_UNTIL_ACK] IPERF3 BITRATE: 251.64 [95.0% CI 250.45 - 252.83] ±4.78 Gbits/sec
SERVER RX PACKETS SZ=512086: 614318.41 [95.0% CI 611402.89 - 617233.93] ±11671.76
CLIENT RX PACKETS SZ=86: 614633.78 [95.0% CI 611697.43 - 617570.13] ±11755.14
Signature · 1 line
Mariusz K.