Skip to content

tcp: BIG TCP implementation

Eric Dumazet netdev 8 versions – Applied

Status from this message: This series was applied to netdev/net-next.git (master)

Get series with b4
b4 am [email protected]

Versions newest first · linked by author and title

  1. v7 Latest13 patches 0 repliesApplied 13/13
  2. v613 patches 9 repliesSuperseded
  3. v513 patches 9 repliesSuperseded
Earlier versions · 5
VersionPostedPatchesRepliesStatus
v4 posting 2/212 patches27 repliesSuperseded
v4 posting 1/214 patches12 repliesSuperseded
v314 patches3 repliesSuperseded
v214 patches21 repliesSuperseded
v115 patches38 repliesSuperseded

Patches in v7 All 13 applied

#Subject and filesSizeRepliesApplied
1/13net: add IFLA_TSO_{MAX_SIZE|SEGS} attributesinclude/uapi/linux/if_link.h net/core/rtnetlink.c tools/include/uapi/linux/if_link.h+10 -00 repliesApplied
2/13net: allow gso_max_size to exceed 65536drivers/net/ethernet/amd/xgbe/xgbe.h drivers/net/ethernet/mellanox/mlx5/core/en_rx.c drivers/net/ethernet/sfc/ef100_nic.c +13 more+40 -170 repliesApplied
3/13net: limit GSO_MAX_SIZE to 524280 bytesinclude/linux/netdevice.h+5 -20 repliesApplied
4/13tcp_cubic: make hystart_ack_delay() aware of BIG TCPnet/ipv4/tcp_cubic.c+2 -20 repliesApplied
5/13ipv6: add struct hop_jumbo_hdr definitioninclude/net/ipv6.h+11 -00 repliesApplied
6/13ipv6/gso: remove temporary HBH/jumbo headerinclude/net/ipv6.h net/ipv6/ip6_offload.c+56 -10 repliesApplied
7/13ipv6/gro: insert temporary HBH/jumbo headernet/ipv6/ip6_offload.c+30 -20 repliesApplied
8/13net: allow gro_max_size to exceed 65536drivers/net/ethernet/mellanox/mlx5/core/en_rx.c include/linux/netdevice.h include/net/ipv6.h +3 more+16 -120 repliesApplied
9/13ipv6: Add hop-by-hop header to jumbograms in ip6_outputinclude/linux/ipv6.h net/ipv6/ip6_output.c+21 -20 repliesApplied
10/13net: loopback: enable BIG TCP packetsdrivers/net/loopback.c+2 -00 repliesApplied
11/13veth: enable BIG TCP packetsdrivers/net/veth.c+1 -00 repliesApplied
12/13mlx4: support BIG TCP packetsdrivers/net/ethernet/mellanox/mlx4/en_netdev.c drivers/net/ethernet/mellanox/mlx4/en_tx.c+41 -90 repliesApplied
13/13mlx5: support BIG TCP packetsdrivers/net/ethernet/mellanox/mlx5/core/en_main.c drivers/net/ethernet/mellanox/mlx5/core/en_tx.c+89 -230 repliesApplied

Reviews trailers per patch

11 of 13 patches have a Reviewed-by or Acked-by.

PatchAlexander DuyckTariq Toukan
1/13A-b
2/13
3/13A-b
4/13A-b
5/13A-b
6/13A-b
7/13A-b
8/13
9/13A-b
10/13A-b
11/13A-b
12/13A-b R-b
13/13R-b

Trailers given in replies are filled; those carried in the patches from earlier versions are outlined with dashes.

Read the v7 thread

Cover letter [PATCH v7 net-next 00/13] tcp: BIG TCP implementation

From: Eric Dumazet <[email protected]>

This series implements BIG TCP as presented in netdev 0x15:

https://netdevconf.info/0x15/session.html?BIG-TCP

Jonathan Corbet made a nice summary: https://lwn.net/Articles/884104/

Standard TSO/GRO packet limit is 64KB

With BIG TCP, we allow bigger TSO/GRO packet sizes for IPv6 traffic.

Note that this feature is by default not enabled, because it might break some eBPF programs assuming TCP header immediately follows IPv6 header.

While tcpdump recognizes the HBH/Jumbo header, standard pcap filters are unable to skip over IPv6 extension headers.

Reducing number of packets traversing networking stack usually improves performance, as shown on this experiment using a 100Gbit NIC, and 4K MTU.

'Standard' performance with current (74KB) limits.
for i in {1..10}; do ./netperf -t TCP_RR -H iroa23 -- -r80000,80000 -O MIN_LATENCY,P90_LATENCY,P99_LATENCY,THROUGHPUT|tail -1; done

77           138          183          8542.19    
79           143          178          8215.28    
70           117          164          9543.39    
80           144          176          8183.71    
78           126          155          9108.47    
80           146          184          8115.19    
71           113          165          9510.96    
74           113          164          9518.74    
79           137          178          8575.04    
73           111          171          9561.73    

Now enable BIG TCP on both hosts.

ip link set dev eth0 gro_max_size 185000 gso_max_size 185000
for i in {1..10}; do ./netperf -t TCP_RR -H iroa23 -- -r80000,80000 -O MIN_LATENCY,P90_LATENCY,P99_LATENCY,THROUGHPUT|tail -1; done

57           83           117          13871.38   
64           118          155          11432.94   
65           116          148          11507.62   
60           105          136          12645.15   
60           103          135          12760.34   
60           102          134          12832.64   
62           109          132          10877.68   
58           82           115          14052.93   
57           83           124          14212.58   
57           82           119          14196.01   

We see an increase of transactions per second, and lower latencies as well.

v7: adopt unsafe_memcpy() in mlx5 to avoid FORTIFY warnings.

v6: fix a compilation error for CONFIG_IPV6=n in
"net: allow gso_max_size to exceed 65536", reported by kernel bots.

v5: Replaced two patches (that were adding new attributes) with patches
from Alexander Duyck. Idea is to reuse existing gso_max_size/gro_max_size

v4: Rebased on top of Jakub series (Merge branch 'tso-gso-limit-split')
max_tso_size is now family independent.

v3: Fixed a typo in RFC number (Alexander)
Added Reviewed-by: tags from Tariq on mlx4/mlx5 parts.

v2: Removed the MAX_SKB_FRAGS change, this belongs to a different series.
Addressed feedback, for Alexander and nvidia folks.

Alexander Duyck (2):
net: allow gso_max_size to exceed 65536
net: allow gro_max_size to exceed 65536

Coco Li (1):
ipv6: Add hop-by-hop header to jumbograms in ip6_output

Eric Dumazet (10):
net: add IFLA_TSO_{MAX_SIZE|SEGS} attributes
net: limit GSO_MAX_SIZE to 524280 bytes
tcp_cubic: make hystart_ack_delay() aware of BIG TCP
ipv6: add struct hop_jumbo_hdr definition ipv6/gso: remove temporary HBH/jumbo header ipv6/gro: insert temporary HBH/jumbo header
net: loopback: enable BIG TCP packets
veth: enable BIG TCP packets
mlx4: support BIG TCP packets
mlx5: support BIG TCP packets

 drivers/net/ethernet/amd/xgbe/xgbe.h          |   3 +-
 .../net/ethernet/mellanox/mlx4/en_netdev.c    |   3 +
 drivers/net/ethernet/mellanox/mlx4/en_tx.c    |  47 ++++++--
 .../net/ethernet/mellanox/mlx5/core/en_main.c |   1 +
 .../net/ethernet/mellanox/mlx5/core/en_rx.c   |   2 +-
 .../net/ethernet/mellanox/mlx5/core/en_tx.c   | 111 ++++++++++++++----
 drivers/net/ethernet/sfc/ef100_nic.c          |   3 +-
 drivers/net/ethernet/sfc/falcon/tx.c          |   3 +-
 drivers/net/ethernet/sfc/tx_common.c          |   3 +-
 drivers/net/ethernet/synopsys/dwc-xlgmac.h    |   3 +-
 drivers/net/hyperv/rndis_filter.c             |   2 +-
 drivers/net/loopback.c                        |   2 +
 drivers/net/veth.c                            |   1 +
 drivers/scsi/fcoe/fcoe.c                      |   2 +-
 include/linux/ipv6.h                          |   1 +
 include/linux/netdevice.h                     |  15 ++-
 include/net/ipv6.h                            |  44 +++++++
 include/uapi/linux/if_link.h                  |   2 +
 net/bpf/test_run.c                            |   2 +-
 net/core/dev.c                                |   9 +-
 net/core/gro.c                                |   8 ++
 net/core/rtnetlink.c                          |  16 ++-
 net/core/sock.c                               |  14 +++
 net/ipv4/tcp_bbr.c                            |   2 +-
 net/ipv4/tcp_cubic.c                          |   4 +-
 net/ipv4/tcp_output.c                         |   2 +-
 net/ipv6/ip6_offload.c                        |  56 ++++++++-
 net/ipv6/ip6_output.c                         |  22 +++-
 net/sctp/output.c                             |   3 +-
 tools/include/uapi/linux/if_link.h            |   2 +
 30 files changed, 321 insertions(+), 67 deletions(-)
Signature · 1 line
2.36.0.550.gb090851708-goog