tcp: BIG TCP implementation
Subsystems (10)
v4 cover tcp: BIG TCP implementation
From: Eric Dumazet <[email protected]>
This series implements BIG TCP as presented in netdev 0x15:
https://netdevconf.info/0x15/session.html?BIG-TCP
Jonathan Corbet made a nice summary: https://lwn.net/Articles/884104/
Standard TSO/GRO packet limit is 64KB
With BIG TCP, we allow bigger TSO/GRO packet sizes for IPv6 traffic.
Note that this feature is by default not enabled, because it might break some eBPF programs assuming TCP header immediately follows IPv6 header.
While tcpdump recognizes the HBH/Jumbo header, standard pcap filters are unable to skip over IPv6 extension headers.
Reducing number of packets traversing networking stack usually improves performance, as shown on this experiment using a 100Gbit NIC, and 4K MTU.
'Standard' performance with current (74KB) limits.
for i in {1..10}; do ./netperf -t TCP_RR -H iroa23 -- -r80000,80000 -O MIN_LATENCY,P90_LATENCY,P99_LATENCY,THROUGHPUT|tail -1; done
77 138 183 8542.19 79 143 178 8215.28 70 117 164 9543.39 80 144 176 8183.71 78 126 155 9108.47 80 146 184 8115.19 71 113 165 9510.96 74 113 164 9518.74 79 137 178 8575.04 73 111 171 9561.73
Now enable BIG TCP on both hosts.
ip link set dev eth0 gro_ipv6_max_size 185000 gso_ipv6_max_size 185000
for i in {1..10}; do ./netperf -t TCP_RR -H iroa23 -- -r80000,80000 -O MIN_LATENCY,P90_LATENCY,P99_LATENCY,THROUGHPUT|tail -1; done
57 83 117 13871.38 64 118 155 11432.94 65 116 148 11507.62 60 105 136 12645.15 60 103 135 12760.34 60 102 134 12832.64 62 109 132 10877.68 58 82 115 14052.93 57 83 124 14212.58 57 82 119 14196.01
We see an increase of transactions per second, and lower latencies as well.
v4: fix compile error for CONFIG_MLX5_CORE_IPOIB=y in mlx5 (Jakub)
v3: Fixed a typo in RFC number (Alexander)
Added Reviewed-by: tags from Tariq on mlx4/mlx5 parts.
v2: Removed the MAX_SKB_FRAGS change, this belongs to a different series.
Addressed feedback, for Alexander and nvidia folks.
Coco Li (5):
ipv6: add dev->gso_ipv6_max_size
ipv6: add GRO_IPV6_MAX_SIZE
ipv6: Add hop-by-hop header to jumbograms in ip6_output
ipvlan: enable BIG TCP Packets
mlx5: support BIG TCP packets
Eric Dumazet (9):
net: add netdev->tso_ipv6_max_size attribute
tcp_cubic: make hystart_ack_delay() aware of BIG TCP
ipv6: add struct hop_jumbo_hdr definition ipv6/gso: remove temporary HBH/jumbo header ipv6/gro: insert temporary HBH/jumbo header
net: loopback: enable BIG TCP packets
bonding: update dev->tso_ipv6_max_size
macvlan: enable BIG TCP Packets
mlx4: support BIG TCP packets
drivers/net/bonding/bond_main.c | 3 + .../net/ethernet/mellanox/mlx4/en_netdev.c | 3 + drivers/net/ethernet/mellanox/mlx4/en_tx.c | 47 +++++++++-- .../net/ethernet/mellanox/mlx5/core/en_main.c | 1 + .../net/ethernet/mellanox/mlx5/core/en_tx.c | 84 +++++++++++++++---- drivers/net/ipvlan/ipvlan_main.c | 1 + drivers/net/loopback.c | 2 + drivers/net/macvlan.c | 1 + include/linux/ipv6.h | 1 + include/linux/netdevice.h | 32 +++++++ include/net/ipv6.h | 44 ++++++++++ include/uapi/linux/if_link.h | 3 + net/core/dev.c | 4 + net/core/gro.c | 20 ++++- net/core/rtnetlink.c | 33 ++++++++ net/core/sock.c | 6 ++ net/ipv4/tcp_cubic.c | 4 +- net/ipv6/ip6_offload.c | 56 ++++++++++++- net/ipv6/ip6_output.c | 22 ++++- tools/include/uapi/linux/if_link.h | 3 + 20 files changed, 336 insertions(+), 34 deletions(-)
Signature · 1 line
2.35.1.616.g0bdcbb4464-goog
PATCH v4 1/14 net: add netdev->tso_ipv6_max_size attribute
From: Eric Dumazet <[email protected]>
Some NIC (or virtual devices) are LSOv2 compatible.
BIG TCP plans using the large LSOv2 feature for IPv6.
New netlink attribute IFLA_TSO_IPV6_MAX_SIZE is defined.
Drivers should use netif_set_tso_ipv6_max_size() to advertize their limit.
Unchanged drivers are not allowing big TSO packets to be sent.
--- include/linux/netdevice.h | 10 ++++++++++ include/uapi/linux/if_link.h | 1 + net/core/dev.c | 2 ++ net/core/rtnetlink.c | 3 +++ tools/include/uapi/linux/if_link.h | 1 + 5 files changed, 17 insertions(+)
include/linux/netdevice.h +10 -0
@@ -1951,6 +1951,7 @@ enum netdev_ml_priv_type {* @dev_registered_tracker: tracker for reference held while * registered * @offload_xstats_l3: L3 HW stats for this netdevice. +* @tso_ipv6_max_size: Maximum size of IPv6 TSO packets (driver/NIC limit) * * FIXME: cleanup struct net_device such that network protocol info * moves out.
@@ -2289,6 +2290,7 @@ struct net_device {netdevice_tracker watchdog_dev_tracker;netdevice_tracker dev_registered_tracker;struct rtnl_hw_stats64 *offload_xstats_l3; +unsigned int tso_ipv6_max_size; };#define to_net_dev (d) container_of(d, struct net_device, dev)
@@ -4888,6 +4890,14 @@ static inline void netif_set_gro_max_size(struct net_device *dev,WRITE_ONCE (dev->gro_max_size, size); } +/* Used by drivers to give their hardware/firmware limit for LSOv2 packets */ +static inline void netif_set_tso_ipv6_max_size (struct net_device *dev, +unsigned int size) +{ + dev->tso_ipv6_max_size = size; +} + +static inline void skb_gso_error_unwind (struct sk_buff *skb,__be16 protocol,int pulled_hlen,u16 mac_offset,int mac_len)
include/uapi/linux/if_link.h +1 -0
@@ -363,6 +363,7 @@ enum {IFLA_PARENT_DEV_NAME ,IFLA_PARENT_DEV_BUS_NAME ,IFLA_GRO_MAX_SIZE , +IFLA_TSO_IPV6_MAX_SIZE , __IFLA_MAX };
net/core/dev.c +2 -0
@@ -10467,6 +10467,8 @@ struct net_device *alloc_netdev_mqs(int sizeof_priv, const char *name, dev->gso_max_size =GSO_MAX_SIZE ; dev->gso_max_segs =GSO_MAX_SEGS ; dev->gro_max_size =GRO_MAX_SIZE ; + dev->tso_ipv6_max_size =GSO_MAX_SIZE ; + dev->upper_level =1 ; dev->lower_level =1 ;#ifdef CONFIG_LOCKDEP
net/core/rtnetlink.c +3 -0
@@ -1027,6 +1027,7 @@ static noinline size_t if_nlmsg_size(const struct net_device *dev, +nla_total_size (4 )/* IFLA_GSO_MAX_SEGS */ +nla_total_size (4 )/* IFLA_GSO_MAX_SIZE */ +nla_total_size (4 )/* IFLA_GRO_MAX_SIZE */ + +nla_total_size (4 )/* IFLA_TSO_IPV6_MAX_SIZE */ +nla_total_size (1 )/* IFLA_OPERSTATE */ +nla_total_size (1 )/* IFLA_LINKMODE */ +nla_total_size (4 )/* IFLA_CARRIER_CHANGES */
@@ -1732,6 +1733,7 @@ static int rtnl_fill_ifinfo(struct sk_buff *skb, nla_put_u32(skb, IFLA_GSO_MAX_SEGS, dev->gso_max_segs) || nla_put_u32(skb, IFLA_GSO_MAX_SIZE, dev->gso_max_size) || nla_put_u32(skb, IFLA_GRO_MAX_SIZE, dev->gro_max_size) || + nla_put_u32(skb, IFLA_TSO_IPV6_MAX_SIZE, dev->tso_ipv6_max_size) ||#ifdef CONFIG_RPS nla_put_u32(skb, IFLA_NUM_RX_QUEUES, dev->num_rx_queues) ||#endif
@@ -1885,6 +1887,7 @@ static const struct nla_policy ifla_policy[IFLA_MAX+1] = { [IFLA_NEW_IFINDEX] = NLA_POLICY_MIN(NLA_S32, 1), [IFLA_PARENT_DEV_NAME] = { .type = NLA_NUL_STRING }, [IFLA_GRO_MAX_SIZE] = { .type = NLA_U32 }, + [IFLA_TSO_IPV6_MAX_SIZE] = { .type = NLA_U32 }, };static const struct nla_policy ifla_info_policy[IFLA_INFO_MAX +1 ] = {
tools/include/uapi/linux/if_link.h +1 -0
@@ -348,6 +348,7 @@ enum {IFLA_PARENT_DEV_NAME ,IFLA_PARENT_DEV_BUS_NAME ,IFLA_GRO_MAX_SIZE , +IFLA_TSO_IPV6_MAX_SIZE , __IFLA_MAX };
Signature · 1 line
2.35.1.616.g0bdcbb4464-goog
PATCH v4 2/14 ipv6: add dev->gso_ipv6_max_size
In reply to Eric DumazetFrom: Coco Li <[email protected]>
This enable TCP stack to build TSO packets bigger than 64KB if the driver is LSOv2 compatible.
This patch introduces new variable gso_ipv6_max_size that is modifiable through ip link.
ip link set dev eth0 gso_ipv6_max_size 185000
User input is capped by driver limit (tso_ipv6_max_size) added in previous patch.
--- include/linux/netdevice.h | 12 ++++++++++++ include/uapi/linux/if_link.h | 1 + net/core/dev.c | 1 + net/core/rtnetlink.c | 15 +++++++++++++++ net/core/sock.c | 6 ++++++ tools/include/uapi/linux/if_link.h | 1 + 6 files changed, 36 insertions(+)
include/linux/netdevice.h +12 -0
@@ -1952,6 +1952,7 @@ enum netdev_ml_priv_type {* registered * @offload_xstats_l3: L3 HW stats for this netdevice. * @tso_ipv6_max_size: Maximum size of IPv6 TSO packets (driver/NIC limit) +* @gso_ipv6_max_size: Maximum size of IPv6 GSO packets (user/admin limit) * * FIXME: cleanup struct net_device such that network protocol info * moves out.
@@ -2291,6 +2292,7 @@ struct net_device {netdevice_tracker dev_registered_tracker;struct rtnl_hw_stats64 *offload_xstats_l3;unsigned int tso_ipv6_max_size; +unsigned int gso_ipv6_max_size; };#define to_net_dev (d) container_of(d, struct net_device, dev)
@@ -4874,6 +4876,10 @@ static inline void netif_set_gso_max_size(struct net_device *dev, {/* dev->gso_max_size is read locklessly from sk_setup_caps() */ WRITE_ONCE (dev->gso_max_size, size); + +/* legacy drivers want to lower gso_max_size, regardless of family. */ + size =min (size, dev->gso_ipv6_max_size); +WRITE_ONCE (dev->gso_ipv6_max_size, size); } static inline void netif_set_gso_max_segs(struct net_device *dev,
@@ -4897,6 +4903,12 @@ static inline void netif_set_tso_ipv6_max_size(struct net_device *dev, dev->tso_ipv6_max_size = size; } +static inline void netif_set_gso_ipv6_max_size (struct net_device *dev, +unsigned int size) +{ + size =min (size, dev->tso_ipv6_max_size); +WRITE_ONCE (dev->gso_ipv6_max_size, size); +} static inline void skb_gso_error_unwind(struct sk_buff *skb, __be16 protocol, int pulled_hlen, u16 mac_offset,
include/uapi/linux/if_link.h +1 -0
@@ -364,6 +364,7 @@ enum {IFLA_PARENT_DEV_BUS_NAME ,IFLA_GRO_MAX_SIZE ,IFLA_TSO_IPV6_MAX_SIZE , +IFLA_GSO_IPV6_MAX_SIZE , __IFLA_MAX };
net/core/dev.c +1 -0
@@ -10468,6 +10468,7 @@ struct net_device *alloc_netdev_mqs(int sizeof_priv, const char *name, dev->gso_max_segs =GSO_MAX_SEGS ; dev->gro_max_size =GRO_MAX_SIZE ; dev->tso_ipv6_max_size =GSO_MAX_SIZE ; + dev->gso_ipv6_max_size =GSO_MAX_SIZE ; dev->upper_level =1 ; dev->lower_level =1 ;
net/core/rtnetlink.c +15 -0
@@ -1028,6 +1028,7 @@ static noinline size_t if_nlmsg_size(const struct net_device *dev, +nla_total_size (4 )/* IFLA_GSO_MAX_SIZE */ +nla_total_size (4 )/* IFLA_GRO_MAX_SIZE */ +nla_total_size (4 )/* IFLA_TSO_IPV6_MAX_SIZE */ + +nla_total_size (4 )/* IFLA_GSO_IPV6_MAX_SIZE */ +nla_total_size (1 )/* IFLA_OPERSTATE */ +nla_total_size (1 )/* IFLA_LINKMODE */ +nla_total_size (4 )/* IFLA_CARRIER_CHANGES */
@@ -1734,6 +1735,7 @@ static int rtnl_fill_ifinfo(struct sk_buff *skb, nla_put_u32(skb, IFLA_GSO_MAX_SIZE, dev->gso_max_size) || nla_put_u32(skb, IFLA_GRO_MAX_SIZE, dev->gro_max_size) || nla_put_u32(skb, IFLA_TSO_IPV6_MAX_SIZE, dev->tso_ipv6_max_size) || + nla_put_u32(skb, IFLA_GSO_IPV6_MAX_SIZE, dev->gso_ipv6_max_size) ||#ifdef CONFIG_RPS nla_put_u32(skb, IFLA_NUM_RX_QUEUES, dev->num_rx_queues) ||#endif
@@ -1888,6 +1890,7 @@ static const struct nla_policy ifla_policy[IFLA_MAX+1] = { [IFLA_PARENT_DEV_NAME] = { .type = NLA_NUL_STRING }, [IFLA_GRO_MAX_SIZE] = { .type = NLA_U32 }, [IFLA_TSO_IPV6_MAX_SIZE] = { .type = NLA_U32 }, + [IFLA_GSO_IPV6_MAX_SIZE] = { .type = NLA_U32 }, };static const struct nla_policy ifla_info_policy[IFLA_INFO_MAX +1 ] = {
@@ -2774,6 +2777,15 @@ static int do_setlink(const struct sk_buff *skb, } } +if (tb[IFLA_GSO_IPV6_MAX_SIZE ]) { +u32 max_size =nla_get_u32 (tb[IFLA_GSO_IPV6_MAX_SIZE ]); + +if (dev->gso_ipv6_max_size ^ max_size) { +netif_set_gso_ipv6_max_size (dev, max_size); + status |=DO_SETLINK_MODIFIED ; + } + } +if (tb[IFLA_GSO_MAX_SEGS ]) {u32 max_segs =nla_get_u32 (tb[IFLA_GSO_MAX_SEGS ]);
@@ -3249,6 +3261,9 @@ struct net_device *rtnl_create_link(struct net *net, const char *ifname,netif_set_gso_max_segs (dev,nla_get_u32 (tb[IFLA_GSO_MAX_SEGS ]));if (tb[IFLA_GRO_MAX_SIZE ])netif_set_gro_max_size (dev,nla_get_u32 (tb[IFLA_GRO_MAX_SIZE ])); +if (tb[IFLA_GSO_IPV6_MAX_SIZE ]) +netif_set_gso_ipv6_max_size (dev, +nla_get_u32 (tb[IFLA_GSO_IPV6_MAX_SIZE ]));return dev; }
net/core/sock.c +6 -0
@@ -2279,6 +2279,12 @@ void sk_setup_caps(struct sock *sk, struct dst_entry *dst) sk->sk_route_caps |=NETIF_F_SG |NETIF_F_HW_CSUM ;/* pairs with the WRITE_ONCE() in netif_set_gso_max_size() */ sk->sk_gso_max_size =READ_ONCE (dst->dev->gso_max_size); +#if IS_ENABLED (CONFIG_IPV6 ) +if (sk->sk_family ==AF_INET6 && +sk_is_tcp (sk) && + !ipv6_addr_v4mapped (&sk->sk_v6_rcv_saddr)) + sk->sk_gso_max_size =READ_ONCE (dst->dev->gso_ipv6_max_size); +#endif sk->sk_gso_max_size -= (MAX_TCP_HEADER +1 );/* pairs with the WRITE_ONCE() in netif_set_gso_max_segs() */ max_segs =max_t (u32,READ_ONCE (dst->dev->gso_max_segs),1 );
tools/include/uapi/linux/if_link.h +1 -0
@@ -349,6 +349,7 @@ enum {IFLA_PARENT_DEV_BUS_NAME ,IFLA_GRO_MAX_SIZE ,IFLA_TSO_IPV6_MAX_SIZE , +IFLA_GSO_IPV6_MAX_SIZE , __IFLA_MAX };
Signature · 1 line
2.35.1.616.g0bdcbb4464-goog
Re: [PATCH v4 net-next 02/14] ipv6: add dev->gso_ipv6_max_size
Quoted 37 lines · Eric Dumazet
From: Coco Li <[email protected]>
This enable TCP stack to build TSO packets bigger than 64KB if the driver is LSOv2 compatible.
This patch introduces new variable
gso_ipv6_max_sizethat is modifiable through ip link.ip link set dev eth0
gso_ipv6_max_size185000User input is capped by driver limit (
tso_ipv6_max_size) added in previous patch.Signed-off-by Coco Li <[email protected]>Signed-off-by Eric Dumazet <[email protected]>--- include/linux/netdevice.h | 12 ++++++++++++ include/uapi/linux/if_link.h | 1 + net/core/dev.c | 1 + net/core/rtnetlink.c | 15 +++++++++++++++ net/core/sock.c | 6 ++++++ tools/include/uapi/linux/if_link.h | 1 + 6 files changed, 36 insertions(+)include/linux/netdevice.h +1
-0@@ -1952,6 +1952,7 @@ enum netdev_ml_priv_type {* registered * @offload_xstats_l3: L3 HW stats for this netdevice. * @tso_ipv6_max_size: Maximum size of IPv6 TSO packets (driver/NIC limit) +* @gso_ipv6_max_size: Maximum size of IPv6 GSO packets (user/admin limit) * * FIXME: cleanup struct net_device such that network protocol info * moves out.
include/linux/netdevice.h struct net_device { · line 2298
Quoted 7 lines
include/linux/netdevice.h +1
-0@@ -2291,6 +2292,7 @@ struct net_device {netdevice_tracker dev_registered_tracker;struct rtnl_hw_stats64 *offload_xstats_l3;unsigned int tso_ipv6_max_size; +unsigned int gso_ipv6_max_size; };#define to_net_dev (d) container_of(d, struct net_device, dev)
Rather than have this as a device specific value would it be advantageous to consider making this a namespace specific sysctl value instead? Something along the lines of:
net.ipv6.conf.*.max_jumbogram_size
It could also be applied generically to the GSO/GRO as the upper limit for any frame assembled by the socket or GRO.
The general idea is that might be desirable for admins to be able to basically just set the maximum size they want to see for IPv6 frames and if we could combine the GRO/GSO logic into a single sysctl that could be set on a namespace basis instead of a device basis which would be more difficult to track down. We already have the per-device limits in the tso_ipv6_max_size for the outgoing frames so it seems like it might make sense to make this per network namespace and defaultable rather than per device and requiring an update for each device instance.
Re: [PATCH v4 net-next 02/14] ipv6: add dev->gso_ipv6_max_size
Quoted 55 lines · Alexander H Duyck
Quoted 37 lines · Eric Dumazet
From: Coco Li <[email protected]>
This enable TCP stack to build TSO packets bigger than 64KB if the driver is LSOv2 compatible.
This patch introduces new variable
gso_ipv6_max_sizethat is modifiable through ip link.ip link set dev eth0
gso_ipv6_max_size185000User input is capped by driver limit (
tso_ipv6_max_size) added in previous patch.Signed-off-by Coco Li <[email protected]>Signed-off-by Eric Dumazet <[email protected]>--- include/linux/netdevice.h | 12 ++++++++++++ include/uapi/linux/if_link.h | 1 + net/core/dev.c | 1 + net/core/rtnetlink.c | 15 +++++++++++++++ net/core/sock.c | 6 ++++++ tools/include/uapi/linux/if_link.h | 1 + 6 files changed, 36 insertions(+)include/linux/netdevice.h +1
-0@@ -1952,6 +1952,7 @@ enum netdev_ml_priv_type {* registered * @offload_xstats_l3: L3 HW stats for this netdevice. * @tso_ipv6_max_size: Maximum size of IPv6 TSO packets (driver/NIC limit) +* @gso_ipv6_max_size: Maximum size of IPv6 GSO packets (user/admin limit) * * FIXME: cleanup struct net_device such that network protocol info * moves out. include/linux/netdevice.h struct net_device { · line 2298
Quoted 7 lines
include/linux/netdevice.h +1
-0@@ -2291,6 +2292,7 @@ struct net_device {netdevice_tracker dev_registered_tracker;struct rtnl_hw_stats64 *offload_xstats_l3;unsigned int tso_ipv6_max_size; +unsigned int gso_ipv6_max_size; };#define to_net_dev (d) container_of(d, struct net_device, dev)Rather than have this as a device specific value would it be advantageous to consider making this a namespace specific sysctl value instead? Something along the lines of:
net.ipv6.conf.*.max_jumbogram_sizeIt could also be applied generically to the GSO/GRO as the upper limit for any frame assembled by the socket or GRO.
Quoted 10 lines
The general idea is that might be desirable for admins to be able to basically just set the maximum size they want to see for IPv6 frames and if we could combine the GRO/GSO logic into a single sysctl that could be set on a namespace basis instead of a device basis which would be more difficult to track down. We already have the per-device limits in the
tso_ipv6_max_sizefor the outgoing frames so it seems like it might make sense to make this per network namespace and defaultable rather than per device and requiring an update for each device instance.
At least Google found it was easier to have per device controls, in terms of testing the feature,
and gradually deploying it.
We have hosts with multiple NIC, of different types. We want to be able to control BIG TCP on a per device basis.
For instance I had a bug in one of the implementation for one (non upstream) driver, that I could mitigate
by setting a different limit only for this NIC, until the host can boot with a fixed kernel.
We use ipvlan, with one private net-ns and IPv6 address per job, we wanted to deploy BIG TCP on a per job basis
I guess that if you want to add a sysctl, automatically overriding the per device setting, this could be done later ?
PATCH v4 3/14 tcp_cubic: make hystart_ack_delay() aware of BIG TCP
In reply to Eric DumazetFrom: Eric Dumazet <[email protected]>
hystart_ack_delay() had the assumption that a TSO packet would not be bigger than GSO_MAX_SIZE.
This will no longer be true.
We should use sk->sk_gso_max_size instead.
This reduces chances of spurious Hystart ACK train detections.
--- net/ipv4/tcp_cubic.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
net/ipv4/tcp_cubic.c +2 -2
@@ -372,7 +372,7 @@ static void cubictcp_state(struct sock *sk, u8 new_state)* We apply another 100% factor because @rate is doubled at this point. * We cap the cushion to 1ms. */ -static u32 hystart_ack_delay (struct sock *sk) +static u32 hystart_ack_delay (const struct sock *sk) {unsigned long rate;
@@ -380,7 +380,7 @@ static u32 hystart_ack_delay(struct sock *sk)if (!rate)return 0 ;return min_t (u64,USEC_PER_MSEC , -div64_ul ((u64 )GSO_MAX_SIZE *4 *USEC_PER_SEC , rate)); +div64_ul ((u64 )sk->sk_gso_max_size *4 *USEC_PER_SEC , rate)); }static void hystart_update (struct sock *sk,u32 delay)
Signature · 1 line
2.35.1.616.g0bdcbb4464-goog
PATCH v4 4/14 ipv6: add struct hop_jumbo_hdr definition
In reply to Eric DumazetFrom: Eric Dumazet <[email protected]>
Following patches will need to add and remove local IPv6 jumbogram options to enable BIG TCP.
--- include/net/ipv6.h | 11 +++++++++++ 1 file changed, 11 insertions(+)
include/net/ipv6.h +11 -0
@@ -151,6 +151,17 @@ struct frag_hdr {__be32 identification; }; +/* +* Jumbo payload option, as described in RFC 2675 2. +*/ +struct hop_jumbo_hdr { +u8 nexthdr; +u8 hdrlen; +u8 tlv_type;/* IPV6_TLV_JUMBO, 0xC2 */ +u8 tlv_len;/* 4 */ +__be32 jumbo_payload_len; +}; +#define IP6_MF 0x0001#define IP6_OFFSET 0xFFF8
Signature · 1 line
2.35.1.616.g0bdcbb4464-goog
PATCH v4 5/14 ipv6/gso: remove temporary HBH/jumbo header
In reply to Eric DumazetFrom: Eric Dumazet <[email protected]>
ipv6 tcp and gro stacks will soon be able to build big TCP packets, with an added temporary Hop By Hop header.
If GSO is involved for these large packets, we need to remove the temporary HBH header before segmentation happens.
v2: perform HBH removal from ipv6_gso_segment() instead of
skb_segment() (Alexander feedback)
--- include/net/ipv6.h | 33 +++++++++++++++++++++++++++++++++ net/ipv6/ip6_offload.c | 24 +++++++++++++++++++++++- 2 files changed, 56 insertions(+), 1 deletion(-)
include/net/ipv6.h +33 -0
@@ -467,6 +467,39 @@ bool ipv6_opt_accepted(const struct sock *sk, const struct sk_buff *skb,struct ipv6_txoptions *ipv6_update_options (struct sock *sk,struct ipv6_txoptions *opt); +/* This helper is specialized for BIG TCP needs. +* It assumes the hop_jumbo_hdr will immediately follow the IPV6 header. +* It assumes headers are already in skb->head. +* Returns 0, or IPPROTO_TCP if a BIG TCP packet is there. +*/ +static inline int ipv6_has_hopopt_jumbo (const struct sk_buff *skb) +{ +const struct hop_jumbo_hdr *jhdr; +const struct ipv6hdr *nhdr; + +if (likely (skb->len <=GRO_MAX_SIZE )) +return 0 ; + +if (skb->protocol !=htons (ETH_P_IPV6 )) +return 0 ; + +if (skb_network_offset (skb) + +sizeof (struct ipv6hdr ) + +sizeof (struct hop_jumbo_hdr ) >skb_headlen (skb)) +return 0 ; + + nhdr =ipv6_hdr (skb); + +if (nhdr->nexthdr !=NEXTHDR_HOP ) +return 0 ; + + jhdr = (const struct hop_jumbo_hdr *) (nhdr +1 ); +if (jhdr->tlv_type !=IPV6_TLV_JUMBO || jhdr->hdrlen !=0 || + jhdr->nexthdr !=IPPROTO_TCP ) +return 0 ; +return jhdr->nexthdr; +} +static inline bool ipv6_accept_ra (struct inet6_dev *idev) {/* If forwarding is enabled, RA are not accepted unless the special
net/ipv6/ip6_offload.c +23 -1
@@ -77,7 +77,7 @@ static struct sk_buff *ipv6_gso_segment(struct sk_buff *skb,struct sk_buff *segs =ERR_PTR (-EINVAL );struct ipv6hdr *ipv6h;const struct net_offload *ops; -int proto; +int proto, nexthdr;struct frag_hdr *fptr;unsigned int payload_len;u8 *prevhdr;
@@ -87,6 +87,28 @@ static struct sk_buff *ipv6_gso_segment(struct sk_buff *skb,bool gso_partial;skb_reset_network_header (skb); + nexthdr =ipv6_has_hopopt_jumbo (skb); +if (nexthdr) { +const int hophdr_len =sizeof (struct hop_jumbo_hdr ); +int err; + + err =skb_cow_head (skb,0 ); +if (err <0 ) +return ERR_PTR (err); + +/* remove the HBH header. +* Layout: [Ethernet header][IPv6 header][HBH][TCP header] +*/ +memmove (skb_mac_header (skb) + hophdr_len, +skb_mac_header (skb), +ETH_HLEN +sizeof (struct ipv6hdr )); + skb->data += hophdr_len; + skb->len -= hophdr_len; + skb->network_header += hophdr_len; + skb->mac_header += hophdr_len; + ipv6h = (struct ipv6hdr *)skb->data; + ipv6h->nexthdr = nexthdr; + } nhoff =skb_network_header (skb) -skb_mac_header (skb);if (unlikely (!pskb_may_pull (skb,sizeof (*ipv6h)))) goto out;
Signature · 1 line
2.35.1.616.g0bdcbb4464-goog
PATCH v4 6/14 ipv6/gro: insert temporary HBH/jumbo header
In reply to Eric DumazetFrom: Eric Dumazet <[email protected]>
Following patch will add GRO_IPV6_MAX_SIZE, allowing gro to build BIG TCP ipv6 packets (bigger than 64K).
This patch changes ipv6_gro_complete() to insert a HBH/jumbo header so that resulting packet can go through IPv6/TCP stacks.
--- net/ipv6/ip6_offload.c | 32 ++++++++++++++++++++++++++++++-- 1 file changed, 30 insertions(+), 2 deletions(-)
net/ipv6/ip6_offload.c +30 -2
@@ -342,15 +342,43 @@ static struct sk_buff *ip4ip6_gro_receive(struct list_head *head, INDIRECT_CALLABLE_SCOPE int ipv6_gro_complete(struct sk_buff *skb, int nhoff) {const struct net_offload *ops; -struct ipv6hdr *iph = (struct ipv6hdr *)(skb->data + nhoff); +struct ipv6hdr *iph;int err = -ENOSYS ; +u32 payload_len;if (skb->encapsulation) {skb_set_inner_protocol (skb,cpu_to_be16 (ETH_P_IPV6 ));skb_set_inner_network_header (skb, nhoff); } - iph->payload_len =htons (skb->len - nhoff -sizeof (*iph)); + payload_len = skb->len - nhoff -sizeof (*iph); +if (unlikely (payload_len >IPV6_MAXPLEN )) { +struct hop_jumbo_hdr *hop_jumbo; +int hoplen =sizeof (*hop_jumbo); + +/* Move network header left */ +memmove (skb_mac_header (skb) - hoplen,skb_mac_header (skb), + skb->transport_header - skb->mac_header); + skb->data -= hoplen; + skb->len += hoplen; + skb->mac_header -= hoplen; + skb->network_header -= hoplen; + iph = (struct ipv6hdr *)(skb->data + nhoff); + hop_jumbo = (struct hop_jumbo_hdr *)(iph +1 ); + +/* Build hop-by-hop options */ + hop_jumbo->nexthdr = iph->nexthdr; + hop_jumbo->hdrlen =0 ; + hop_jumbo->tlv_type =IPV6_TLV_JUMBO ; + hop_jumbo->tlv_len =4 ; + hop_jumbo->jumbo_payload_len =htonl (payload_len + hoplen); + + iph->nexthdr =NEXTHDR_HOP ; + iph->payload_len =0 ; + }else { + iph = (struct ipv6hdr *)(skb->data + nhoff); + iph->payload_len =htons (payload_len); + } nhoff +=sizeof (*iph) +ipv6_exthdrs_len (iph, &ops);if (WARN_ON (!ops || !ops->callbacks.gro_complete))
Signature · 1 line
2.35.1.616.g0bdcbb4464-goog
Re: [PATCH v4 net-next 06/14] ipv6/gro: insert temporary HBH/jumbo header
Quoted 5 lines · Eric Dumazet
From: Eric Dumazet <[email protected]>
Following patch will add
GRO_IPV6_MAX_SIZE, allowing gro to build BIG TCP ipv6 packets (bigger than 64K).
This looks like it belongs in the next patch, not this one. This patch is adding the HBH header.
Quoted 58 lines
This patch changes
ipv6_gro_complete()to insert a HBH/jumbo header so that resulting packet can go through IPv6/TCP stacks.Signed-off-by Eric Dumazet <[email protected]>--- net/ipv6/ip6_offload.c | 32 ++++++++++++++++++++++++++++++-- 1 file changed, 30 insertions(+), 2 deletions(-)net/ipv6/ip6_offload.c +30
-2@@ -342,15 +342,43 @@ static struct sk_buff *ip4ip6_gro_receive(struct list_head *head, INDIRECT_CALLABLE_SCOPE int ipv6_gro_complete(struct sk_buff *skb, int nhoff) {const struct net_offload *ops; -struct ipv6hdr *iph = (struct ipv6hdr *)(skb->data + nhoff); +struct ipv6hdr *iph;int err = -ENOSYS ; +u32 payload_len;if (skb->encapsulation) {skb_set_inner_protocol (skb,cpu_to_be16 (ETH_P_IPV6 ));skb_set_inner_network_header (skb, nhoff); } - iph->payload_len =htons (skb->len - nhoff -sizeof (*iph)); + payload_len = skb->len - nhoff -sizeof (*iph); +if (unlikely (payload_len >IPV6_MAXPLEN )) { +struct hop_jumbo_hdr *hop_jumbo; +int hoplen =sizeof (*hop_jumbo); + +/* Move network header left */ +memmove (skb_mac_header (skb) - hoplen,skb_mac_header (skb), + skb->transport_header - skb->mac_header); + skb->data -= hoplen; + skb->len += hoplen; + skb->mac_header -= hoplen; + skb->network_header -= hoplen; + iph = (struct ipv6hdr *)(skb->data + nhoff); + hop_jumbo = (struct hop_jumbo_hdr *)(iph +1 ); + +/* Build hop-by-hop options */ + hop_jumbo->nexthdr = iph->nexthdr; + hop_jumbo->hdrlen =0 ; + hop_jumbo->tlv_type =IPV6_TLV_JUMBO ; + hop_jumbo->tlv_len =4 ; + hop_jumbo->jumbo_payload_len =htonl (payload_len + hoplen); + + iph->nexthdr =NEXTHDR_HOP ; + iph->payload_len =0 ; + }else { + iph = (struct ipv6hdr *)(skb->data + nhoff); + iph->payload_len =htons (payload_len); + } nhoff +=sizeof (*iph) +ipv6_exthdrs_len (iph, &ops);if (WARN_ON (!ops || !ops->callbacks.gro_complete))
Re: [PATCH v4 net-next 06/14] ipv6/gro: insert temporary HBH/jumbo header
Quoted 10 lines · Alexander H Duyck
Quoted 5 lines · Eric Dumazet
From: Eric Dumazet <[email protected]>
Following patch will add
GRO_IPV6_MAX_SIZE, allowing gro to build BIG TCP ipv6 packets (bigger than 64K).This looks like it belongs in the next patch, not this one. This patch is adding the HBH header.
What do you mean by "it belongs" ?
Do you want me to squash the patches, or remove the first sentence ?
I am confused.
Quoted 4 lines
Quoted 4 lines
This patch changes
ipv6_gro_complete()to insert a HBH/jumbo header so that resulting packet can go through IPv6/TCP stacks.
RE: [PATCH v4 net-next 06/14] ipv6/gro: insert temporary HBH/jumbo header
Quoted 23 lines
-----Original Message-----
From: Eric Dumazet <[email protected]>
Sent: Tuesday, March 15, 2022 9:02 AM
To: Alexander H Duyck <[email protected]>
Cc: Eric Dumazet <[email protected]>; David S . Miller <[email protected]>; Jakub Kicinski <[email protected]>; netdev <[email protected]>; Alexander Duyck <[email protected]>; Coco Li <[email protected]>
Subject: Re: [PATCH v4 net-next 06/14] ipv6/gro: insert temporary HBH/jumbo headerQuoted 8 lines · Alexander H Duyck
Quoted 5 lines · Eric Dumazet
From: Eric Dumazet <[email protected]>
Following patch will add
GRO_IPV6_MAX_SIZE, allowing gro to build BIG TCP ipv6 packets (bigger than 64K).
Quoted 8 lines
Quoted 2 lines
This looks like it belongs in the next patch, not this one. This patch is adding the HBH header.
What do you mean by "it belongs" ?
Do you want me to squash the patches, or remove the first sentence ?
I am confused.
It is about the sentence. Your next patch essentially has that as the title and actually does add GRO_IPV6_MAX_SIZE. I wasn't sure if you reordered the patches or split them. However as I recall I didn't see anything in this patch that added GRO_IPV6_MAX_SIZE.
Re: [PATCH v4 net-next 06/14] ipv6/gro: insert temporary HBH/jumbo header
Quoted 27 lines · Alexander Duyck
Quoted 27 lines
-----Original Message-----
From: Eric Dumazet <[email protected]>
Sent: Tuesday, March 15, 2022 9:02 AM
To: Alexander H Duyck <[email protected]>
Cc: Eric Dumazet <[email protected]>; David S . Miller <[email protected]>; Jakub Kicinski <[email protected]>; netdev <[email protected]>; Alexander Duyck <[email protected]>; Coco Li <[email protected]>
Subject: Re: [PATCH v4 net-next 06/14] ipv6/gro: insert temporary HBH/jumbo headerQuoted 10 lines · Alexander H Duyck
Quoted 5 lines · Eric Dumazet
From: Eric Dumazet <[email protected]>
Following patch will add
GRO_IPV6_MAX_SIZE, allowing gro to build BIG TCP ipv6 packets (bigger than 64K).This looks like it belongs in the next patch, not this one. This patch is adding the HBH header.
Quoted 7 lines
Quoted 5 lines
What do you mean by "it belongs" ?
Do you want me to squash the patches, or remove the first sentence ?
I am confused.
It is about the sentence. Your next patch essentially has that as the title and actually does add
GRO_IPV6_MAX_SIZE. I wasn't sure if you reordered the patches or split them. However as I recall I didn't see anything in this patch that addedGRO_IPV6_MAX_SIZE.
I used "Following patch will", meaning the patch following _this_ one, sorry if this is confusing.
I would have used "This patch is ..." if I wanted to describe what this patch is doing.
Patches were not reordered, and have two different authors.
RE: [PATCH v4 net-next 06/14] ipv6/gro: insert temporary HBH/jumbo header
Quoted 53 lines
-----Original Message-----
From: Eric Dumazet <[email protected]>
Sent: Tuesday, March 15, 2022 9:11 AM
To: Alexander Duyck <[email protected]>
Cc: Alexander H Duyck <[email protected]>; Eric Dumazet <[email protected]>; David S . Miller <[email protected]>; Jakub Kicinski <[email protected]>; netdev <[email protected]>; Coco Li <[email protected]>
Subject: Re: [PATCH v4 net-next 06/14] ipv6/gro: insert temporary HBH/jumbo headerQuoted 10 lines · Alexander Duyck
Quoted 10 lines
-----Original Message-----
From: Eric Dumazet <[email protected]>
Sent: Tuesday, March 15, 2022 9:02 AM
To: Alexander H Duyck <[email protected]>
Cc: Eric Dumazet <[email protected]>; David S . Miller <[email protected]>; Jakub Kicinski <[email protected]>; netdev <[email protected]>; Alexander DuyckQuoted 17 lines
Quoted 17 lines
Coco Li <[email protected]>
Subject: Re: [PATCH v4 net-next 06/14] ipv6/gro: insert temporary HBH/jumbo headerQuoted 10 lines · Alexander H Duyck
Quoted 5 lines · Eric Dumazet
From: Eric Dumazet <[email protected]>
Following patch will add
GRO_IPV6_MAX_SIZE, allowing gro to build BIG TCP ipv6 packets (bigger than 64K).This looks like it belongs in the next patch, not this one. This patch is adding the HBH header.
Quoted 7 lines
Quoted 5 lines
What do you mean by "it belongs" ?
Do you want me to squash the patches, or remove the first sentence ?
I am confused.
It is about the sentence. Your next patch essentially has that as the title and
actually does add
GRO_IPV6_MAX_SIZE. I wasn't sure if you reordered the patches or split them. However as I recall I didn't see anything in this patch that addedGRO_IPV6_MAX_SIZE.
Quoted 7 lines
I used "Following patch will", meaning the patch following _this_ one, sorry if this is confusing.
I would have used "This patch is ..." if I wanted to describe what this patch is doing.
Patches were not reordered, and have two different authors.
Yeah, the problem is I read "Following patch" as in the "The patch below". I would probably drop the line since it doesn't add much to the patch itself.
PATCH v4 7/14 ipv6: add GRO_IPV6_MAX_SIZE
In reply to Eric DumazetFrom: Coco Li <[email protected]>
Enable GRO to have IPv6 specific limit for max packet size.
This patch introduces new dev->gro_ipv6_max_size that is modifiable through ip link.
ip link set dev eth0 gro_ipv6_max_size 185000
Note that this value is only considered if bigger than gro_max_size, and for non encapsulated TCP/ipv6 packets.
--- include/linux/netdevice.h | 10 ++++++++++ include/uapi/linux/if_link.h | 1 + net/core/dev.c | 1 + net/core/gro.c | 20 ++++++++++++++++++-- net/core/rtnetlink.c | 15 +++++++++++++++ tools/include/uapi/linux/if_link.h | 1 + 6 files changed, 46 insertions(+), 2 deletions(-)
include/linux/netdevice.h +10 -0
@@ -1944,6 +1944,8 @@ enum netdev_ml_priv_type {* keep a list of interfaces to be deleted. * @gro_max_size: Maximum size of aggregated packet in generic * receive offload (GRO) +* @gro_ipv6_max_size: Maximum size of aggregated packet in generic +* receive offload (GRO), for IPv6 * * @dev_addr_shadow: Copy of @dev_addr to catch direct writes. * @linkwatch_dev_tracker: refcount tracker used by linkwatch.
@@ -2140,6 +2142,7 @@ struct net_device {int napi_defer_hard_irqs;#define GRO_MAX_SIZE 65536unsigned int gro_max_size; +unsigned int gro_ipv6_max_size;rx_handler_func_t __rcu *rx_handler;void __rcu *rx_handler_data;
@@ -4910,6 +4913,13 @@ static inline void netif_set_gso_ipv6_max_size(struct net_device *dev,WRITE_ONCE (dev->gso_ipv6_max_size, size); } +static inline void netif_set_gro_ipv6_max_size (struct net_device *dev, +unsigned int size) +{ +/* This pairs with the READ_ONCE() in skb_gro_receive() */ +WRITE_ONCE (dev->gro_ipv6_max_size, size); +} +static inline void skb_gso_error_unwind (struct sk_buff *skb,__be16 protocol,int pulled_hlen,u16 mac_offset,int mac_len)
include/uapi/linux/if_link.h +1 -0
@@ -365,6 +365,7 @@ enum {IFLA_GRO_MAX_SIZE ,IFLA_TSO_IPV6_MAX_SIZE ,IFLA_GSO_IPV6_MAX_SIZE , +IFLA_GRO_IPV6_MAX_SIZE , __IFLA_MAX };
net/core/dev.c +1 -0
@@ -10469,6 +10469,7 @@ struct net_device *alloc_netdev_mqs(int sizeof_priv, const char *name, dev->gro_max_size =GRO_MAX_SIZE ; dev->tso_ipv6_max_size =GSO_MAX_SIZE ; dev->gso_ipv6_max_size =GSO_MAX_SIZE ; + dev->gro_ipv6_max_size =GRO_MAX_SIZE ; dev->upper_level =1 ; dev->lower_level =1 ;
net/core/gro.c +18 -2
@@ -136,11 +136,27 @@ int skb_gro_receive(struct sk_buff *p, struct sk_buff *skb)unsigned int new_truesize;struct sk_buff *lp; +if (unlikely (NAPI_GRO_CB (skb)->flush)) +return -E2BIG ; +/* pairs with WRITE_ONCE() in netif_set_gro_max_size() */ gro_max_size =READ_ONCE (p->dev->gro_max_size); -if (unlikely (p->len + len >= gro_max_size ||NAPI_GRO_CB (skb)->flush)) -return -E2BIG ; +if (unlikely (p->len + len >= gro_max_size)) { +/* pairs with WRITE_ONCE() in netif_set_gro_ipv6_max_size() */ +unsigned int gro6_max_size =READ_ONCE (p->dev->gro_ipv6_max_size); + +if (gro6_max_size > gro_max_size && + p->protocol ==htons (ETH_P_IPV6 ) && +skb_headroom (p) >=sizeof (struct hop_jumbo_hdr ) && +ipv6_hdr (p)->nexthdr ==IPPROTO_TCP && + !p->encapsulation) + gro_max_size = gro6_max_size; + +if (p->len + len >= gro_max_size) +return -E2BIG ; + } + lp =NAPI_GRO_CB (p)->last; pinfo =skb_shinfo (lp);
net/core/rtnetlink.c +15 -0
@@ -1029,6 +1029,7 @@ static noinline size_t if_nlmsg_size(const struct net_device *dev, +nla_total_size (4 )/* IFLA_GRO_MAX_SIZE */ +nla_total_size (4 )/* IFLA_TSO_IPV6_MAX_SIZE */ +nla_total_size (4 )/* IFLA_GSO_IPV6_MAX_SIZE */ + +nla_total_size (4 )/* IFLA_GRO_IPV6_MAX_SIZE */ +nla_total_size (1 )/* IFLA_OPERSTATE */ +nla_total_size (1 )/* IFLA_LINKMODE */ +nla_total_size (4 )/* IFLA_CARRIER_CHANGES */
@@ -1736,6 +1737,7 @@ static int rtnl_fill_ifinfo(struct sk_buff *skb,nla_put_u32 (skb,IFLA_GRO_MAX_SIZE , dev->gro_max_size) ||nla_put_u32 (skb,IFLA_TSO_IPV6_MAX_SIZE , dev->tso_ipv6_max_size) ||nla_put_u32 (skb,IFLA_GSO_IPV6_MAX_SIZE , dev->gso_ipv6_max_size) || +nla_put_u32 (skb,IFLA_GRO_IPV6_MAX_SIZE , dev->gro_ipv6_max_size) || #ifdef CONFIG_RPS nla_put_u32(skb, IFLA_NUM_RX_QUEUES, dev->num_rx_queues) || #endif
@@ -1891,6 +1893,7 @@ static const struct nla_policy ifla_policy[IFLA_MAX+1] = { [IFLA_GRO_MAX_SIZE] = { .type = NLA_U32 }, [IFLA_TSO_IPV6_MAX_SIZE] = { .type = NLA_U32 }, [IFLA_GSO_IPV6_MAX_SIZE] = { .type = NLA_U32 }, + [IFLA_GRO_IPV6_MAX_SIZE] = { .type = NLA_U32 }, };static const struct nla_policy ifla_info_policy[IFLA_INFO_MAX +1 ] = {
@@ -2786,6 +2789,15 @@ static int do_setlink(const struct sk_buff *skb, } } +if (tb[IFLA_GRO_IPV6_MAX_SIZE ]) { +u32 max_size =nla_get_u32 (tb[IFLA_GRO_IPV6_MAX_SIZE ]); + +if (dev->gro_ipv6_max_size ^ max_size) { +netif_set_gro_ipv6_max_size (dev, max_size); + status |=DO_SETLINK_MODIFIED ; + } + } +if (tb[IFLA_GSO_MAX_SEGS ]) {u32 max_segs =nla_get_u32 (tb[IFLA_GSO_MAX_SEGS ]);
@@ -3264,6 +3276,9 @@ struct net_device *rtnl_create_link(struct net *net, const char *ifname,if (tb[IFLA_GSO_IPV6_MAX_SIZE ])netif_set_gso_ipv6_max_size (dev,nla_get_u32 (tb[IFLA_GSO_IPV6_MAX_SIZE ])); +if (tb[IFLA_GRO_IPV6_MAX_SIZE ]) +netif_set_gro_ipv6_max_size (dev, +nla_get_u32 (tb[IFLA_GRO_IPV6_MAX_SIZE ]));return dev; }
tools/include/uapi/linux/if_link.h +1 -0
@@ -350,6 +350,7 @@ enum {IFLA_GRO_MAX_SIZE ,IFLA_TSO_IPV6_MAX_SIZE ,IFLA_GSO_IPV6_MAX_SIZE , +IFLA_GRO_IPV6_MAX_SIZE , __IFLA_MAX };
Signature · 1 line
2.35.1.616.g0bdcbb4464-goog
PATCH v4 8/14 ipv6: Add hop-by-hop header to jumbograms in ip6_output
In reply to Eric DumazetFrom: Coco Li <[email protected]>
Instead of simply forcing a 0 payload_len in IPv6 header, implement RFC 2675 and insert a custom extension header.
Note that only TCP stack is currently potentially generating jumbograms, and that this extension header is purely local, it wont be sent on a physical link.
This is needed so that packet capture (tcpdump and friends) can properly dissect these large packets.
--- include/linux/ipv6.h | 1 + net/ipv6/ip6_output.c | 22 ++++++++++++++++++++-- 2 files changed, 21 insertions(+), 2 deletions(-)
include/linux/ipv6.h +1 -0
@@ -144,6 +144,7 @@ struct inet6_skb_parm {#define IP6SKB_L3SLAVE 64#define IP6SKB_JUMBOGRAM 128#define IP6SKB_SEG6 256 +#define IP6SKB_FAKEJUMBO 512 };#if defined(CONFIG_NET_L3_MASTER_DEV )
net/ipv6/ip6_output.c +20 -2
@@ -180,7 +180,9 @@ static int __ip6_finish_output(struct net *net, struct sock *sk, struct sk_buff#endif mtu =ip6_skb_dst_mtu (skb); -if (skb_is_gso (skb) && !skb_gso_validate_network_len (skb, mtu)) +if (skb_is_gso (skb) && + !(IP6CB (skb)->flags &IP6SKB_FAKEJUMBO ) && + !skb_gso_validate_network_len (skb, mtu))return ip6_finish_output_gso_slowpath_drop (net, sk, skb, mtu); if ((skb->len > mtu && !skb_is_gso(skb)) ||
@@ -251,6 +253,8 @@ int ip6_xmit(const struct sock *sk, struct sk_buff *skb, struct flowi6 *fl6,struct dst_entry *dst =skb_dst (skb);struct net_device *dev = dst->dev;struct inet6_dev *idev =ip6_dst_idev (dst); +struct hop_jumbo_hdr *hop_jumbo; +int hoplen =sizeof (*hop_jumbo);unsigned int head_room;struct ipv6hdr *hdr;u8 proto = fl6->flowi6_proto;
@@ -258,7 +262,7 @@ int ip6_xmit(const struct sock *sk, struct sk_buff *skb, struct flowi6 *fl6,int hlimit =-1 ;u32 mtu; - head_room =sizeof (struct ipv6hdr ) +LL_RESERVED_SPACE (dev); + head_room =sizeof (struct ipv6hdr ) + hoplen +LL_RESERVED_SPACE (dev);if (opt) head_room += opt->opt_nflen + opt->opt_flen;
@@ -281,6 +285,20 @@ int ip6_xmit(const struct sock *sk, struct sk_buff *skb, struct flowi6 *fl6, &fl6->saddr); } +if (unlikely (seg_len >IPV6_MAXPLEN )) { + hop_jumbo =skb_push (skb, hoplen); + + hop_jumbo->nexthdr = proto; + hop_jumbo->hdrlen =0 ; + hop_jumbo->tlv_type =IPV6_TLV_JUMBO ; + hop_jumbo->tlv_len =4 ; + hop_jumbo->jumbo_payload_len =htonl (seg_len + hoplen); + + proto =IPPROTO_HOPOPTS ; + seg_len =0 ; +IP6CB (skb)->flags |=IP6SKB_FAKEJUMBO ; + } +skb_push (skb,sizeof (struct ipv6hdr ));skb_reset_network_header (skb); hdr =ipv6_hdr (skb);
Signature · 1 line
2.35.1.616.g0bdcbb4464-goog
PATCH v4 9/14 net: loopback: enable BIG TCP packets
In reply to Eric DumazetFrom: Eric Dumazet <[email protected]>
Set the driver limit to 512 KB per TSO ipv6 packet.
This allows the admin/user to set a GSO ipv6 limit up to this value.
Tested:
ip link set dev lo gso_ipv6_max_size 200000 netperf -H ::1 -t TCP_RR -l 100 -- -r 80000,80000 &
tcpdump shows :
18:28:42.962116 IP6 ::1 > ::1: HBH 40051 > 63780: Flags [P.], seq 3626480001:3626560001, ack 3626560001, win 17743, options [nop,nop,TS val 3771179265 ecr 3771179265], length 80000 18:28:42.962138 IP6 ::1.63780 > ::1.40051: Flags [.], ack 3626560001, win 17743, options [nop,nop,TS val 3771179265 ecr 3771179265], length 0 18:28:42.962152 IP6 ::1 > ::1: HBH 63780 > 40051: Flags [P.], seq 3626560001:3626640001, ack 3626560001, win 17743, options [nop,nop,TS val 3771179265 ecr 3771179265], length 80000 18:28:42.962157 IP6 ::1.40051 > ::1.63780: Flags [.], ack 3626640001, win 17743, options [nop,nop,TS val 3771179265 ecr 3771179265], length 0 18:28:42.962180 IP6 ::1 > ::1: HBH 40051 > 63780: Flags [P.], seq 3626560001:3626640001, ack 3626640001, win 17743, options [nop,nop,TS val 3771179265 ecr 3771179265], length 80000 18:28:42.962214 IP6 ::1.63780 > ::1.40051: Flags [.], ack 3626640001, win 17743, options [nop,nop,TS val 3771179266 ecr 3771179265], length 0 18:28:42.962228 IP6 ::1 > ::1: HBH 63780 > 40051: Flags [P.], seq 3626640001:3626720001, ack 3626640001, win 17743, options [nop,nop,TS val 3771179266 ecr 3771179265], length 80000 18:28:42.962233 IP6 ::1.40051 > ::1.63780: Flags [.], ack 3626720001, win 17743, options [nop,nop,TS val 3771179266 ecr 3771179266], length 0
--- drivers/net/loopback.c | 2 ++ 1 file changed, 2 insertions(+)
drivers/net/loopback.c +2 -0
@@ -191,6 +191,8 @@ static void gen_lo_setup(struct net_device *dev, dev->netdev_ops = dev_ops; dev->needs_free_netdev = true; dev->priv_destructor = dev_destructor; + +netif_set_tso_ipv6_max_size (dev,512 *1024 ); }/* The loopback device is special. There is only one instance
Signature · 1 line
2.35.1.616.g0bdcbb4464-goog
PATCH v4 10/14 bonding: update dev->tso_ipv6_max_size
In reply to Eric DumazetFrom: Eric Dumazet <[email protected]>
Use the minimal value found in the set of lower devices.
--- drivers/net/bonding/bond_main.c | 3 +++ 1 file changed, 3 insertions(+)
drivers/net/bonding/bond_main.c +3 -0
@@ -1420,6 +1420,7 @@ static void bond_compute_features(struct bonding *bond)struct slave *slave;unsigned short max_hard_header_len =ETH_HLEN ;unsigned int gso_max_size =GSO_MAX_SIZE ; +unsigned int tso_ipv6_max_size = ~0U ;u16 gso_max_segs =GSO_MAX_SEGS ;if (!bond_has_slaves (bond))
@@ -1450,6 +1451,7 @@ static void bond_compute_features(struct bonding *bond) max_hard_header_len = slave->dev->hard_header_len; gso_max_size =min (gso_max_size, slave->dev->gso_max_size); + tso_ipv6_max_size =min (tso_ipv6_max_size, slave->dev->tso_ipv6_max_size); gso_max_segs =min (gso_max_segs, slave->dev->gso_max_segs); } bond_dev->hard_header_len = max_hard_header_len;
@@ -1465,6 +1467,7 @@ static void bond_compute_features(struct bonding *bond) bond_dev->mpls_features = mpls_features;netif_set_gso_max_segs (bond_dev, gso_max_segs);netif_set_gso_max_size (bond_dev, gso_max_size); +netif_set_tso_ipv6_max_size (bond_dev, tso_ipv6_max_size); bond_dev->priv_flags &= ~IFF_XMIT_DST_RELEASE ; if ((bond_dev->priv_flags & IFF_XMIT_DST_RELEASE_PERM) &&
Signature · 1 line
2.35.1.616.g0bdcbb4464-goog
PATCH v4 11/14 macvlan: enable BIG TCP Packets
In reply to Eric DumazetFrom: Eric Dumazet <[email protected]>
Inherit tso_ipv6_max_size from lower device.
--- drivers/net/macvlan.c | 1 + 1 file changed, 1 insertion(+)
drivers/net/macvlan.c +1 -0
@@ -902,6 +902,7 @@ static int macvlan_init(struct net_device *dev) dev->hw_enc_features |= dev->features;netif_set_gso_max_size (dev, lowerdev->gso_max_size);netif_set_gso_max_segs (dev, lowerdev->gso_max_segs); +netif_set_tso_ipv6_max_size (dev, lowerdev->tso_ipv6_max_size); dev->hard_header_len = lowerdev->hard_header_len;macvlan_set_lockdep_class (dev);
Signature · 1 line
2.35.1.616.g0bdcbb4464-goog
PATCH v4 12/14 ipvlan: enable BIG TCP Packets
In reply to Eric DumazetFrom: Coco Li <[email protected]>
Inherit tso_ipv6_max_size from physical device.
Tested:
eth0 tso_ipv6_max_size is set to 524288
ip link add link eth0 name ipvl1 type ipvlan
ip -d link show ipvl1
10: ipvl1@eth0:...
ipvlan mode l3 bridge addrgenmode eui64 numtxqueues 1 numrxqueues 1 gso_max_size 65536 gso_max_segs 65535 gro_max_size 65536 gso_ipv6_max_size 65535 tso_ipv6_max_size 524288 gro_ipv6_max_size 65536
--- drivers/net/ipvlan/ipvlan_main.c | 1 + 1 file changed, 1 insertion(+)
drivers/net/ipvlan/ipvlan_main.c +1 -0
@@ -141,6 +141,7 @@ static int ipvlan_init(struct net_device *dev) dev->hw_enc_features |= dev->features;netif_set_gso_max_size (dev, phy_dev->gso_max_size);netif_set_gso_max_segs (dev, phy_dev->gso_max_segs); +netif_set_tso_ipv6_max_size (dev, phy_dev->tso_ipv6_max_size); dev->hard_header_len = phy_dev->hard_header_len;netdev_lockdep_set_classes (dev);
Signature · 1 line
2.35.1.616.g0bdcbb4464-goog
PATCH v4 13/14 mlx4: support BIG TCP packets
In reply to Eric DumazetFrom: Eric Dumazet <[email protected]>
mlx4 supports LSOv2 just fine.
IPv6 stack inserts a temporary Hop-by-Hop header
with JUMBO TLV for big packets.
We need to ignore the HBH header when populating TX descriptor.
Tested:
Before: (not enabling bigger TSO/GRO packets)
ip link set dev eth0 gso_ipv6_max_size 65536 gro_ipv6_max_size 65536
netperf -H lpaa18 -t TCP_RR -T2,2 -l 10 -Cc -- -r 70000,70000
MIGRATED TCP REQUEST/RESPONSE TEST from ::0 (::) port 0 AF_INET6 to lpaa18.prod.google.com () port 0 AF_INET6 : first burst 0 : cpu bind Local /Remote
Socket Size Request Resp. Elapsed Trans. CPU CPU S.dem S.dem Send Recv Size Size Time Rate local remote local remote bytes bytes bytes bytes secs. per sec % S % S us/Tr us/Tr
262144 540000 70000 70000 10.00 6591.45 0.86 1.34 62.490 97.446
262144 540000
After: (enabling bigger TSO/GRO packets)
ip link set dev eth0 gso_ipv6_max_size 185000 gro_ipv6_max_size 185000
netperf -H lpaa18 -t TCP_RR -T2,2 -l 10 -Cc -- -r 70000,70000
MIGRATED TCP REQUEST/RESPONSE TEST from ::0 (::) port 0 AF_INET6 to lpaa18.prod.google.com () port 0 AF_INET6 : first burst 0 : cpu bind Local /Remote
Socket Size Request Resp. Elapsed Trans. CPU CPU S.dem S.dem Send Recv Size Size Time Rate local remote local remote bytes bytes bytes bytes secs. per sec % S % S us/Tr us/Tr
262144 540000 70000 70000 10.00 8383.95 0.95 1.01 54.432 57.584
262144 540000
--- .../net/ethernet/mellanox/mlx4/en_netdev.c | 3 ++ drivers/net/ethernet/mellanox/mlx4/en_tx.c | 47 +++++++++++++++---- 2 files changed, 41 insertions(+), 9 deletions(-)
drivers/net/ethernet/mellanox/mlx4/en_netdev.c +3 -0
@@ -3417,6 +3417,9 @@ int mlx4_en_init_netdev(struct mlx4_en_dev *mdev, int port, dev->min_mtu =ETH_MIN_MTU ; dev->max_mtu = priv->max_mtu; +/* supports LSOv2 packets, 512KB limit has been tested. */ +netif_set_tso_ipv6_max_size (dev,512 *1024 ); + mdev->pndev[port] = dev; mdev->upper[port] =NULL ;
drivers/net/ethernet/mellanox/mlx4/en_tx.c +38 -9
103 lines not on this page · raw message
Signature · 1 line
2.35.1.616.g0bdcbb4464-goog
PATCH v4 14/14 mlx5: support BIG TCP packets
In reply to Eric DumazetFrom: Coco Li <[email protected]>
mlx5 supports LSOv2.
IPv6 gro/tcp stacks insert a temporary Hop-by-Hop header with JUMBO TLV for big packets.
We need to ignore/skip this HBH header when populating TX descriptor.
Note that ipv6_has_hopopt_jumbo() only recognizes very specific packet layout, thus mlx5e_sq_xmit_wqe() is taking care of this layout only.
v2: clear hopbyhop in mlx5e_tx_get_gso_ihs()
v4: fix compile error for CONFIG_MLX5_CORE_IPOIB=y
--- .../net/ethernet/mellanox/mlx5/core/en_main.c | 1 + .../net/ethernet/mellanox/mlx5/core/en_tx.c | 84 +++++++++++++++---- 2 files changed, 69 insertions(+), 16 deletions(-)
drivers/net/ethernet/mellanox/mlx5/core/en_main.c +1 -0
@@ -4910,6 +4910,7 @@ static void mlx5e_build_nic_netdev(struct net_device *netdev) netdev->priv_flags |=IFF_UNICAST_FLT ; +netif_set_tso_ipv6_max_size (netdev,512 *1024 );mlx5e_set_netdev_dev_addr (netdev);mlx5e_ipsec_build_netdev (priv);mlx5e_tls_build_netdev (priv);
drivers/net/ethernet/mellanox/mlx5/core/en_tx.c +68 -16
158 lines not on this page · raw message
Signature · 1 line
2.35.1.616.g0bdcbb4464-goog
Quoted 51 lines · Eric Dumazet
From: Eric Dumazet <[email protected]>
This series implements BIG TCP as presented in netdev
0x15:https://netdevconf.info/0x15/session.html?BIG-TCP
Jonathan Corbet made a nice summary: https://lwn.net/Articles/884104/
Standard TSO/GRO packet limit is 64KB
With BIG TCP, we allow bigger TSO/GRO packet sizes for IPv6 traffic.
Note that this feature is by default not enabled, because it might break some eBPF programs assuming TCP header immediately follows IPv6 header.
While tcpdump recognizes the HBH/Jumbo header, standard pcap filters are unable to skip over IPv6 extension headers.
Reducing number of packets traversing networking stack usually improves performance, as shown on this experiment using a 100Gbit NIC, and 4K MTU.
'Standard' performance with current (74KB) limits.
for i in {1..10}; do ./netperf -tTCP_RR-H iroa23 -- -r80000,80000 -O MIN_LATENCY,P90_LATENCY,P99_LATENCY,THROUGHPUT|tail -1; done77 138 183 8542.19 79 143 178 8215.28 70 117 164 9543.39 80 144 176 8183.71 78 126 155 9108.47 80 146 184 8115.19 71 113 165 9510.96 74 113 164 9518.74 79 137 178 8575.04 73 111 171 9561.73Now enable BIG TCP on both hosts.
ip link set dev eth0
gro_ipv6_max_size185000gso_ipv6_max_size185000
for i in {1..10}; do ./netperf -tTCP_RR-H iroa23 -- -r80000,80000 -O MIN_LATENCY,P90_LATENCY,P99_LATENCY,THROUGHPUT|tail -1; done57 83 117 13871.38 64 118 155 11432.94 65 116 148 11507.62 60 105 136 12645.15 60 103 135 12760.34 60 102 134 12832.64 62 109 132 10877.68 58 82 115 14052.93 57 83 124 14212.58 57 82 119 14196.01We see an increase of transactions per second, and lower latencies as well.
Quoted 7 lines
v4: fix compile error for
CONFIG_MLX5_CORE_IPOIB=yin mlx5 (Jakub)v3: Fixed a typo in RFC number (Alexander)
Added Reviewed-by: tags from Tariq on mlx4/mlx5 parts.v2: Removed the
MAX_SKB_FRAGSchange, this belongs to a different series.
Addressed feedback, for Alexander and nvidia folks.
One concern with this patch set is the addition of all the max_size netdev attributes for tsov6, gsov6, and grov6. For the gsov6 and grov6 maxes I really think these make more sense as sysctl values since it feels more like a protocol change rather than a netdev specific one.
If I recall correctly the addition of gso_max_size and gso_max_segs were added as a workaround for NICs that couldn't handle offloading frames larger than a certain size. This feels like increasing the scope of the workaround rather than adding a new feature.
I didn't see the patch that went by for gro_max_size but I am not a fan of the way it was added since it would make more sense as a sysctl which controlled the stack instead of something that is device specific since as far as the device is concerned it received MTU size frames, and GRO happens above the device. I suppose it makes things symmetric with gso_max_size, but at the same time it isn't really a device specific attribute since the work happens in the stack above the device.
Do we need to add the IPv6 specific version of the tso_ipv6_max_size? Could we instead just allow setting the gso_max_size value larger than 64K? Then it would just be a matter of having a protocol specific max size check to pull us back down to GSO_MAX_SIZE in the case of non-ipv6 frames.
Quoted 71 lines · Alexander H Duyck
Quoted 51 lines · Eric Dumazet
From: Eric Dumazet <[email protected]>
This series implements BIG TCP as presented in netdev
0x15:https://netdevconf.info/0x15/session.html?BIG-TCP
Jonathan Corbet made a nice summary: https://lwn.net/Articles/884104/
Standard TSO/GRO packet limit is 64KB
With BIG TCP, we allow bigger TSO/GRO packet sizes for IPv6 traffic.
Note that this feature is by default not enabled, because it might break some eBPF programs assuming TCP header immediately follows IPv6 header.
While tcpdump recognizes the HBH/Jumbo header, standard pcap filters are unable to skip over IPv6 extension headers.
Reducing number of packets traversing networking stack usually improves performance, as shown on this experiment using a 100Gbit NIC, and 4K MTU.
'Standard' performance with current (74KB) limits.
for i in {1..10}; do ./netperf -tTCP_RR-H iroa23 -- -r80000,80000 -O MIN_LATENCY,P90_LATENCY,P99_LATENCY,THROUGHPUT|tail -1; done77 138 183 8542.19 79 143 178 8215.28 70 117 164 9543.39 80 144 176 8183.71 78 126 155 9108.47 80 146 184 8115.19 71 113 165 9510.96 74 113 164 9518.74 79 137 178 8575.04 73 111 171 9561.73Now enable BIG TCP on both hosts.
ip link set dev eth0
gro_ipv6_max_size185000gso_ipv6_max_size185000
for i in {1..10}; do ./netperf -tTCP_RR-H iroa23 -- -r80000,80000 -O MIN_LATENCY,P90_LATENCY,P99_LATENCY,THROUGHPUT|tail -1; done57 83 117 13871.38 64 118 155 11432.94 65 116 148 11507.62 60 105 136 12645.15 60 103 135 12760.34 60 102 134 12832.64 62 109 132 10877.68 58 82 115 14052.93 57 83 124 14212.58 57 82 119 14196.01We see an increase of transactions per second, and lower latencies as well.
Quoted 7 lines
v4: fix compile error for
CONFIG_MLX5_CORE_IPOIB=yin mlx5 (Jakub)v3: Fixed a typo in RFC number (Alexander)
Added Reviewed-by: tags from Tariq on mlx4/mlx5 parts.v2: Removed the
MAX_SKB_FRAGSchange, this belongs to a different series.
Addressed feedback, for Alexander and nvidia folks.One concern with this patch set is the addition of all the
max_sizenetdev attributes for tsov6, gsov6, and grov6. For the gsov6 and grov6 maxes I really think these make more sense as sysctl values since it feels more like a protocol change rather than a netdev specific one.If I recall correctly the addition of
gso_max_sizeandgso_max_segswere added as a workaround for NICs that couldn't handle offloading frames larger than a certain size. This feels like increasing the scope of the workaround rather than adding a new feature.
Quoted 9 lines
I didn't see the patch that went by for
gro_max_sizebut I am not a fan of the way it was added since it would make more sense as a sysctl which controlled the stack instead of something that is device specific since as far as the device is concerned it received MTU size frames, and GRO happens above the device. I suppose it makes things symmetric withgso_max_size, but at the same time it isn't really a device specific attribute since the work happens in the stack above the device.
We already have per device gso_max_size and gso_max_segs.
GRO max size being per device is nice. There are cases where a host has multiple NIC,
one of them being used for incoming traffic that needs to be forwarded.
Maybe the changelog was not clear enough, but being able to lower gro_max_size is also a way to prevent frag_list being used, so that most NIC support TSO just fine.
Quoted 5 lines
Do we need to add the IPv6 specific version of the
tso_ipv6_max_size? Could we instead just allow setting thegso_max_sizevalue larger than 64K? Then it would just be a matter of having a protocol specific max size check to pull us back down toGSO_MAX_SIZEin the case of non-ipv6 frames.
Not sure why adding attributes is an issue really, more flexibility seems better to me.
One day, if someone adds LSOv2 to IPv4, I prefer being able to selectively turn on this support,
after tests have concluded nothing broke.
Having to turn off LSOv2 in emergency because of some bug in LSOv2 IPv4 implementation would be bad.
Quoted 86 lines · Eric Dumazet
Quoted 71 lines · Alexander H Duyck
Quoted 51 lines · Eric Dumazet
From: Eric Dumazet <[email protected]>
This series implements BIG TCP as presented in netdev
0x15:https://netdevconf.info/0x15/session.html?BIG-TCP
Jonathan Corbet made a nice summary: https://lwn.net/Articles/884104/
Standard TSO/GRO packet limit is 64KB
With BIG TCP, we allow bigger TSO/GRO packet sizes for IPv6 traffic.
Note that this feature is by default not enabled, because it might break some eBPF programs assuming TCP header immediately follows IPv6 header.
While tcpdump recognizes the HBH/Jumbo header, standard pcap filters are unable to skip over IPv6 extension headers.
Reducing number of packets traversing networking stack usually improves performance, as shown on this experiment using a 100Gbit NIC, and 4K MTU.
'Standard' performance with current (74KB) limits.
for i in {1..10}; do ./netperf -tTCP_RR-H iroa23 -- -r80000,80000 -O MIN_LATENCY,P90_LATENCY,P99_LATENCY,THROUGHPUT|tail -1; done77 138 183 8542.19 79 143 178 8215.28 70 117 164 9543.39 80 144 176 8183.71 78 126 155 9108.47 80 146 184 8115.19 71 113 165 9510.96 74 113 164 9518.74 79 137 178 8575.04 73 111 171 9561.73Now enable BIG TCP on both hosts.
ip link set dev eth0
gro_ipv6_max_size185000gso_ipv6_max_size185000
for i in {1..10}; do ./netperf -tTCP_RR-H iroa23 -- -r80000,80000 -O MIN_LATENCY,P90_LATENCY,P99_LATENCY,THROUGHPUT|tail -1; done57 83 117 13871.38 64 118 155 11432.94 65 116 148 11507.62 60 105 136 12645.15 60 103 135 12760.34 60 102 134 12832.64 62 109 132 10877.68 58 82 115 14052.93 57 83 124 14212.58 57 82 119 14196.01We see an increase of transactions per second, and lower latencies as well.
Quoted 7 lines
v4: fix compile error for
CONFIG_MLX5_CORE_IPOIB=yin mlx5 (Jakub)v3: Fixed a typo in RFC number (Alexander)
Added Reviewed-by: tags from Tariq on mlx4/mlx5 parts.v2: Removed the
MAX_SKB_FRAGSchange, this belongs to a different series.
Addressed feedback, for Alexander and nvidia folks.One concern with this patch set is the addition of all the
max_sizenetdev attributes for tsov6, gsov6, and grov6. For the gsov6 and grov6 maxes I really think these make more sense as sysctl values since it feels more like a protocol change rather than a netdev specific one.If I recall correctly the addition of
gso_max_sizeandgso_max_segswere added as a workaround for NICs that couldn't handle offloading frames larger than a certain size. This feels like increasing the scope of the workaround rather than adding a new feature.Quoted 9 lines
I didn't see the patch that went by for
gro_max_sizebut I am not a fan of the way it was added since it would make more sense as a sysctl which controlled the stack instead of something that is device specific since as far as the device is concerned it received MTU size frames, and GRO happens above the device. I suppose it makes things symmetric withgso_max_size, but at the same time it isn't really a device specific attribute since the work happens in the stack above the device.We already have per device
gso_max_sizeandgso_max_segs.
Quoted 7 lines
GRO max size being per device is nice. There are cases where a host has multiple NIC,
one of them being used for incoming traffic that needs to be forwarded.Maybe the changelog was not clear enough, but being able to lower
gro_max_sizeis also a way to preventfrag_listbeing used, so that most NIC support TSO just fine.
The point is gso_max_size and gso_max_segs were workarounds for devices. Basically they weren't added until it was found that specific NICs were having issues with segmenting frames either larger than a specific size or number of segments.
I suppose we can keep gro_max_size in place if we are wanting to say that it ties back into the device. There may be some benefit there if we end up with some devices allocating skbs that can aggregate more segments than others, however in that case that seems more like a segment limit than a size limit. Maybe something like gro_max_segs would make more sense, or I suppose we end up with both eventually.
Quoted 9 lines
Quoted 5 lines
Do we need to add the IPv6 specific version of the
tso_ipv6_max_size? Could we instead just allow setting thegso_max_sizevalue larger than 64K? Then it would just be a matter of having a protocol specific max size check to pull us back down toGSO_MAX_SIZEin the case of non-ipv6 frames.Not sure why adding attributes is an issue really, more flexibility seems better to me.
Quoted 6 lines
One day, if someone adds LSOv2 to IPv4, I prefer being able to selectively turn on this support,
after tests have concluded nothing broke.Having to turn off LSOv2 in emergency because of some bug in LSOv2 IPv4 implementation would be bad.
You already have a means of turning off LSOv2 in the form of gso_max_size. Remember it was put there as a workaround for broken devices that couldn't fully support LSOv1. In my mind we would reuse the gso_max_size, and I suppose gro_max_size to control the GSO types supported by the device. If it is set for 64K or less then it only supports LSOv1, if it is set higher, then it can support GSOv2. I'm not sure it makes sense to split gso_max_size into two versions. If a device supports GSO larger than 64K then what is the point in having gso_max_size around when it will always be set to 64K because the device isn't broken. Otherwise we are going to end up creating a bunch of duplications.
What I am getting at is that it would be nice to have a stack level control and then a device level control. The device control is there to say what size drivers support when segmenting, whereas the stack level control says if the protocol wants to try sending down frames larger than 64K. So essentially all non-IPv6 protocols will cap at 64K, whereas IPv6 can go beyond that with huge frames. Then you get the best of both worlds.
Quoted 92 lines · Alexander Duyck
Quoted 90 lines · Eric Dumazet
Quoted 71 lines · Alexander H Duyck
Quoted 51 lines · Eric Dumazet
From: Eric Dumazet <[email protected]>
This series implements BIG TCP as presented in netdev
0x15:https://netdevconf.info/0x15/session.html?BIG-TCP
Jonathan Corbet made a nice summary: https://lwn.net/Articles/884104/
Standard TSO/GRO packet limit is 64KB
With BIG TCP, we allow bigger TSO/GRO packet sizes for IPv6 traffic.
Note that this feature is by default not enabled, because it might break some eBPF programs assuming TCP header immediately follows IPv6 header.
While tcpdump recognizes the HBH/Jumbo header, standard pcap filters are unable to skip over IPv6 extension headers.
Reducing number of packets traversing networking stack usually improves performance, as shown on this experiment using a 100Gbit NIC, and 4K MTU.
'Standard' performance with current (74KB) limits.
for i in {1..10}; do ./netperf -tTCP_RR-H iroa23 -- -r80000,80000 -O MIN_LATENCY,P90_LATENCY,P99_LATENCY,THROUGHPUT|tail -1; done77 138 183 8542.19 79 143 178 8215.28 70 117 164 9543.39 80 144 176 8183.71 78 126 155 9108.47 80 146 184 8115.19 71 113 165 9510.96 74 113 164 9518.74 79 137 178 8575.04 73 111 171 9561.73Now enable BIG TCP on both hosts.
ip link set dev eth0
gro_ipv6_max_size185000gso_ipv6_max_size185000
for i in {1..10}; do ./netperf -tTCP_RR-H iroa23 -- -r80000,80000 -O MIN_LATENCY,P90_LATENCY,P99_LATENCY,THROUGHPUT|tail -1; done57 83 117 13871.38 64 118 155 11432.94 65 116 148 11507.62 60 105 136 12645.15 60 103 135 12760.34 60 102 134 12832.64 62 109 132 10877.68 58 82 115 14052.93 57 83 124 14212.58 57 82 119 14196.01We see an increase of transactions per second, and lower latencies as well.
Quoted 7 lines
v4: fix compile error for
CONFIG_MLX5_CORE_IPOIB=yin mlx5 (Jakub)v3: Fixed a typo in RFC number (Alexander)
Added Reviewed-by: tags from Tariq on mlx4/mlx5 parts.v2: Removed the
MAX_SKB_FRAGSchange, this belongs to a different series.
Addressed feedback, for Alexander and nvidia folks.One concern with this patch set is the addition of all the
max_sizenetdev attributes for tsov6, gsov6, and grov6. For the gsov6 and grov6 maxes I really think these make more sense as sysctl values since it feels more like a protocol change rather than a netdev specific one.If I recall correctly the addition of
gso_max_sizeandgso_max_segswere added as a workaround for NICs that couldn't handle offloading frames larger than a certain size. This feels like increasing the scope of the workaround rather than adding a new feature.Quoted 9 lines
I didn't see the patch that went by for
gro_max_sizebut I am not a fan of the way it was added since it would make more sense as a sysctl which controlled the stack instead of something that is device specific since as far as the device is concerned it received MTU size frames, and GRO happens above the device. I suppose it makes things symmetric withgso_max_size, but at the same time it isn't really a device specific attribute since the work happens in the stack above the device.We already have per device
gso_max_sizeandgso_max_segs.GRO max size being per device is nice. There are cases where a host has multiple NIC,
one of them being used for incoming traffic that needs to be forwarded.
Quoted 8 lines
Quoted 3 lines
Maybe the changelog was not clear enough, but being able to lower
gro_max_sizeis also a way to preventfrag_listbeing used, so that most NIC support TSO just fine.The point is
gso_max_sizeandgso_max_segswere workarounds for devices. Basically they weren't added until it was found that specific NICs were having issues with segmenting frames either larger than a specific size or number of segments.
These settings are used in our tests, when we want to precisely control size of TSO.
There are not only used by drivers.
Quoted 24 lines
I suppose we can keep
gro_max_sizein place if we are wanting to say that it ties back into the device. There may be some benefit there if we end up with some devices allocating skbs that can aggregate more segments than others, however in that case that seems more like a segment limit than a size limit. Maybe something likegro_max_segswould make more sense, or I suppose we end up with both eventually.Quoted 9 lines
Quoted 5 lines
Do we need to add the IPv6 specific version of the
tso_ipv6_max_size? Could we instead just allow setting thegso_max_sizevalue larger than 64K? Then it would just be a matter of having a protocol specific max size check to pull us back down toGSO_MAX_SIZEin the case of non-ipv6 frames.Not sure why adding attributes is an issue really, more flexibility seems better to me.
Quoted 6 lines
One day, if someone adds LSOv2 to IPv4, I prefer being able to selectively turn on this support,
after tests have concluded nothing broke.Having to turn off LSOv2 in emergency because of some bug in LSOv2 IPv4 implementation would be bad.
Quoted 11 lines
You already have a means of turning off LSOv2 in the form of
gso_max_size. Remember it was put there as a workaround for broken devices that couldn't fully support LSOv1. In my mind we would reuse thegso_max_size, and I supposegro_max_sizeto control the GSO types supported by the device. If it is set for 64K or less then it only supports LSOv1, if it is set higher, then it can support GSOv2. I'm not sure it makes sense to splitgso_max_sizeinto two versions. If a device supports GSO larger than 64K then what is the point in havinggso_max_sizearound when it will always be set to 64K because the device isn't broken. Otherwise we are going to end up creating a bunch of duplications.
OK, but if a buggy user script is currently doing this:
ip link set dev eth1 gso_max_size 70000
Old kernels were ignoring this request.
Suddenly a new kernel comes, and user ends up with a broken setup.
Quoted 8 lines
What I am getting at is that it would be nice to have a stack level control and then a device level control. The device control is there to say what size drivers support when segmenting, whereas the stack level control says if the protocol wants to try sending down frames larger than 64K. So essentially all non-IPv6 protocols will cap at 64K, whereas IPv6 can go beyond that with huge frames. Then you get the best of both worlds.
Can you explain how the per ipvlan setting would be allowed ?
To me sysctls are the old ways of controlling things, not mentioning adding more of them slow down netns creation and dismantles.
Quoted 104 lines · Eric Dumazet
Quoted 92 lines · Alexander Duyck
Quoted 90 lines · Eric Dumazet
Quoted 71 lines · Alexander H Duyck
Quoted 51 lines · Eric Dumazet
From: Eric Dumazet <[email protected]>
This series implements BIG TCP as presented in netdev
0x15:https://netdevconf.info/0x15/session.html?BIG-TCP
Jonathan Corbet made a nice summary: https://lwn.net/Articles/884104/
Standard TSO/GRO packet limit is 64KB
With BIG TCP, we allow bigger TSO/GRO packet sizes for IPv6 traffic.
Note that this feature is by default not enabled, because it might break some eBPF programs assuming TCP header immediately follows IPv6 header.
While tcpdump recognizes the HBH/Jumbo header, standard pcap filters are unable to skip over IPv6 extension headers.
Reducing number of packets traversing networking stack usually improves performance, as shown on this experiment using a 100Gbit NIC, and 4K MTU.
'Standard' performance with current (74KB) limits.
for i in {1..10}; do ./netperf -tTCP_RR-H iroa23 -- -r80000,80000 -O MIN_LATENCY,P90_LATENCY,P99_LATENCY,THROUGHPUT|tail -1; done77 138 183 8542.19 79 143 178 8215.28 70 117 164 9543.39 80 144 176 8183.71 78 126 155 9108.47 80 146 184 8115.19 71 113 165 9510.96 74 113 164 9518.74 79 137 178 8575.04 73 111 171 9561.73Now enable BIG TCP on both hosts.
ip link set dev eth0
gro_ipv6_max_size185000gso_ipv6_max_size185000
for i in {1..10}; do ./netperf -tTCP_RR-H iroa23 -- -r80000,80000 -O MIN_LATENCY,P90_LATENCY,P99_LATENCY,THROUGHPUT|tail -1; done57 83 117 13871.38 64 118 155 11432.94 65 116 148 11507.62 60 105 136 12645.15 60 103 135 12760.34 60 102 134 12832.64 62 109 132 10877.68 58 82 115 14052.93 57 83 124 14212.58 57 82 119 14196.01We see an increase of transactions per second, and lower latencies as well.
Quoted 7 lines
v4: fix compile error for
CONFIG_MLX5_CORE_IPOIB=yin mlx5 (Jakub)v3: Fixed a typo in RFC number (Alexander)
Added Reviewed-by: tags from Tariq on mlx4/mlx5 parts.v2: Removed the
MAX_SKB_FRAGSchange, this belongs to a different series.
Addressed feedback, for Alexander and nvidia folks.One concern with this patch set is the addition of all the
max_sizenetdev attributes for tsov6, gsov6, and grov6. For the gsov6 and grov6 maxes I really think these make more sense as sysctl values since it feels more like a protocol change rather than a netdev specific one.If I recall correctly the addition of
gso_max_sizeandgso_max_segswere added as a workaround for NICs that couldn't handle offloading frames larger than a certain size. This feels like increasing the scope of the workaround rather than adding a new feature.Quoted 9 lines
I didn't see the patch that went by for
gro_max_sizebut I am not a fan of the way it was added since it would make more sense as a sysctl which controlled the stack instead of something that is device specific since as far as the device is concerned it received MTU size frames, and GRO happens above the device. I suppose it makes things symmetric withgso_max_size, but at the same time it isn't really a device specific attribute since the work happens in the stack above the device.We already have per device
gso_max_sizeandgso_max_segs.GRO max size being per device is nice. There are cases where a host has multiple NIC,
one of them being used for incoming traffic that needs to be forwarded.Quoted 8 lines
Quoted 3 lines
Maybe the changelog was not clear enough, but being able to lower
gro_max_sizeis also a way to preventfrag_listbeing used, so that most NIC support TSO just fine.The point is
gso_max_sizeandgso_max_segswere workarounds for devices. Basically they weren't added until it was found that specific NICs were having issues with segmenting frames either larger than a specific size or number of segments.
Quoted 4 lines
These settings are used in our tests, when we want to precisely control size of TSO.
There are not only used by drivers.
Yes, but what I am getting at is that for most drivers they are always set to 64K. I get that you are using them to test things, and you still could still use them after the change. All I am asking for is us not to fork them into 3 different things where each one has a specific protocol it supports.
Quoted 36 lines
Quoted 24 lines
I suppose we can keep
gro_max_sizein place if we are wanting to say that it ties back into the device. There may be some benefit there if we end up with some devices allocating skbs that can aggregate more segments than others, however in that case that seems more like a segment limit than a size limit. Maybe something likegro_max_segswould make more sense, or I suppose we end up with both eventually.Quoted 9 lines
Quoted 5 lines
Do we need to add the IPv6 specific version of the
tso_ipv6_max_size? Could we instead just allow setting thegso_max_sizevalue larger than 64K? Then it would just be a matter of having a protocol specific max size check to pull us back down toGSO_MAX_SIZEin the case of non-ipv6 frames.Not sure why adding attributes is an issue really, more flexibility seems better to me.
Quoted 6 lines
One day, if someone adds LSOv2 to IPv4, I prefer being able to selectively turn on this support,
after tests have concluded nothing broke.Having to turn off LSOv2 in emergency because of some bug in LSOv2 IPv4 implementation would be bad.
Quoted 11 lines
You already have a means of turning off LSOv2 in the form of
gso_max_size. Remember it was put there as a workaround for broken devices that couldn't fully support LSOv1. In my mind we would reuse thegso_max_size, and I supposegro_max_sizeto control the GSO types supported by the device. If it is set for 64K or less then it only supports LSOv1, if it is set higher, then it can support GSOv2. I'm not sure it makes sense to splitgso_max_sizeinto two versions. If a device supports GSO larger than 64K then what is the point in havinggso_max_sizearound when it will always be set to 64K because the device isn't broken. Otherwise we are going to end up creating a bunch of duplications.
Quoted 7 lines
OK, but if a buggy user script is currently doing this:
ip link set dev eth1
gso_max_size70000Old kernels were ignoring this request.
Suddenly a new kernel comes, and user ends up with a broken setup.
That could already happen. Recall that I mentioned this was added as a workaround. So I would argue the code was already broken as there were devices that didn't support 64K and somehow we are allowing that.
We probably need to add a function call to the drivers in order to control the setting of this as there are some drivers that cannot support the current limit of 64K. Then if it isn't set we just default to 64K. For the drivers that support larger we can then overwrite the function when setting up the jumbogram segmentation support value.
Quoted 9 lines
Quoted 8 lines
What I am getting at is that it would be nice to have a stack level control and then a device level control. The device control is there to say what size drivers support when segmenting, whereas the stack level control says if the protocol wants to try sending down frames larger than 64K. So essentially all non-IPv6 protocols will cap at 64K, whereas IPv6 can go beyond that with huge frames. Then you get the best of both worlds.
Quoted 4 lines
Can you explain how the per ipvlan setting would be allowed ?
To me sysctls are the old ways of controlling things, not mentioning adding more of them slow down netns creation and dismantles.
So what I am suggesting is splitting it into two pieces. One being gso_max_size which can be set to values larger than 64K if that is what we want to support on that device. The other being a protocol specific value where you can configure things to use a value greater than 64K if that is supported by the protocol. That way if there is a problem in the stack you can turn it off in one place, and if there is a problem in the device you can turn it off in another.