BIG TCP for UDP tunnels
Subsystems (17)
cover BIG TCP for UDP tunnels
This series consists of two parts that will be submitted separately:
01-11: Remove hop-by-hop header for BIG TCP IPv6.
12-17: Fix up things that prevent BIG TCP from working with tunnels.
I kept them both here for the sake of big picture.
There are a few places that make assumptions about skb->len being smaller than 64k and/or that store it in 16-bit fields, trimming the length. The first step to enable BIG TCP with VXLAN and GENEVE tunnels is to patch those places to handle bigger lengths properly (patches 12-17). This is enough to make IPv4 in IPv4 work with BIG TCP, but when either the outer or the inner protocol is IPv6, the current BIG TCP code inserts a hop-by-hop extension header that stores the actual 32-bit length of the packet.
This additional hop-by-hop header turns out problematic for encapsulated cases, because:
1. The drivers don't strip it, and they'd all need to know the structure of each tunnel protocol in order to strip it correctly.
2. Even if (1) is implemented, it would be an additional performance penalty per aggregated packet.
3. The skb_gso_validate_network_len check is skipped in ip6_finish_output_gso when IP6SKB_FAKEJUMBO is set, but it seems that it would make sense to do the actual validation, just taking into account the length of the HBH header. When the support for tunnels is added, it becomes trickier, because there may be one or two HBH headers, depending on whether it's IPv6 in IPv6 or not.
At the same time, having an HBH header to store the 32-bit length is not strictly necessary, as BIG TCP IPv4 doesn't do anything like this and just restores the length from skb->len. The same thing can be done for BIG TCP IPv6 (patches 01-11).
The only reason why we keep inserting HBH seems to be for the tools that parse the packets, but the above drawbacks seem to outweigh this, and the tools can be patched (like they need to, in order to be able to parse BIG TCP IPv4 now). I have a patch for tcpdump.
Removing HBH from BIG TCP would allow to simplify the implementation significantly, and align it with BIG TCP IPv4.
Daniel Borkmann (1):
geneve: Enable BIG TCP packets
Maxim Mikityanskiy (16):
net/ipv6: Introduce payload_len helpers net/ipv6: Drop HBH for BIG TCP on TX side net/ipv6: Drop HBH for BIG TCP on RX side net/ipv6: Remove jumbo_remove step from TX path net/mlx5e: Remove jumbo_remove step from TX path net/mlx4: Remove jumbo_remove step from TX path
ice: Remove jumbo_remove step from TX path
bnxt_en: Remove jumbo_remove step from TX path
gve: Remove jumbo_remove step from TX path
net: mana: Remove jumbo_remove step from TX path net/ipv6: Remove HBH helpers
net: Enable BIG TCP with partial GSO
udp: Support gro_ipv4_max_size > 65536
udp: Validate UDP length in udp_gro_receive
udp: Set length in UDP header to 0 for big GSO packets
vxlan: Enable BIG TCP packets
drivers/net/ethernet/broadcom/bnxt/bnxt.c | 21 ----- drivers/net/ethernet/google/gve/gve_tx_dqo.c | 3 - drivers/net/ethernet/intel/ice/ice_txrx.c | 3 - drivers/net/ethernet/mellanox/mlx4/en_tx.c | 42 ++-------- .../net/ethernet/mellanox/mlx5/core/en_tx.c | 75 +++--------------- drivers/net/ethernet/microsoft/mana/mana_en.c | 3 - drivers/net/geneve.c | 2 + drivers/net/vxlan/vxlan_core.c | 2 + include/linux/ipv6.h | 21 ++++- include/net/ipv6.h | 79 ------------------- include/net/netfilter/nf_tables_ipv6.h | 4 +- net/bridge/br_netfilter_ipv6.c | 2 +- net/bridge/netfilter/nf_conntrack_bridge.c | 4 +- net/core/dev.c | 3 +- net/core/gro.c | 2 - net/core/skbuff.c | 10 +-- net/ipv4/udp.c | 5 +- net/ipv4/udp_offload.c | 12 ++- net/ipv4/udp_tunnel_core.c | 2 +- net/ipv6/ip6_input.c | 2 +- net/ipv6/ip6_offload.c | 36 +-------- net/ipv6/ip6_output.c | 20 +---- net/ipv6/ip6_udp_tunnel.c | 2 +- net/ipv6/output_core.c | 7 +- net/netfilter/ipvs/ip_vs_xmit.c | 2 +- net/netfilter/nf_conntrack_ovs.c | 2 +- net/netfilter/nf_log_syslog.c | 2 +- net/sched/sch_cake.c | 2 +- 28 files changed, 83 insertions(+), 287 deletions(-)
Signature · 1 line
2.49.0
RFC PATCH 1/17 net/ipv6: Introduce payload_len helpers
From: Maxim Mikityanskiy <[email protected]>
The next commits will transition away from using the hop-by-hop extension header to encode packet length for BIG TCP. Add wrappers around ip6->payload_len that return the actual value if it's non-zero, and calculate it from skb->len if payload_len is set to zero (and a symmetrical setter).
The new helpers are used wherever the surrounding code supports the hop-by-hop jumbo header for BIG TCP IPv6, or the corresponding IPv4 code uses skb_ip_totlen (e.g., in include/net/netfilter/nf_tables_ipv6.h).
No behavioral change in this commit.
--- include/linux/ipv6.h | 20 ++++++++++++++++++++ include/net/ipv6.h | 2 -- include/net/netfilter/nf_tables_ipv6.h | 4 ++-- net/bridge/br_netfilter_ipv6.c | 2 +- net/bridge/netfilter/nf_conntrack_bridge.c | 4 ++-- net/ipv6/ip6_input.c | 2 +- net/ipv6/ip6_offload.c | 7 +++---- net/ipv6/output_core.c | 7 +------ net/netfilter/ipvs/ip_vs_xmit.c | 2 +- net/netfilter/nf_conntrack_ovs.c | 2 +- net/netfilter/nf_log_syslog.c | 2 +- net/sched/sch_cake.c | 2 +- 12 files changed, 34 insertions(+), 22 deletions(-)
include/linux/ipv6.h +20 -0
@@ -125,6 +125,26 @@ static inline unsigned int ipv6_transport_len(const struct sk_buff *skb)skb_network_header_len (skb); } +static inline unsigned int ipv6_payload_len (const struct sk_buff *skb,const struct ipv6hdr *ip6) +{ +u32 len =ntohs (ip6->payload_len); + +return (len || !skb_is_gso (skb) || !skb_is_gso_tcp (skb)) ? + len : skb->len -skb_network_offset (skb) -sizeof (struct ipv6hdr ); +} + +static inline unsigned int skb_ipv6_payload_len (const struct sk_buff *skb) +{ +return ipv6_payload_len (skb,ipv6_hdr (skb)); +} + +#define IPV6_MAXPLEN 65535 + +static inline void ipv6_set_payload_len (struct ipv6hdr *ip6,unsigned int len) +{ + ip6->payload_len = len <=IPV6_MAXPLEN ?htons (len) :0 ; +} +/* This structure contains results of exthdrs parsing as offsets from skb->nh.
include/net/ipv6.h +0 -2
@@ -25,8 +25,6 @@ struct ip_tunnel_info;#define SIN6_LEN_RFC2133 24 -#define IPV6_MAXPLEN 65535 -/* * NextHeader field of IPv6 header */
include/net/netfilter/nf_tables_ipv6.h +2 -2
@@ -42,7 +42,7 @@ static inline int __nft_set_pktinfo_ipv6_validate(struct nft_pktinfo *pkt)if (ip6h->version !=6 )return -1 ; - pkt_len =ntohs (ip6h->payload_len); + pkt_len =ipv6_payload_len (pkt->skb, ip6h); skb_len = pkt->skb->len -skb_network_offset (pkt->skb);if (pkt_len +sizeof (*ip6h) > skb_len)return -1 ;
@@ -86,7 +86,7 @@ static inline int nft_set_pktinfo_ipv6_ingress(struct nft_pktinfo *pkt)if (ip6h->version !=6 ) goto inhdr_error; - pkt_len =ntohs (ip6h->payload_len); + pkt_len =ipv6_payload_len (pkt->skb, ip6h);if (pkt_len +sizeof (*ip6h) > pkt->skb->len) { idev =__in6_dev_get (nft_in (pkt));__IP6_INC_STATS (nft_net (pkt), idev,IPSTATS_MIB_INTRUNCATEDPKTS );
net/bridge/br_netfilter_ipv6.c +1 -1
@@ -58,7 +58,7 @@ int br_validate_ipv6(struct net *net, struct sk_buff *skb)if (hdr->version !=6 ) goto inhdr_error; - pkt_len =ntohs (hdr->payload_len); + pkt_len =ipv6_payload_len (skb, hdr);if (hdr->nexthdr ==NEXTHDR_HOP &&nf_ip6_check_hbh_len (skb, &pkt_len)) goto drop;
net/bridge/netfilter/nf_conntrack_bridge.c +2 -2
@@ -230,7 +230,7 @@ static int nf_ct_br_ipv6_check(const struct sk_buff *skb)if (hdr->version !=6 )return -1 ; - len =ntohs (hdr->payload_len) +sizeof (struct ipv6hdr ) + nhoff; + len =ipv6_payload_len (skb, hdr) +sizeof (struct ipv6hdr ) + nhoff;if (skb->len < len)return -1 ;
@@ -270,7 +270,7 @@ static unsigned int nf_ct_bridge_pre(void *priv, struct sk_buff *skb,if (!pskb_may_pull (skb,sizeof (struct ipv6hdr )))return NF_ACCEPT ; - len =sizeof (struct ipv6hdr ) +ntohs (ipv6_hdr (skb)->payload_len); + len =sizeof (struct ipv6hdr ) +skb_ipv6_payload_len (skb);if (pskb_trim_rcsum (skb, len))return NF_ACCEPT ;
net/ipv6/ip6_input.c +1 -1
@@ -260,7 +260,7 @@ static struct sk_buff *ip6_rcv_core(struct sk_buff *skb, struct net_device *dev, skb->transport_header = skb->network_header +sizeof (*hdr);IP6CB (skb)->nhoff = offsetof(struct ipv6hdr , nexthdr); - pkt_len =ntohs (hdr->payload_len); + pkt_len =ipv6_payload_len (skb, hdr);/* pkt_len may be zero if Jumbo payload option is present */ if (pkt_len || hdr->nexthdr !=NEXTHDR_HOP ) {
net/ipv6/ip6_offload.c +3 -4
@@ -370,12 +370,11 @@ INDIRECT_CALLABLE_SCOPE int ipv6_gro_complete(struct sk_buff *skb, int nhoff) hop_jumbo->jumbo_payload_len =htonl (payload_len + hoplen); iph->nexthdr =NEXTHDR_HOP ; - iph->payload_len =0 ; - }else { - iph = (struct ipv6hdr *)(skb->data + nhoff); - iph->payload_len =htons (payload_len); } + iph = (struct ipv6hdr *)(skb->data + nhoff); +ipv6_set_payload_len (iph, payload_len); + nhoff +=sizeof (*iph) +ipv6_exthdrs_len (iph, &ops);if (WARN_ON (!ops || !ops->callbacks.gro_complete)) goto out;
net/ipv6/output_core.c +1 -6
@@ -123,12 +123,7 @@ EXPORT_SYMBOL(ip6_dst_hoplimit);int __ip6_local_out (struct net *net,struct sock *sk,struct sk_buff *skb) { -int len; - - len = skb->len -sizeof (struct ipv6hdr ); -if (len >IPV6_MAXPLEN ) - len =0 ; -ipv6_hdr (skb)->payload_len =htons (len); +ipv6_set_payload_len (ipv6_hdr (skb), skb->len -sizeof (struct ipv6hdr ));IP6CB (skb)->nhoff = offsetof(struct ipv6hdr , nexthdr);/* if egress device is enslaved to an L3 master device pass the
net/netfilter/ipvs/ip_vs_xmit.c +1 -1
@@ -947,7 +947,7 @@ ip_vs_prepare_tunneled_skb(struct sk_buff *skb, int skb_af, *next_protocol =IPPROTO_IPV6 ;if (payload_len) *payload_len = -ntohs (old_ipv6h->payload_len) + +ipv6_payload_len (skb, old_ipv6h) +sizeof (*old_ipv6h); old_dsfield =ipv6_get_dsfield (old_ipv6h); *ttl = old_ipv6h->hop_limit;
net/netfilter/nf_conntrack_ovs.c +1 -1
@@ -121,7 +121,7 @@ int nf_ct_skb_network_trim(struct sk_buff *skb, int family) len =skb_ip_totlen (skb);break ;case NFPROTO_IPV6 : - len =ntohs (ipv6_hdr (skb)->payload_len); + len =skb_ipv6_payload_len (skb);if (ipv6_hdr (skb)->nexthdr ==NEXTHDR_HOP ) {int err =nf_ip6_check_hbh_len (skb, &len);
net/netfilter/nf_log_syslog.c +1 -1
@@ -561,7 +561,7 @@ dump_ipv6_packet(struct net *net, struct nf_log_buf *m,/* Max length: 44 "LEN=65535 TC=255 HOPLIMIT=255 FLOWLBL=FFFFF " */ nf_log_buf_add (m,"LEN=%zu TC=%u HOPLIMIT=%u FLOWLBL=%u " , -ntohs (ih->payload_len) +sizeof (struct ipv6hdr ), +ipv6_payload_len (skb, ih) +sizeof (struct ipv6hdr ), (ntohl (*(__be32 *)ih) &0x0ff00000 ) >>20 , ih->hop_limit, (ntohl (*(__be32 *)ih) &0x000fffff ));
net/sched/sch_cake.c +1 -1
@@ -1266,7 +1266,7 @@ static struct sk_buff *cake_ack_filter(struct cake_sched_data *q, ipv6_addr_cmp(&ipv6h_check->daddr, &ipv6h->daddr)) continue; - seglen =ntohs (ipv6h_check->payload_len); + seglen =ipv6_payload_len (skb, ipv6h_check); }else {WARN_ON (1 );/* shouldn't happen */ continue ;
Signature · 1 line
2.49.0
RFC PATCH 2/17 net/ipv6: Drop HBH for BIG TCP on TX side
In reply to Maxim MikityanskiyFrom: Maxim Mikityanskiy <[email protected]>
BIG TCP IPv6 inserts a hop-by-hop extension header to indicate the real IPv6 payload length when it doesn't fit into the 16-bit field in the IPv6 header itself. While it helps tools parse the packet, it also requires every driver that supports TSO and BIG TCP to remove this 8-byte extension header. It might not sound that bad until we try to apply it to tunneled traffic. Currently, the drivers don't attempt to strip HBH if skb->encapsulation = 1. Moreover, trying to do so would require dissecting different tunnel protocols and making corresponding adjustments on case-by-case basis, which would slow down the fastpath (potentially also requiring adjusting checksums in outer headers).
At the same time, BIG TCP IPv4 doesn't insert any extra headers and just calculates the payload length from skb->len, significantly simplifying implementing BIG TCP for tunnels.
Stop inserting HBH when building BIG TCP GSO SKBs.
--- include/linux/ipv6.h | 1 - net/ipv6/ip6_output.c | 20 +++----------------- 2 files changed, 3 insertions(+), 18 deletions(-)
include/linux/ipv6.h +0 -1
@@ -174,7 +174,6 @@ struct inet6_skb_parm {#define IP6SKB_L3SLAVE 64#define IP6SKB_JUMBOGRAM 128#define IP6SKB_SEG6 256 -#define IP6SKB_FAKEJUMBO 512#define IP6SKB_MULTIPATH 1024 };
net/ipv6/ip6_output.c +3 -17
@@ -185,8 +185,7 @@ ip6_finish_output_gso_slowpath_drop(struct net *net, struct sock *sk,static int ip6_finish_output_gso (struct net *net,struct sock *sk,struct sk_buff *skb,unsigned int mtu) { -if (!(IP6CB (skb)->flags &IP6SKB_FAKEJUMBO ) && - !skb_gso_validate_network_len (skb, mtu)) +if (!skb_gso_validate_network_len (skb, mtu))return ip6_finish_output_gso_slowpath_drop (net, sk, skb, mtu);return ip6_finish_output2 (net, sk, skb);
@@ -273,8 +272,6 @@ int ip6_xmit(const struct sock *sk, struct sk_buff *skb, struct flowi6 *fl6,struct dst_entry *dst =skb_dst (skb);struct net_device *dev = dst->dev;struct inet6_dev *idev =ip6_dst_idev (dst); -struct hop_jumbo_hdr *hop_jumbo; -int hoplen =sizeof (*hop_jumbo);unsigned int head_room;struct ipv6hdr *hdr;u8 proto = fl6->flowi6_proto;
@@ -282,7 +279,7 @@ int ip6_xmit(const struct sock *sk, struct sk_buff *skb, struct flowi6 *fl6,int hlimit =-1 ;u32 mtu; - head_room =sizeof (struct ipv6hdr ) + hoplen +LL_RESERVED_SPACE (dev); + head_room =sizeof (struct ipv6hdr ) +LL_RESERVED_SPACE (dev);if (opt) head_room += opt->opt_nflen + opt->opt_flen;
@@ -309,19 +306,8 @@ int ip6_xmit(const struct sock *sk, struct sk_buff *skb, struct flowi6 *fl6, &fl6->saddr); } -if (unlikely (seg_len >IPV6_MAXPLEN )) { - hop_jumbo =skb_push (skb, hoplen); - - hop_jumbo->nexthdr = proto; - hop_jumbo->hdrlen =0 ; - hop_jumbo->tlv_type =IPV6_TLV_JUMBO ; - hop_jumbo->tlv_len =4 ; - hop_jumbo->jumbo_payload_len =htonl (seg_len + hoplen); - - proto =IPPROTO_HOPOPTS ; +if (unlikely (seg_len >IPV6_MAXPLEN )) seg_len =0 ; -IP6CB (skb)->flags |=IP6SKB_FAKEJUMBO ; - }skb_push (skb,sizeof (struct ipv6hdr ));skb_reset_network_header (skb);
Signature · 1 line
2.49.0
RFC PATCH 3/17 net/ipv6: Drop HBH for BIG TCP on RX side
In reply to Maxim MikityanskiyFrom: Maxim Mikityanskiy <[email protected]>
Complementary to the previous commit, stop inserting HBH when building BIG TCP GRO SKBs.
--- net/core/gro.c | 2 -- net/ipv6/ip6_offload.c | 28 +--------------------------- 2 files changed, 1 insertion(+), 29 deletions(-)
net/core/gro.c +0 -2
@@ -114,8 +114,6 @@ int skb_gro_receive(struct sk_buff *p, struct sk_buff *skb)if (unlikely (p->len + len >=GRO_LEGACY_MAX_SIZE )) {if (NAPI_GRO_CB (skb)->proto !=IPPROTO_TCP || - (p->protocol ==htons (ETH_P_IPV6 ) && -skb_headroom (p) <sizeof (struct hop_jumbo_hdr )) || p->encapsulation)return -E2BIG ; }
net/ipv6/ip6_offload.c +1 -27
@@ -340,40 +340,14 @@ INDIRECT_CALLABLE_SCOPE int ipv6_gro_complete(struct sk_buff *skb, int nhoff)const struct net_offload *ops;struct ipv6hdr *iph;int err = -ENOSYS ; -u32 payload_len;if (skb->encapsulation) {skb_set_inner_protocol (skb,cpu_to_be16 (ETH_P_IPV6 ));skb_set_inner_network_header (skb, nhoff); } - payload_len = skb->len - nhoff -sizeof (*iph); -if (unlikely (payload_len >IPV6_MAXPLEN )) { -struct hop_jumbo_hdr *hop_jumbo; -int hoplen =sizeof (*hop_jumbo); - -/* Move network header left */ -memmove (skb_mac_header (skb) - hoplen,skb_mac_header (skb), - skb->transport_header - skb->mac_header); - skb->data -= hoplen; - skb->len += hoplen; - skb->mac_header -= hoplen; - skb->network_header -= hoplen; - iph = (struct ipv6hdr *)(skb->data + nhoff); - hop_jumbo = (struct hop_jumbo_hdr *)(iph +1 ); - -/* Build hop-by-hop options */ - hop_jumbo->nexthdr = iph->nexthdr; - hop_jumbo->hdrlen =0 ; - hop_jumbo->tlv_type =IPV6_TLV_JUMBO ; - hop_jumbo->tlv_len =4 ; - hop_jumbo->jumbo_payload_len =htonl (payload_len + hoplen); - - iph->nexthdr =NEXTHDR_HOP ; - } - iph = (struct ipv6hdr *)(skb->data + nhoff); -ipv6_set_payload_len (iph, payload_len); +ipv6_set_payload_len (iph, skb->len - nhoff -sizeof (*iph)); nhoff +=sizeof (*iph) +ipv6_exthdrs_len (iph, &ops);if (WARN_ON (!ops || !ops->callbacks.gro_complete))
Signature · 1 line
2.49.0
RFC PATCH 4/17 net/ipv6: Remove jumbo_remove step from TX path
In reply to Maxim MikityanskiyFrom: Maxim Mikityanskiy <[email protected]>
Now that the kernel doesn't insert HBH for BIG TCP IPv6 packets, remove unnecessary steps from the GSO TX path, that used to check and remove HBH.
--- net/core/dev.c | 3 +-- net/ipv6/ip6_offload.c | 5 +---- 2 files changed, 2 insertions(+), 6 deletions(-)
net/core/dev.c +1 -2
@@ -3882,8 +3882,7 @@ int skb_csum_hwoffload_help(struct sk_buff *skb,if (features & (NETIF_F_IP_CSUM |NETIF_F_IPV6_CSUM )) {if (vlan_get_protocol (skb) ==htons (ETH_P_IPV6 ) && -skb_network_header_len (skb) !=sizeof (struct ipv6hdr ) && - !ipv6_has_hopopt_jumbo (skb)) +skb_network_header_len (skb) !=sizeof (struct ipv6hdr )) goto sw_checksum;switch (skb->csum_offset) {
net/ipv6/ip6_offload.c +1 -4
@@ -110,7 +110,7 @@ static struct sk_buff *ipv6_gso_segment(struct sk_buff *skb,struct sk_buff *segs =ERR_PTR (-EINVAL );struct ipv6hdr *ipv6h;const struct net_offload *ops; -int proto, err; +int proto;struct frag_hdr *fptr;unsigned int payload_len;u8 *prevhdr;
@@ -120,9 +120,6 @@ static struct sk_buff *ipv6_gso_segment(struct sk_buff *skb,bool gso_partial;skb_reset_network_header (skb); - err =ipv6_hopopt_jumbo_remove (skb); -if (err) -return ERR_PTR (err); nhoff =skb_network_header (skb) -skb_mac_header (skb);if (unlikely (!pskb_may_pull (skb,sizeof (*ipv6h)))) goto out;
Signature · 1 line
2.49.0
RFC PATCH 5/17 net/mlx5e: Remove jumbo_remove step from TX path
In reply to Maxim MikityanskiyFrom: Maxim Mikityanskiy <[email protected]>
Now that the kernel doesn't insert HBH for BIG TCP IPv6 packets, remove unnecessary steps from the mlx5e and mlx5i TX path, that used to check and remove HBH.
--- .../net/ethernet/mellanox/mlx5/core/en_tx.c | 75 +++---------------- 1 file changed, 12 insertions(+), 63 deletions(-)
drivers/net/ethernet/mellanox/mlx5/core/en_tx.c +12 -63
@@ -146,12 +146,11 @@ mlx5e_txwqe_build_eseg_csum(struct mlx5e_txqsq *sq, struct sk_buff *skb,* to inline later in the transmit descriptor */ static inline u16 -mlx5e_tx_get_gso_ihs (struct mlx5e_txqsq *sq,struct sk_buff *skb,int *hopbyhop) +mlx5e_tx_get_gso_ihs (struct mlx5e_txqsq *sq,struct sk_buff *skb) {struct mlx5e_sq_stats *stats = sq->stats;u16 ihs; - *hopbyhop =0 ;if (skb->encapsulation) {if (skb_shinfo (skb)->gso_type &SKB_GSO_UDP_L4 ) ihs = skb_inner_transport_offset(skb) +
@@ -161,17 +160,12 @@ mlx5e_tx_get_gso_ihs(struct mlx5e_txqsq *sq, struct sk_buff *skb, int *hopbyhop) stats->tso_inner_packets++; stats->tso_inner_bytes += skb->len - ihs; }else { -if (skb_shinfo (skb)->gso_type &SKB_GSO_UDP_L4 ) { +if (skb_shinfo (skb)->gso_type &SKB_GSO_UDP_L4 ) ihs =skb_transport_offset (skb) +sizeof (struct udphdr ); - }else { +else ihs =skb_tcp_all_headers (skb); -if (ipv6_has_hopopt_jumbo (skb)) { - *hopbyhop =sizeof (struct hop_jumbo_hdr ); - ihs -=sizeof (struct hop_jumbo_hdr ); - } - } stats->tso_packets++; - stats->tso_bytes += skb->len - ihs - *hopbyhop; + stats->tso_bytes += skb->len - ihs; }return ihs;
@@ -233,7 +227,6 @@ struct mlx5e_tx_attr {__be16 mss;u16 insz;u8 opcode; -u8 hopbyhop; };struct mlx5e_tx_wqe_attr {
@@ -270,16 +263,14 @@ static void mlx5e_sq_xmit_prepare(struct mlx5e_txqsq *sq, struct sk_buff *skb,struct mlx5e_sq_stats *stats = sq->stats;if (skb_is_gso (skb)) { -int hopbyhop; -u16 ihs =mlx5e_tx_get_gso_ihs (sq, skb, &hopbyhop); +u16 ihs =mlx5e_tx_get_gso_ihs (sq, skb); *attr = (struct mlx5e_tx_attr ) { .opcode =MLX5_OPCODE_LSO , .mss =cpu_to_be16 (skb_shinfo (skb)->gso_size), .ihs = ihs, .num_bytes = skb->len + (skb_shinfo (skb)->gso_segs -1 ) * ihs, - .headlen =skb_headlen (skb) - ihs - hopbyhop, - .hopbyhop = hopbyhop, + .headlen =skb_headlen (skb) - ihs, }; stats->packets +=skb_shinfo (skb)->gso_segs;
@@ -434,7 +425,6 @@ mlx5e_sq_xmit_wqe(struct mlx5e_txqsq *sq, struct sk_buff *skb,struct mlx5_wqe_data_seg *dseg;struct mlx5e_tx_wqe_info *wi;u16 ihs = attr->ihs; -struct ipv6hdr *h6;struct mlx5e_sq_stats *stats = sq->stats;int num_dma;
@@ -451,28 +441,7 @@ mlx5e_sq_xmit_wqe(struct mlx5e_txqsq *sq, struct sk_buff *skb,if (ihs) {u8 *start = eseg->inline_hdr.start; -if (unlikely (attr->hopbyhop)) { -/* remove the HBH header. -* Layout: [Ethernet header][IPv6 header][HBH][TCP header] -*/ -if (skb_vlan_tag_present (skb)) { -mlx5e_insert_vlan (start, skb,ETH_HLEN +sizeof (*h6)); - ihs +=VLAN_HLEN ; - h6 = (struct ipv6hdr *)(start +sizeof (struct vlan_ethhdr )); - }else { -unsafe_memcpy (start, skb->data, -ETH_HLEN +sizeof (*h6), -MLX5_UNSAFE_MEMCPY_DISCLAIMER ); - h6 = (struct ipv6hdr *)(start +ETH_HLEN ); - } - h6->nexthdr =IPPROTO_TCP ; -/* Copy the TCP header after the IPv6 one */ -memcpy (h6 +1 , - skb->data +ETH_HLEN +sizeof (*h6) + -sizeof (struct hop_jumbo_hdr ), -tcp_hdrlen (skb)); -/* Leave ipv6 payload_len set to 0, as LSO v2 specs request. */ - }else if (skb_vlan_tag_present (skb)) { +if (skb_vlan_tag_present (skb)) {mlx5e_insert_vlan (start, skb, ihs); ihs +=VLAN_HLEN ; stats->added_vlan_packets++;
@@ -492,7 +461,7 @@ mlx5e_sq_xmit_wqe(struct mlx5e_txqsq *sq, struct sk_buff *skb, } dseg += wqe_attr->ds_cnt_ids; - num_dma =mlx5e_txwqe_build_dsegs (sq, skb, skb->data + attr->ihs + attr->hopbyhop, + num_dma =mlx5e_txwqe_build_dsegs (sq, skb, skb->data + attr->ihs, attr->headlen, dseg);if (unlikely (num_dma <0 )) goto err_drop;
@@ -1015,34 +984,14 @@ void mlx5i_sq_xmit(struct mlx5e_txqsq *sq, struct sk_buff *skb, eseg->mss = attr.mss;if (attr.ihs) { -if (unlikely (attr.hopbyhop)) { -struct ipv6hdr *h6; - -/* remove the HBH header. -* Layout: [Ethernet header][IPv6 header][HBH][TCP header] -*/ -unsafe_memcpy (eseg->inline_hdr.start, skb->data, -ETH_HLEN +sizeof (*h6), -MLX5_UNSAFE_MEMCPY_DISCLAIMER ); - h6 = (struct ipv6hdr *)((char *)eseg->inline_hdr.start +ETH_HLEN ); - h6->nexthdr =IPPROTO_TCP ; -/* Copy the TCP header after the IPv6 one */ -unsafe_memcpy (h6 +1 , - skb->data +ETH_HLEN +sizeof (*h6) + -sizeof (struct hop_jumbo_hdr ), -tcp_hdrlen (skb), -MLX5_UNSAFE_MEMCPY_DISCLAIMER ); -/* Leave ipv6 payload_len set to 0, as LSO v2 specs request. */ - }else { -unsafe_memcpy (eseg->inline_hdr.start, skb->data, - attr.ihs, -MLX5_UNSAFE_MEMCPY_DISCLAIMER ); - } +unsafe_memcpy (eseg->inline_hdr.start, skb->data, + attr.ihs, +MLX5_UNSAFE_MEMCPY_DISCLAIMER ); eseg->inline_hdr.sz =cpu_to_be16 (attr.ihs); dseg += wqe_attr.ds_cnt_inl; } - num_dma =mlx5e_txwqe_build_dsegs (sq, skb, skb->data + attr.ihs + attr.hopbyhop, + num_dma =mlx5e_txwqe_build_dsegs (sq, skb, skb->data + attr.ihs, attr.headlen, dseg);if (unlikely (num_dma <0 )) goto err_drop;
Signature · 1 line
2.49.0
RFC PATCH 6/17 net/mlx4: Remove jumbo_remove step from TX path
In reply to Maxim MikityanskiyFrom: Maxim Mikityanskiy <[email protected]>
Now that the kernel doesn't insert HBH for BIG TCP IPv6 packets, remove unnecessary steps from the mlx4 TX path, that used to check and remove HBH.
--- drivers/net/ethernet/mellanox/mlx4/en_tx.c | 42 +++++----------------- 1 file changed, 8 insertions(+), 34 deletions(-)
drivers/net/ethernet/mellanox/mlx4/en_tx.c +8 -34
@@ -636,28 +636,20 @@ static int get_real_size(const struct sk_buff *skb,struct net_device *dev, int *lso_header_size, bool *inline_ok, - void **pfrag, - int *hopbyhop) + void **pfrag) { struct mlx4_en_priv *priv = netdev_priv(dev);int real_size;if (shinfo->gso_size) { *inline_ok = false; - *hopbyhop =0 ;if (skb->encapsulation) { *lso_header_size =skb_inner_tcp_all_headers (skb); }else { -/* Detects large IPV6 TCP packets and prepares for removal of -* HBH header that has been pushed by ip6_xmit(), -* mainly so that tcpdump can dissect them. -*/ -if (ipv6_has_hopopt_jumbo (skb)) - *hopbyhop =sizeof (struct hop_jumbo_hdr ); *lso_header_size =skb_tcp_all_headers (skb); } real_size =CTRL_SIZE + shinfo->nr_frags *DS_SIZE + -ALIGN (*lso_header_size - *hopbyhop +4 ,DS_SIZE ); +ALIGN (*lso_header_size +4 ,DS_SIZE );if (unlikely (*lso_header_size !=skb_headlen (skb))) {/* We add a segment for the skb linear buffer only if * it contains data */
@@ -884,7 +876,6 @@ netdev_tx_t mlx4_en_xmit(struct sk_buff *skb, struct net_device *dev)int desc_size;int real_size;u32 index, bf_index; -struct ipv6hdr *h6;__be32 op_own;int lso_header_size;void *fragptr =NULL ;
@@ -893,7 +884,6 @@ netdev_tx_t mlx4_en_xmit(struct sk_buff *skb, struct net_device *dev)bool stop_queue;bool inline_ok;u8 data_offset; -int hopbyhop;bool bf_ok; tx_ind =skb_get_queue_mapping (skb);
@@ -903,7 +893,7 @@ netdev_tx_t mlx4_en_xmit(struct sk_buff *skb, struct net_device *dev) goto tx_drop; real_size =get_real_size (skb, shinfo, dev, &lso_header_size, - &inline_ok, &fragptr, &hopbyhop); + &inline_ok, &fragptr);if (unlikely (!real_size)) goto tx_drop_count;
@@ -956,7 +946,7 @@ netdev_tx_t mlx4_en_xmit(struct sk_buff *skb, struct net_device *dev) data = &tx_desc->data; data_offset = offsetof(struct mlx4_en_tx_desc , data); }else { -int lso_align =ALIGN (lso_header_size - hopbyhop +4 ,DS_SIZE ); +int lso_align =ALIGN (lso_header_size +4 ,DS_SIZE ); data = (void *)&tx_desc->lso + lso_align; data_offset = offsetof(struct mlx4_en_tx_desc , lso) + lso_align;
@@ -1021,31 +1011,15 @@ netdev_tx_t mlx4_en_xmit(struct sk_buff *skb, struct net_device *dev) ((ring->prod & ring->size) ?cpu_to_be32 (MLX4_EN_BIT_DESC_OWN ) :0 ); - lso_header_size -= hopbyhop;/* Fill in the LSO prefix */ tx_desc->lso.mss_hdr_size =cpu_to_be32 ( shinfo->gso_size <<16 | lso_header_size); +/* Copy headers; +* note that we already verified that it is linear +*/ +memcpy (tx_desc->lso.header, skb->data, lso_header_size); -if (unlikely (hopbyhop)) { -/* remove the HBH header. -* Layout: [Ethernet header][IPv6 header][HBH][TCP header] -*/ -memcpy (tx_desc->lso.header, skb->data,ETH_HLEN +sizeof (*h6)); - h6 = (struct ipv6hdr *)((char *)tx_desc->lso.header +ETH_HLEN ); - h6->nexthdr =IPPROTO_TCP ; -/* Copy the TCP header after the IPv6 one */ -memcpy (h6 +1 , - skb->data +ETH_HLEN +sizeof (*h6) + -sizeof (struct hop_jumbo_hdr ), -tcp_hdrlen (skb)); -/* Leave ipv6 payload_len set to 0, as LSO v2 specs request. */ - }else { -/* Copy headers; -* note that we already verified that it is linear -*/ -memcpy (tx_desc->lso.header, skb->data, lso_header_size); - } ring->tso_packets++; i = shinfo->gso_segs;
Signature · 1 line
2.49.0
RFC PATCH 7/17 ice: Remove jumbo_remove step from TX path
In reply to Maxim MikityanskiyFrom: Maxim Mikityanskiy <[email protected]>
Now that the kernel doesn't insert HBH for BIG TCP IPv6 packets, remove unnecessary steps from the ice TX path, that used to check and remove HBH.
--- drivers/net/ethernet/intel/ice/ice_txrx.c | 3 --- 1 file changed, 3 deletions(-)
drivers/net/ethernet/intel/ice/ice_txrx.c +0 -3
@@ -2440,9 +2440,6 @@ ice_xmit_frame_ring(struct sk_buff *skb, struct ice_tx_ring *tx_ring)ice_trace (xmit_frame_ring, tx_ring, skb); -if (unlikely (ipv6_hopopt_jumbo_remove (skb))) - goto out_drop; - count =ice_xmit_desc_count (skb);if (ice_chk_linearize (skb, count)) {if (__skb_linearize (skb))
Signature · 1 line
2.49.0
RFC PATCH 8/17 bnxt_en: Remove jumbo_remove step from TX path
In reply to Maxim MikityanskiyFrom: Maxim Mikityanskiy <[email protected]>
Now that the kernel doesn't insert HBH for BIG TCP IPv6 packets, remove unnecessary steps from the bnxt_en TX path, that used to check and remove HBH.
--- drivers/net/ethernet/broadcom/bnxt/bnxt.c | 21 --------------------- 1 file changed, 21 deletions(-)
drivers/net/ethernet/broadcom/bnxt/bnxt.c +0 -21
@@ -511,9 +511,6 @@ static netdev_tx_t bnxt_start_xmit(struct sk_buff *skb, struct net_device *dev)return NETDEV_TX_BUSY ; } -if (unlikely (ipv6_hopopt_jumbo_remove (skb))) - goto tx_free; - length = skb->len; len =skb_headlen (skb); last_frag =skb_shinfo (skb)->nr_frags;
@@ -13663,7 +13660,6 @@ static bool bnxt_exthdr_check(struct bnxt *bp, struct sk_buff *skb, int nw_off, u8 **nextp) {struct ipv6hdr *ip6h = (struct ipv6hdr *)(skb->data + nw_off); -struct hop_jumbo_hdr *jhdr;int hdr_count =0 ;u8 *nexthdr;int start;
@@ -13692,24 +13688,7 @@ static bool bnxt_exthdr_check(struct bnxt *bp, struct sk_buff *skb, int nw_off,if (hdrlen >64 )return false; -/* The ext header may be a hop-by-hop header inserted for -* big TCP purposes. This will be removed before sending -* from NIC, so do not count it. -*/ -if (*nexthdr ==NEXTHDR_HOP ) { -if (likely (skb->len <=GRO_LEGACY_MAX_SIZE )) - goto increment_hdr; - - jhdr = (struct hop_jumbo_hdr *)hp; -if (jhdr->tlv_type !=IPV6_TLV_JUMBO || jhdr->hdrlen !=0 || - jhdr->nexthdr !=IPPROTO_TCP ) - goto increment_hdr; - - goto next_hdr; - } -increment_hdr: hdr_count++; -next_hdr: nexthdr = &hp->nexthdr; start += hdrlen; }
Signature · 1 line
2.49.0
RFC PATCH 9/17 gve: Remove jumbo_remove step from TX path
In reply to Maxim MikityanskiyFrom: Maxim Mikityanskiy <[email protected]>
Now that the kernel doesn't insert HBH for BIG TCP IPv6 packets, remove unnecessary steps from the gve TX path, that used to check and remove HBH.
--- drivers/net/ethernet/google/gve/gve_tx_dqo.c | 3 --- 1 file changed, 3 deletions(-)
drivers/net/ethernet/google/gve/gve_tx_dqo.c +0 -3
@@ -925,9 +925,6 @@ static int gve_try_tx_skb(struct gve_priv *priv, struct gve_tx_ring *tx,int num_buffer_descs;int total_num_descs; -if (skb_is_gso (skb) &&unlikely (ipv6_hopopt_jumbo_remove (skb))) - goto drop; -if (tx->dqo.qpl) {/* We do not need to verify the number of buffers used per * packet or per segment in case of TSO as with 2K size buffers
Signature · 1 line
2.49.0
RFC PATCH 10/17 net: mana: Remove jumbo_remove step from TX path
In reply to Maxim MikityanskiyFrom: Maxim Mikityanskiy <[email protected]>
Now that the kernel doesn't insert HBH for BIG TCP IPv6 packets, remove unnecessary steps from the mana TX path, that used to check and remove HBH.
--- drivers/net/ethernet/microsoft/mana/mana_en.c | 3 --- 1 file changed, 3 deletions(-)
drivers/net/ethernet/microsoft/mana/mana_en.c +0 -3
@@ -262,9 +262,6 @@ netdev_tx_t mana_start_xmit(struct sk_buff *skb, struct net_device *ndev)if (skb_cow_head (skb,MANA_HEADROOM )) goto tx_drop_count; -if (unlikely (ipv6_hopopt_jumbo_remove (skb))) - goto tx_drop_count; - txq = &apc->tx_qp[txq_idx].txq; gdma_sq = txq->gdma_sq; cq = &apc->tx_qp[txq_idx].tx_cq;
Signature · 1 line
2.49.0
RFC PATCH 11/17 net/ipv6: Remove HBH helpers
In reply to Maxim MikityanskiyFrom: Maxim Mikityanskiy <[email protected]>
Now that the HBH jumbo helpers are not used by any driver or GSO, remove them altogether.
--- include/net/ipv6.h | 77 ---------------------------------------------- 1 file changed, 77 deletions(-)
include/net/ipv6.h +0 -77
@@ -149,17 +149,6 @@ struct frag_hdr {__be32 identification; }; -/* -* Jumbo payload option, as described in RFC 2675 2. -*/ -struct hop_jumbo_hdr { -u8 nexthdr; -u8 hdrlen; -u8 tlv_type;/* IPV6_TLV_JUMBO, 0xC2 */ -u8 tlv_len;/* 4 */ -__be32 jumbo_payload_len; -}; -#define IP6_MF 0x0001#define IP6_OFFSET 0xFFF8
@@ -462,72 +451,6 @@ bool ipv6_opt_accepted(const struct sock *sk, const struct sk_buff *skb,struct ipv6_txoptions *ipv6_update_options (struct sock *sk,struct ipv6_txoptions *opt); -/* This helper is specialized for BIG TCP needs. -* It assumes the hop_jumbo_hdr will immediately follow the IPV6 header. -* It assumes headers are already in skb->head. -* Returns: 0, or IPPROTO_TCP if a BIG TCP packet is there. -*/ -static inline int ipv6_has_hopopt_jumbo (const struct sk_buff *skb) -{ -const struct hop_jumbo_hdr *jhdr; -const struct ipv6hdr *nhdr; - -if (likely (skb->len <=GRO_LEGACY_MAX_SIZE )) -return 0 ; - -if (skb->protocol !=htons (ETH_P_IPV6 )) -return 0 ; - -if (skb_network_offset (skb) + -sizeof (struct ipv6hdr ) + -sizeof (struct hop_jumbo_hdr ) >skb_headlen (skb)) -return 0 ; - - nhdr =ipv6_hdr (skb); - -if (nhdr->nexthdr !=NEXTHDR_HOP ) -return 0 ; - - jhdr = (const struct hop_jumbo_hdr *) (nhdr +1 ); -if (jhdr->tlv_type !=IPV6_TLV_JUMBO || jhdr->hdrlen !=0 || - jhdr->nexthdr !=IPPROTO_TCP ) -return 0 ; -return jhdr->nexthdr; -} - -/* Return 0 if HBH header is successfully removed -* Or if HBH removal is unnecessary (packet is not big TCP) -* Return error to indicate dropping the packet -*/ -static inline int ipv6_hopopt_jumbo_remove (struct sk_buff *skb) -{ -const int hophdr_len =sizeof (struct hop_jumbo_hdr ); -int nexthdr =ipv6_has_hopopt_jumbo (skb); -struct ipv6hdr *h6; - -if (!nexthdr) -return 0 ; - -if (skb_cow_head (skb,0 )) -return -1 ; - -/* Remove the HBH header. -* Layout: [Ethernet header][IPv6 header][HBH][L4 Header] -*/ -memmove (skb_mac_header (skb) + hophdr_len,skb_mac_header (skb), -skb_network_header (skb) -skb_mac_header (skb) + -sizeof (struct ipv6hdr )); - -__skb_pull (skb, hophdr_len); - skb->network_header += hophdr_len; - skb->mac_header += hophdr_len; - - h6 =ipv6_hdr (skb); - h6->nexthdr = nexthdr; - -return 0 ; -} -static inline bool ipv6_accept_ra (const struct inet6_dev *idev) {s32 accept_ra =READ_ONCE (idev->cnf.accept_ra);
Signature · 1 line
2.49.0
RFC PATCH 12/17 net: Enable BIG TCP with partial GSO
In reply to Maxim MikityanskiyFrom: Maxim Mikityanskiy <[email protected]>
skb_segment is called for partial GSO, when netif_needs_gso returns true in validate_xmit_skb. Partial GSO is needed, for example, when segmentation of tunneled traffic is offloaded to a NIC that only supports inner checksum offload.
Currently, skb_segment clamps the segment length to 65534 bytes, because gso_size == 65535 is a special value GSO_BY_FRAGS, and we don't want to accidentally assign mss = 65535, as it would fall into the GSO_BY_FRAGS check further in the function.
This implementation, however, artificially blocks len > 65534, which is possible since the introduction of BIG TCP. To allow bigger lengths and avoid resegmentation of BIG TCP packets, store the gso_by_frags flag in the beginning and don't use a special value of mss for this purpose after mss was modified.
--- net/core/skbuff.c | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-)
net/core/skbuff.c +5 -5
@@ -4696,6 +4696,7 @@ struct sk_buff *skb_segment(struct sk_buff *head_skb,struct sk_buff *tail =NULL ;struct sk_buff *list_skb =skb_shinfo (head_skb)->frag_list;unsigned int mss =skb_shinfo (head_skb)->gso_size; +bool gso_by_frags = mss ==GSO_BY_FRAGS ;unsigned int doffset = head_skb->data -skb_mac_header (head_skb);unsigned int offset = doffset;unsigned int tnl_hlen =skb_tnl_header_len (head_skb);
@@ -4711,7 +4712,7 @@ struct sk_buff *skb_segment(struct sk_buff *head_skb,int nfrags, pos;if ((skb_shinfo (head_skb)->gso_type &SKB_GSO_DODGY ) && - mss !=GSO_BY_FRAGS && mss !=skb_headlen (head_skb)) { + !gso_by_frags && mss !=skb_headlen (head_skb)) {struct sk_buff *check_skb;for (check_skb = list_skb; check_skb; check_skb = check_skb->next) {
@@ -4739,7 +4740,7 @@ struct sk_buff *skb_segment(struct sk_buff *head_skb, sg = !!(features &NETIF_F_SG ); csum = !!can_checksum_protocol (features, proto); -if (sg && csum && (mss !=GSO_BY_FRAGS )) { +if (sg && csum && !gso_by_frags) {if (!(features &NETIF_F_GSO_PARTIAL )) {struct sk_buff *iter;unsigned int frag_len;
@@ -4773,9 +4774,8 @@ struct sk_buff *skb_segment(struct sk_buff *head_skb,/* GSO partial only requires that we trim off any excess that * doesn't fit into an MSS sized block, so take care of that * now. -* Cap len to not accidentally hit GSO_BY_FRAGS. */ - partial_segs =min (len,GSO_BY_FRAGS -1 ) / mss; + partial_segs = len / mss;if (partial_segs >1 ) mss *= partial_segs;else
@@ -4799,7 +4799,7 @@ struct sk_buff *skb_segment(struct sk_buff *head_skb,int hsize;int size; -if (unlikely (mss ==GSO_BY_FRAGS )) { +if (unlikely (gso_by_frags)) { len = list_skb->len; }else { len = head_skb->len - offset;
Signature · 1 line
2.49.0
RFC PATCH 13/17 udp: Support gro_ipv4_max_size > 65536
In reply to Maxim MikityanskiyFrom: Maxim Mikityanskiy <[email protected]>
Currently, gro_max_size and gro_ipv4_max_size can be set to values bigger than 65536, and GRO will happily aggregate UDP to the configured size (for example, with TCP traffic in VXLAN tunnels). However, udp_gro_complete uses the 16-bit length field in the UDP header to store the length of the aggregated packet. It leads to the packet truncation later in __udp4_lib_rcv.
Fix this by storing 0 to the UDP length field and by restoring the real length from skb->len in __udp4_lib_rcv.
--- net/ipv4/udp.c | 5 ++++- net/ipv4/udp_offload.c | 7 +++++-- 2 files changed, 9 insertions(+), 3 deletions(-)
net/ipv4/udp.c +4 -1
@@ -2642,7 +2642,7 @@ int __udp4_lib_rcv(struct sk_buff *skb, struct udp_table *udptable, {struct sock *sk =NULL ;struct udphdr *uh; -unsigned short ulen; +unsigned int ulen;struct rtable *rt =skb_rtable (skb);__be32 saddr, daddr;struct net *net =dev_net (skb->dev);
@@ -2666,6 +2666,9 @@ int __udp4_lib_rcv(struct sk_buff *skb, struct udp_table *udptable, goto short_packet;if (proto ==IPPROTO_UDP ) { +if (!ulen) + ulen = skb->len; +/* UDP validates ulen. */ if (ulen <sizeof (*uh) ||pskb_trim_rcsum (skb, ulen)) goto short_packet;
net/ipv4/udp_offload.c +5 -2
@@ -930,12 +930,15 @@ static int udp_gro_complete_segment(struct sk_buff *skb)int udp_gro_complete (struct sk_buff *skb,int nhoff,udp_lookup_t lookup) { -__be16 newlen =htons (skb->len - nhoff); +unsigned int newlen = skb->len - nhoff;struct udphdr *uh = (struct udphdr *)(skb->data + nhoff);struct sock *sk;int err; - uh->len = newlen; +if (newlen <=GRO_LEGACY_MAX_SIZE ) + uh->len =htons (newlen); +else + uh->len =0 ; sk =INDIRECT_CALL_INET (lookup, udp6_lib_lookup_skb, udp4_lib_lookup_skb, skb, uh->source, uh->dest);
Signature · 1 line
2.49.0
RFC PATCH 14/17 udp: Validate UDP length in udp_gro_receive
In reply to Maxim MikityanskiyFrom: Maxim Mikityanskiy <[email protected]>
In the previous commit we started using uh->len = 0 as a marker of a GRO packet bigger than 65536 bytes. To prevent abuse by maliciously crafted packets, check the length in the UDP header in udp_gro_receive. Note that a similar check is present in udp_gro_receive_segment, but not in the UDP socket gro_receive flow.
--- net/ipv4/udp_offload.c | 5 +++++ 1 file changed, 5 insertions(+)
net/ipv4/udp_offload.c +5 -0
@@ -794,6 +794,7 @@ struct sk_buff *udp_gro_receive(struct list_head *head, struct sk_buff *skb,struct sk_buff *p;struct udphdr *uh2;unsigned int off =skb_gro_offset (skb); +unsigned int ulen;int flush =1 ;/* We can do L4 aggregation only if the packet can't land in a tunnel
@@ -826,6 +827,10 @@ struct sk_buff *udp_gro_receive(struct list_head *head, struct sk_buff *skb, !NAPI_GRO_CB(skb)->csum_valid)) goto out; + ulen =ntohs (uh->len); +if (ulen <=sizeof (*uh) || ulen !=skb_gro_len (skb)) + goto out; +/* mark that this skb passed once through the tunnel gro layer */ NAPI_GRO_CB (skb)->encap_mark =1 ;
Signature · 1 line
2.49.0
RFC PATCH 15/17 udp: Set length in UDP header to 0 for big GSO packets
In reply to Maxim MikityanskiyFrom: Maxim Mikityanskiy <[email protected]>
skb->len may be bigger than 65535 in UDP-based tunnels that have BIG TCP enabled. If GSO aggregates packets that large, set the length in the UDP header to 0, so that tcpdump can print such packets properly (treating them as RFC 2675 jumbograms). Later in the pipeline, __udp_gso_segment will set uh->len to the size of individual packets.
--- net/ipv4/udp_tunnel_core.c | 2 +- net/ipv6/ip6_udp_tunnel.c | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-)
net/ipv4/udp_tunnel_core.c +1 -1
@@ -179,7 +179,7 @@ void udp_tunnel_xmit_skb(struct rtable *rt, struct sock *sk, struct sk_buff *skb uh->dest = dst_port; uh->source = src_port; - uh->len =htons (skb->len); + uh->len = skb->len <=GRO_LEGACY_MAX_SIZE ?htons (skb->len) :0 ;memset (&(IPCB (skb)->opt),0 ,sizeof (IPCB (skb)->opt));
net/ipv6/ip6_udp_tunnel.c +1 -1
@@ -92,7 +92,7 @@ int udp_tunnel6_xmit_skb(struct dst_entry *dst, struct sock *sk, uh->dest = dst_port; uh->source = src_port; - uh->len =htons (skb->len); + uh->len = skb->len <=GRO_LEGACY_MAX_SIZE ?htons (skb->len) :0 ;skb_dst_set (skb, dst);
Signature · 1 line
2.49.0
RFC PATCH 16/17 vxlan: Enable BIG TCP packets
In reply to Maxim MikityanskiyFrom: Maxim Mikityanskiy <[email protected]>
In Cilium we do support BIG TCP, but so far the latter has only been enabled for direct routing use-cases. A lot of users rely on Cilium with vxlan/geneve tunneling though. The underlying kernel infra for tunneling has not been supporting BIG TCP up to this point.
Given we do now, bump tso_max_size for vxlan netdevs up to GSO_MAX_SIZE to allow the admin to use BIG TCP with vxlan tunnels.
BIG TCP on vxlan disabled:
Standard MTU:
# netperf -H 10.1.0.2 -t TCP_STREAM -l60
MIGRATED TCP STREAM TEST from 0.0.0.0 (0.0.0.0) port 0 AF_INET to 10.1.0.2 () port 0 AF_INET : demo
Recv Send Send
Socket Socket Message Elapsed
Size Size Size Time Throughput
bytes bytes bytes secs. 10^6bits/sec131072 16384 16384 30.00 34440.00
8k MTU:
# netperf -H 10.1.0.2 -t TCP_STREAM -l60
MIGRATED TCP STREAM TEST from 0.0.0.0 (0.0.0.0) port 0 AF_INET to 10.1.0.2 () port 0 AF_INET : demo
Recv Send Send
Socket Socket Message Elapsed
Size Size Size Time Throughput
bytes bytes bytes secs. 10^6bits/sec262144 32768 32768 30.00 55684.26
BIG TCP on vxlan enabled:
Standard MTU:
# netperf -H 10.1.0.2 -t TCP_STREAM -l60
MIGRATED TCP STREAM TEST from 0.0.0.0 (0.0.0.0) port 0 AF_INET to 10.1.0.2 () port 0 AF_INET : demo
Recv Send Send
Socket Socket Message Elapsed
Size Size Size Time Throughput
bytes bytes bytes secs. 10^6bits/sec131072 16384 16384 30.00 39564.78
8k MTU:
# netperf -H 10.1.0.2 -t TCP_STREAM -l60
MIGRATED TCP STREAM TEST from 0.0.0.0 (0.0.0.0) port 0 AF_INET to 10.1.0.2 () port 0 AF_INET : demo
Recv Send Send
Socket Socket Message Elapsed
Size Size Size Time Throughput
bytes bytes bytes secs. 10^6bits/sec262144 32768 32768 30.00 61466.47
When tunnel offloads are not enabled/exposed and we fully need to rely on SW-based segmentation on transmit (e.g. in case of Azure) then the more aggressive batching also has a visible effect. Below example was on the same setup as with above benchmarks but with HW support disabled:
# ethtool -k enp10s0f0np0 | grep udp tx-udp_tnl-segmentation: off tx-udp_tnl-csum-segmentation: off tx-udp-segmentation: off rx-udp_tunnel-port-offload: off rx-udp-gro-forwarding: off
Before:
# netperf -H 10.1.0.2 -t TCP_STREAM -l60
MIGRATED TCP STREAM TEST from 0.0.0.0 (0.0.0.0) port 0 AF_INET to 10.1.0.2 () port 0 AF_INET : demo
Recv Send Send
Socket Socket Message Elapsed
Size Size Size Time Throughput
bytes bytes bytes secs. 10^6bits/sec131072 16384 16384 60.00 21820.82
After:
# netperf -H 10.1.0.2 -t TCP_STREAM -l60
MIGRATED TCP STREAM TEST from 0.0.0.0 (0.0.0.0) port 0 AF_INET to 10.1.0.2 () port 0 AF_INET : demo
Recv Send Send
Socket Socket Message Elapsed
Size Size Size Time Throughput
bytes bytes bytes secs. 10^6bits/sec131072 16384 16384 60.00 29390.78
Example receive side:
swapper 0 [002] 4712.645070: net:netif_receive_skb: dev=enp10s0f0np0 skbaddr=0xffff8f3b086e0200 len=129542
ffffffff8cfe3aaa __netif_receive_skb_core.constprop.0+0x6ca ([kernel.kallsyms])
ffffffff8cfe3aaa __netif_receive_skb_core.constprop.0+0x6ca ([kernel.kallsyms])
ffffffff8cfe47dd __netif_receive_skb_list_core+0xed ([kernel.kallsyms])
ffffffff8cfe4e52 netif_receive_skb_list_internal+0x1d2 ([kernel.kallsyms])
ffffffff8d0210d8 gro_complete.constprop.0+0x108 ([kernel.kallsyms])
ffffffff8d021724 dev_gro_receive+0x4e4 ([kernel.kallsyms])
ffffffff8d021a99 gro_receive_skb+0x89 ([kernel.kallsyms])
ffffffffc06edb71 mlx5e_handle_rx_cqe_mpwrq+0x131 ([kernel.kallsyms])
ffffffffc06ee38a mlx5e_poll_rx_cq+0x9a ([kernel.kallsyms])
ffffffffc06ef2c7 mlx5e_napi_poll+0x107 ([kernel.kallsyms])
ffffffff8cfe586d __napi_poll+0x2d ([kernel.kallsyms])
ffffffff8cfe5f8d net_rx_action+0x20d ([kernel.kallsyms])
ffffffff8c35d252 handle_softirqs+0xe2 ([kernel.kallsyms])
ffffffff8c35d556 __irq_exit_rcu+0xd6 ([kernel.kallsyms])
ffffffff8c35d81e irq_exit_rcu+0xe ([kernel.kallsyms])
ffffffff8d2602b8 common_interrupt+0x98 ([kernel.kallsyms])
ffffffff8c000da7 asm_common_interrupt+0x27 ([kernel.kallsyms])
ffffffff8d2645c5 cpuidle_enter_state+0xd5 ([kernel.kallsyms])
ffffffff8cf6358e cpuidle_enter+0x2e ([kernel.kallsyms])
ffffffff8c3ba932 call_cpuidle+0x22 ([kernel.kallsyms])
ffffffff8c3bfb5e do_idle+0x1ce ([kernel.kallsyms])
ffffffff8c3bfd79 cpu_startup_entry+0x29 ([kernel.kallsyms])
ffffffff8c30a6c2 start_secondary+0x112 ([kernel.kallsyms])
ffffffff8c2c142d common_startup_64+0x13e ([kernel.kallsyms])Example transmit side:
swapper 0 [005] 4768.021375: net:net_dev_xmit: dev=enp10s0f0np0 skbaddr=0xffff8af32ebe1200 len=129556 rc=0
ffffffffa75e19c3 dev_hard_start_xmit+0x173 ([kernel.kallsyms])
ffffffffa75e19c3 dev_hard_start_xmit+0x173 ([kernel.kallsyms])
ffffffffa7653823 sch_direct_xmit+0x143 ([kernel.kallsyms])
ffffffffa75e2780 __dev_queue_xmit+0xc70 ([kernel.kallsyms])
ffffffffa76a1205 ip_finish_output2+0x265 ([kernel.kallsyms])
ffffffffa76a1577 __ip_finish_output+0x87 ([kernel.kallsyms])
ffffffffa76a165b ip_finish_output+0x2b ([kernel.kallsyms])
ffffffffa76a179e ip_output+0x5e ([kernel.kallsyms])
ffffffffa76a19d5 ip_local_out+0x35 ([kernel.kallsyms])
ffffffffa770d0e5 iptunnel_xmit+0x185 ([kernel.kallsyms])
ffffffffc179634e nf_nat_used_tuple_new.cold+0x1129 ([kernel.kallsyms])
ffffffffc17a7301 vxlan_xmit_one+0xc21 ([kernel.kallsyms])
ffffffffc17a80a2 vxlan_xmit+0x4a2 ([kernel.kallsyms])
ffffffffa75e18af dev_hard_start_xmit+0x5f ([kernel.kallsyms])
ffffffffa75e1d3f __dev_queue_xmit+0x22f ([kernel.kallsyms])
ffffffffa76a1205 ip_finish_output2+0x265 ([kernel.kallsyms])
ffffffffa76a1577 __ip_finish_output+0x87 ([kernel.kallsyms])
ffffffffa76a165b ip_finish_output+0x2b ([kernel.kallsyms])
ffffffffa76a179e ip_output+0x5e ([kernel.kallsyms])
ffffffffa76a1de2 __ip_queue_xmit+0x1b2 ([kernel.kallsyms])
ffffffffa76a2135 ip_queue_xmit+0x15 ([kernel.kallsyms])
ffffffffa76c70a2 __tcp_transmit_skb+0x522 ([kernel.kallsyms])
ffffffffa76c931a tcp_write_xmit+0x65a ([kernel.kallsyms])
ffffffffa76cb42e tcp_tsq_write+0x5e ([kernel.kallsyms])
ffffffffa76cb7ef tcp_tasklet_func+0x10f ([kernel.kallsyms])
ffffffffa695d9f7 tasklet_action_common+0x107 ([kernel.kallsyms])
ffffffffa695db99 tasklet_action+0x29 ([kernel.kallsyms])
ffffffffa695d252 handle_softirqs+0xe2 ([kernel.kallsyms])
ffffffffa695d556 __irq_exit_rcu+0xd6 ([kernel.kallsyms])
ffffffffa695d81e irq_exit_rcu+0xe ([kernel.kallsyms])
ffffffffa78602b8 common_interrupt+0x98 ([kernel.kallsyms])
ffffffffa6600da7 asm_common_interrupt+0x27 ([kernel.kallsyms])
ffffffffa78645c5 cpuidle_enter_state+0xd5 ([kernel.kallsyms])
ffffffffa756358e cpuidle_enter+0x2e ([kernel.kallsyms])
ffffffffa69ba932 call_cpuidle+0x22 ([kernel.kallsyms])
ffffffffa69bfb5e do_idle+0x1ce ([kernel.kallsyms])
ffffffffa69bfd79 cpu_startup_entry+0x29 ([kernel.kallsyms])
ffffffffa690a6c2 start_secondary+0x112 ([kernel.kallsyms])
ffffffffa68c142d common_startup_64+0x13e ([kernel.kallsyms])--- drivers/net/vxlan/vxlan_core.c | 2 ++ 1 file changed, 2 insertions(+)
drivers/net/vxlan/vxlan_core.c +2 -0
@@ -3330,6 +3330,8 @@ static void vxlan_setup(struct net_device *dev) dev->hw_features |=NETIF_F_RXCSUM ; dev->hw_features |=NETIF_F_GSO_SOFTWARE ;netif_keep_dst (dev); +netif_set_tso_max_size (dev,GSO_MAX_SIZE ); + dev->priv_flags |=IFF_NO_QUEUE ; dev->change_proto_down = true; dev->lltx = true;
Signature · 1 line
2.49.0
RFC PATCH 17/17 geneve: Enable BIG TCP packets
In reply to Maxim MikityanskiyFrom: Daniel Borkmann <[email protected]>
In Cilium we do support BIG TCP, but so far the latter has only been enabled for direct routing use-cases. A lot of users rely on Cilium with vxlan/geneve tunneling though. The underlying kernel infra for tunneling has not been supporting BIG TCP up to this point.
Given we do now, bump tso_max_size for geneve netdevs up to GSO_MAX_SIZE to allow the admin to use BIG TCP with geneve tunnels.
BIG TCP on geneve disabled:
Standard MTU:
# netperf -H 10.1.0.2 -t TCP_STREAM -l60
MIGRATED TCP STREAM TEST from 0.0.0.0 (0.0.0.0) port 0 AF_INET to 10.1.0.2 () port 0 AF_INET : demo
Recv Send Send
Socket Socket Message Elapsed
Size Size Size Time Throughput
bytes bytes bytes secs. 10^6bits/sec131072 16384 16384 30.00 37391.34
8k MTU:
# netperf -H 10.1.0.2 -t TCP_STREAM -l60
MIGRATED TCP STREAM TEST from 0.0.0.0 (0.0.0.0) port 0 AF_INET to 10.1.0.2 () port 0 AF_INET : demo
Recv Send Send
Socket Socket Message Elapsed
Size Size Size Time Throughput
bytes bytes bytes secs. 10^6bits/sec262144 32768 32768 60.00 58030.19
BIG TCP on geneve enabled:
Standard MTU:
# netperf -H 10.1.0.2 -t TCP_STREAM -l60
MIGRATED TCP STREAM TEST from 0.0.0.0 (0.0.0.0) port 0 AF_INET to 10.1.0.2 () port 0 AF_INET : demo
Recv Send Send
Socket Socket Message Elapsed
Size Size Size Time Throughput
bytes bytes bytes secs. 10^6bits/sec131072 16384 16384 30.00 40891.57
8k MTU:
# netperf -H 10.1.0.2 -t TCP_STREAM -l60
MIGRATED TCP STREAM TEST from 0.0.0.0 (0.0.0.0) port 0 AF_INET to 10.1.0.2 () port 0 AF_INET : demo
Recv Send Send
Socket Socket Message Elapsed
Size Size Size Time Throughput
bytes bytes bytes secs. 10^6bits/sec262144 32768 32768 60.00 61458.39
Example receive side:
swapper 0 [008] 3682.509996: net:netif_receive_skb: dev=geneve0 skbaddr=0xffff8f3b0a781800 len=129492
ffffffff8cfe3aaa __netif_receive_skb_core.constprop.0+0x6ca ([kernel.kallsyms])
ffffffff8cfe3aaa __netif_receive_skb_core.constprop.0+0x6ca ([kernel.kallsyms])
ffffffff8cfe47dd __netif_receive_skb_list_core+0xed ([kernel.kallsyms])
ffffffff8cfe4e52 netif_receive_skb_list_internal+0x1d2 ([kernel.kallsyms])
ffffffff8cfe573c napi_complete_done+0x7c ([kernel.kallsyms])
ffffffff8d046c23 gro_cell_poll+0x83 ([kernel.kallsyms])
ffffffff8cfe586d __napi_poll+0x2d ([kernel.kallsyms])
ffffffff8cfe5f8d net_rx_action+0x20d ([kernel.kallsyms])
ffffffff8c35d252 handle_softirqs+0xe2 ([kernel.kallsyms])
ffffffff8c35d556 __irq_exit_rcu+0xd6 ([kernel.kallsyms])
ffffffff8c35d81e irq_exit_rcu+0xe ([kernel.kallsyms])
ffffffff8d2602b8 common_interrupt+0x98 ([kernel.kallsyms])
ffffffff8c000da7 asm_common_interrupt+0x27 ([kernel.kallsyms])
ffffffff8d2645c5 cpuidle_enter_state+0xd5 ([kernel.kallsyms])
ffffffff8cf6358e cpuidle_enter+0x2e ([kernel.kallsyms])
ffffffff8c3ba932 call_cpuidle+0x22 ([kernel.kallsyms])
ffffffff8c3bfb5e do_idle+0x1ce ([kernel.kallsyms])
ffffffff8c3bfd79 cpu_startup_entry+0x29 ([kernel.kallsyms])
ffffffff8c30a6c2 start_secondary+0x112 ([kernel.kallsyms])
ffffffff8c2c142d common_startup_64+0x13e ([kernel.kallsyms])Example transmit side:
swapper 0 [002] 3403.688687: net:net_dev_xmit: dev=enp10s0f0np0 skbaddr=0xffff8af31d104ae8 len=129556 rc=0
ffffffffa75e19c3 dev_hard_start_xmit+0x173 ([kernel.kallsyms])
ffffffffa75e19c3 dev_hard_start_xmit+0x173 ([kernel.kallsyms])
ffffffffa7653823 sch_direct_xmit+0x143 ([kernel.kallsyms])
ffffffffa75e2780 __dev_queue_xmit+0xc70 ([kernel.kallsyms])
ffffffffa76a1205 ip_finish_output2+0x265 ([kernel.kallsyms])
ffffffffa76a1577 __ip_finish_output+0x87 ([kernel.kallsyms])
ffffffffa76a165b ip_finish_output+0x2b ([kernel.kallsyms])
ffffffffa76a179e ip_output+0x5e ([kernel.kallsyms])
ffffffffa76a19d5 ip_local_out+0x35 ([kernel.kallsyms])
ffffffffa770d0e5 iptunnel_xmit+0x185 ([kernel.kallsyms])
ffffffffc179634e nf_nat_used_tuple_new.cold+0x1129 ([kernel.kallsyms])
ffffffffc179d3e0 geneve_xmit+0x920 ([kernel.kallsyms])
ffffffffa75e18af dev_hard_start_xmit+0x5f ([kernel.kallsyms])
ffffffffa75e1d3f __dev_queue_xmit+0x22f ([kernel.kallsyms])
ffffffffa76a1205 ip_finish_output2+0x265 ([kernel.kallsyms])
ffffffffa76a1577 __ip_finish_output+0x87 ([kernel.kallsyms])
ffffffffa76a165b ip_finish_output+0x2b ([kernel.kallsyms])
ffffffffa76a179e ip_output+0x5e ([kernel.kallsyms])
ffffffffa76a1de2 __ip_queue_xmit+0x1b2 ([kernel.kallsyms])
ffffffffa76a2135 ip_queue_xmit+0x15 ([kernel.kallsyms])
ffffffffa76c70a2 __tcp_transmit_skb+0x522 ([kernel.kallsyms])
ffffffffa76c931a tcp_write_xmit+0x65a ([kernel.kallsyms])
ffffffffa76ca3b9 __tcp_push_pending_frames+0x39 ([kernel.kallsyms])
ffffffffa76c1fb6 tcp_rcv_established+0x276 ([kernel.kallsyms])
ffffffffa76d3957 tcp_v4_do_rcv+0x157 ([kernel.kallsyms])
ffffffffa76d6053 tcp_v4_rcv+0x1243 ([kernel.kallsyms])
ffffffffa769b8ea ip_protocol_deliver_rcu+0x2a ([kernel.kallsyms])
ffffffffa769bab7 ip_local_deliver_finish+0x77 ([kernel.kallsyms])
ffffffffa769bb4d ip_local_deliver+0x6d ([kernel.kallsyms])
ffffffffa769abe7 ip_sublist_rcv_finish+0x37 ([kernel.kallsyms])
ffffffffa769b713 ip_sublist_rcv+0x173 ([kernel.kallsyms])
ffffffffa769bde2 ip_list_rcv+0x102 ([kernel.kallsyms])
ffffffffa75e4868 __netif_receive_skb_list_core+0x178 ([kernel.kallsyms])
ffffffffa75e4e52 netif_receive_skb_list_internal+0x1d2 ([kernel.kallsyms])
ffffffffa75e573c napi_complete_done+0x7c ([kernel.kallsyms])
ffffffffa7646c23 gro_cell_poll+0x83 ([kernel.kallsyms])
ffffffffa75e586d __napi_poll+0x2d ([kernel.kallsyms])
ffffffffa75e5f8d net_rx_action+0x20d ([kernel.kallsyms])
ffffffffa695d252 handle_softirqs+0xe2 ([kernel.kallsyms])
ffffffffa695d556 __irq_exit_rcu+0xd6 ([kernel.kallsyms])
ffffffffa695d81e irq_exit_rcu+0xe ([kernel.kallsyms])
ffffffffa78602b8 common_interrupt+0x98 ([kernel.kallsyms])
ffffffffa6600da7 asm_common_interrupt+0x27 ([kernel.kallsyms])
ffffffffa78645c5 cpuidle_enter_state+0xd5 ([kernel.kallsyms])
ffffffffa756358e cpuidle_enter+0x2e ([kernel.kallsyms])
ffffffffa69ba932 call_cpuidle+0x22 ([kernel.kallsyms])
ffffffffa69bfb5e do_idle+0x1ce ([kernel.kallsyms])
ffffffffa69bfd79 cpu_startup_entry+0x29 ([kernel.kallsyms])
ffffffffa690a6c2 start_secondary+0x112 ([kernel.kallsyms])
ffffffffa68c142d common_startup_64+0x13e ([kernel.kallsyms])--- drivers/net/geneve.c | 2 ++ 1 file changed, 2 insertions(+)
drivers/net/geneve.c +2 -0
@@ -1221,6 +1221,8 @@ static void geneve_setup(struct net_device *dev) dev->max_mtu =IP_MAX_MTU -GENEVE_BASE_HLEN - dev->hard_header_len;netif_keep_dst (dev); +netif_set_tso_max_size (dev,GSO_MAX_SIZE ); + dev->priv_flags &= ~IFF_TX_SKB_SHARING ; dev->priv_flags |=IFF_LIVE_ADDR_CHANGE |IFF_NO_QUEUE ; dev->lltx = true;
Signature · 1 line
2.49.0
Quoted 7 lines · Maxim Mikityanskiy
This series consists of two parts that will be submitted separately:
The only reason why we keep inserting HBH seems to be for the tools that parse the packets, but the above drawbacks seem to outweigh this, and the tools can be patched (like they need to, in order to be able to parse BIG TCP IPv4 now). I have a patch for tcpdump.
This came multiple times.
I want to see the patches coming in the tools, before patches landing in linux.
Having the promise that it can be done is not enough, sorry !
Quoted 10 lines · Eric Dumazet
Quoted 7 lines · Maxim Mikityanskiy
This series consists of two parts that will be submitted separately:
The only reason why we keep inserting HBH seems to be for the tools that parse the packets, but the above drawbacks seem to outweigh this, and the tools can be patched (like they need to, in order to be able to parse BIG TCP IPv4 now). I have a patch for tcpdump.
Quoted 5 lines
This came multiple times.
I want to see the patches coming in the tools, before patches landing in linux.
Having the promise that it can be done is not enough, sorry !
I do have a patch. I opened a pull request and pushed it there: