TUN/VirtioNet USO features support.
Subsystems (5)
v3 cover TUN/VirtioNet USO features support.
Added new offloads for TUN devices TUN_F_USO4 and TUN_F_USO6. Technically they enable NETIF_F_GSO_UDP_L4
(and only if USO4 & USO6 are set simultaneously). It allows the transmission of large UDP packets.
Different features USO4 and USO6 are required for qemu where Windows guests can enable disable USO receives for IPv4 and IPv6 separately.
On the other side, Linux can't really differentiate USO4 and USO6, for now. For now, to enable USO for TUN it requires enabling USO4 and USO6 together. In the future, there would be a mechanism to control UDP_L4 GSO separately.
New types for virtio-net already in virtio-net specification:
https://github.com/oasis-tcs/virtio-spec/issues/120
Test it WIP Qemu https://github.com/daynix/qemu/tree/USOv3
Andrew (5):
uapi/linux/if_tun.h: Added new offload types for USO4/6. driver/net/tun: Added features for USO. uapi/linux/virtio_net.h: Added USO types. linux/virtio_net.h: Support USO offload in vnet header. drivers/net/virtio_net.c: Added USO support.
Andrew Melnychenko (1):
udp: allow header check for dodgy GSO_UDP_L4 packets.
drivers/net/tap.c | 10 ++++++++-- drivers/net/tun.c | 8 +++++++- drivers/net/virtio_net.c | 19 +++++++++++++++---- include/linux/virtio_net.h | 9 +++++++++ include/uapi/linux/if_tun.h | 2 ++ include/uapi/linux/virtio_net.h | 5 +++++ net/ipv4/udp_offload.c | 2 +- 7 files changed, 47 insertions(+), 8 deletions(-)
Signature · 6 lines
2.37.2 _______________________________________________ Virtualization mailing list [email protected] https://lists.linuxfoundation.org/mailman/listinfo/virtualization
PATCH v3 1/6 udp: allow header check for dodgy GSO_UDP_L4 packets.
Packets from TAP devices with USO offload converts
to GSO_UDP_L4 & GSO_DODGY sk buffers.
Added changes allow skipping segmentation for DODGY/ROBUST packets.
--- net/ipv4/udp_offload.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
net/ipv4/udp_offload.c +1 -1
@@ -387,7 +387,7 @@ static struct sk_buff *udp4_ufo_fragment(struct sk_buff *skb,if (!pskb_may_pull (skb,sizeof (struct udphdr ))) goto out; -if (skb_shinfo (skb)->gso_type &SKB_GSO_UDP_L4 ) +if (skb_shinfo (skb)->gso_type &SKB_GSO_UDP_L4 && !skb_gso_ok (skb, features |NETIF_F_GSO_ROBUST ))return __udp_gso_segment (skb, features, false); mss =skb_shinfo (skb)->gso_size;
Signature · 6 lines
2.37.2 _______________________________________________ Virtualization mailing list [email protected] https://lists.linuxfoundation.org/mailman/listinfo/virtualization
Re: [PATCH v3 1/6] udp: allow header check for dodgy GSO_UDP_L4 packets.
net/ipv4/udp_offload.c static struct sk_buff *udp4_ufo_fragment(struct sk_buff *skb, · line 390
Quoted 10 lines · Andrew Melnychenko
net/ipv4/udp_offload.c +1
-1@@ -387,7 +387,7 @@ static struct sk_buff *udp4_ufo_fragment(struct sk_buff *skb,if (!pskb_may_pull (skb,sizeof (struct udphdr ))) goto out; -if (skb_shinfo (skb)->gso_type &SKB_GSO_UDP_L4 ) +if (skb_shinfo (skb)->gso_type &SKB_GSO_UDP_L4 && !skb_gso_ok (skb, features |NETIF_F_GSO_ROBUST ))
that line needs to be wrapped.
Quoted 3 lines
net/ipv4/udp_offload.c +0
-0return __udp_gso_segment (skb, features, false); mss =skb_shinfo (skb)->gso_size;
Re: [PATCH v3 1/6] udp: allow header check for dodgy GSO_UDP_L4 packets.
Hi all,
Quoted 10 lines · David Ahern
Quoted 8 lines · Andrew Melnychenko
net/ipv4/udp_offload.c +0
-0@@ -387,7 +387,7 @@ static struct sk_buff *udp4_ufo_fragment(struct sk_buff *skb,if (!pskb_may_pull (skb,sizeof (struct udphdr ))) goto out;
Quoted 4 lines
net/ipv4/udp_offload.c static struct sk_buff *udp4_ufo_fragment(struct sk_buff *skb, · line 390
Quoted 2 lines
net/ipv4/udp_offload.c +1
-1-if (skb_shinfo (skb)->gso_type &SKB_GSO_UDP_L4 ) +if (skb_shinfo (skb)->gso_type &SKB_GSO_UDP_L4 && !skb_gso_ok (skb, features |NETIF_F_GSO_ROBUST ))that line needs to be wrapped.
Ok, I'll wrap it.
Quoted 5 lines
Quoted 4 lines
net/ipv4/udp_offload.c +0
-0return __udp_gso_segment (skb, features, false); mss =skb_shinfo (skb)->gso_size;
PATCH v3 2/6 uapi/linux/if_tun.h: Added new offload types for USO4/6.
In reply to Andrew MelnychenkoAdded 2 additional offlloads for USO(IPv4 & IPv6). Separate offloads are required for Windows VM guests, g.e. Windows may set USO rx only for IPv4.
--- include/uapi/linux/if_tun.h | 2 ++ 1 file changed, 2 insertions(+)
include/uapi/linux/if_tun.h +2 -0
@@ -88,6 +88,8 @@#define TUN_F_TSO6 0x04/* I can handle TSO for IPv6 packets */ #define TUN_F_TSO_ECN 0x08/* I can handle TSO with ECN bits. */ #define TUN_F_UFO 0x10/* I can handle UFO packets */ +#define TUN_F_USO4 0x20/* I can handle USO for IPv4 packets */ +#define TUN_F_USO6 0x40/* I can handle USO for IPv6 packets */ /* Protocol info prepended to the packets (when IFF_NO_PI is not set) */ #define TUN_PKT_STRIP 0x0001
Signature · 6 lines
2.37.2 _______________________________________________ Virtualization mailing list [email protected] https://lists.linuxfoundation.org/mailman/listinfo/virtualization
PATCH v3 3/6 driver/net/tun: Added features for USO.
In reply to Andrew MelnychenkoAdded support for USO4 and USO6.
For now, to "enable" USO, it's required to set both USO4 and USO6 simultaneously. USO enables NETIF_F_GSO_UDP_L4.
--- drivers/net/tap.c | 10 ++++++++-- drivers/net/tun.c | 8 +++++++- 2 files changed, 15 insertions(+), 3 deletions(-)
drivers/net/tap.c +8 -2
@@ -957,6 +957,10 @@ static int set_offload(struct tap_queue *q, unsigned long arg)if (arg &TUN_F_TSO6 ) feature_mask |=NETIF_F_TSO6 ; } + +/* TODO: for now USO4 and USO6 should work simultaneously */ +if ((arg & (TUN_F_USO4 |TUN_F_USO6 )) == (TUN_F_USO4 |TUN_F_USO6 )) + features |=NETIF_F_GSO_UDP_L4 ; }/* tun/tap driver inverts the usage for TSO offloads, where
@@ -967,7 +971,8 @@ static int set_offload(struct tap_queue *q, unsigned long arg)* When user space turns off TSO, we turn off GSO/LRO so that * user-space will not receive TSO frames. */ -if (feature_mask & (NETIF_F_TSO |NETIF_F_TSO6 )) +if (feature_mask & (NETIF_F_TSO |NETIF_F_TSO6 ) || + (feature_mask & (TUN_F_USO4 |TUN_F_USO6 )) == (TUN_F_USO4 |TUN_F_USO6 )) features |=RX_OFFLOADS ;else features &= ~RX_OFFLOADS ;
@@ -1091,7 +1096,8 @@ static long tap_ioctl(struct file *file, unsigned int cmd,case TUNSETOFFLOAD :/* let the user check for future flags */ if (arg & ~(TUN_F_CSUM |TUN_F_TSO4 |TUN_F_TSO6 | -TUN_F_TSO_ECN |TUN_F_UFO )) +TUN_F_TSO_ECN |TUN_F_UFO | +TUN_F_USO4 |TUN_F_USO6 ))return -EINVAL ;rtnl_lock ();
drivers/net/tun.c +7 -1
@@ -185,7 +185,7 @@ struct tun_struct {struct net_device *dev;netdev_features_t set_features;#define TUN_USER_FEATURES (NETIF_F_HW_CSUM|NETIF_F_TSO_ECN|NETIF_F_TSO| \ - NETIF_F_TSO6) + NETIF_F_TSO6 | NETIF_F_GSO_UDP_L4)int align;int vnet_hdr_sz;
@@ -2871,6 +2871,12 @@ static int set_offload(struct tun_struct *tun, unsigned long arg) } arg &= ~TUN_F_UFO ; + +/* TODO: for now USO4 and USO6 should work simultaneously */ +if (arg &TUN_F_USO4 && arg &TUN_F_USO6 ) { + features |=NETIF_F_GSO_UDP_L4 ; + arg &= ~(TUN_F_USO4 |TUN_F_USO6 ); + } }/* This gives the user a way to test for new features in future by
Signature · 6 lines
2.37.2 _______________________________________________ Virtualization mailing list [email protected] https://lists.linuxfoundation.org/mailman/listinfo/virtualization
PATCH v3 4/6 uapi/linux/virtio_net.h: Added USO types.
In reply to Andrew MelnychenkoAdded new GSO type for USO: VIRTIO_NET_HDR_GSO_UDP_L4. Feature VIRTIO_NET_F_HOST_USO allows to enable NETIF_F_GSO_UDP_L4. Separated VIRTIO_NET_F_GUEST_USO4 & VIRTIO_NET_F_GUEST_USO6 features required for Windows guests.
--- include/uapi/linux/virtio_net.h | 5 +++++ 1 file changed, 5 insertions(+)
include/uapi/linux/virtio_net.h +5 -0
@@ -57,6 +57,10 @@* Steering */ #define VIRTIO_NET_F_CTRL_MAC_ADDR 23/* Set MAC address */ #define VIRTIO_NET_F_NOTF_COAL 53/* Guest can handle notifications coalescing */ +#define VIRTIO_NET_F_GUEST_USO4 54/* Guest can handle USOv4 in. */ +#define VIRTIO_NET_F_GUEST_USO6 55/* Guest can handle USOv6 in. */ +#define VIRTIO_NET_F_HOST_USO 56/* Host can handle USO in. */ +#define VIRTIO_NET_F_HASH_REPORT 57/* Supports hash report */ #define VIRTIO_NET_F_RSS 60/* Supports RSS RX steering */ #define VIRTIO_NET_F_RSC_EXT 61/* extended coalescing info */
@@ -130,6 +134,7 @@ struct virtio_net_hdr_v1 {#define VIRTIO_NET_HDR_GSO_TCPV4 1/* GSO frame, IPv4 TCP (TSO) */ #define VIRTIO_NET_HDR_GSO_UDP 3/* GSO frame, IPv4 UDP (UFO) */ #define VIRTIO_NET_HDR_GSO_TCPV6 4/* GSO frame, IPv6 TCP */ +#define VIRTIO_NET_HDR_GSO_UDP_L4 5/* GSO frame, IPv4& IPv6 UDP (USO) */ #define VIRTIO_NET_HDR_GSO_ECN 0x80/* TCP has ECN set */ __u8 gso_type;__virtio16 hdr_len;/* Ethernet + IP + tcp/udp hdrs */
Signature · 6 lines
2.37.2 _______________________________________________ Virtualization mailing list [email protected] https://lists.linuxfoundation.org/mailman/listinfo/virtualization
PATCH v3 5/6 linux/virtio_net.h: Support USO offload in vnet header.
In reply to Andrew MelnychenkoNow, it's possible to convert USO vnet packets from/to skb. Added support for GSO_UDP_L4 offload.
--- include/linux/virtio_net.h | 9 +++++++++ 1 file changed, 9 insertions(+)
include/linux/virtio_net.h +9 -0
@@ -15,6 +15,7 @@ static inline bool virtio_net_hdr_match_proto(__be16 protocol, __u8 gso_type)case VIRTIO_NET_HDR_GSO_TCPV6 :return protocol ==cpu_to_be16 (ETH_P_IPV6 );case VIRTIO_NET_HDR_GSO_UDP : +case VIRTIO_NET_HDR_GSO_UDP_L4 :return protocol ==cpu_to_be16 (ETH_P_IP ) || protocol ==cpu_to_be16 (ETH_P_IPV6 );default :
@@ -31,6 +32,7 @@ static inline int virtio_net_hdr_set_proto(struct sk_buff *skb,switch (hdr->gso_type & ~VIRTIO_NET_HDR_GSO_ECN ) {case VIRTIO_NET_HDR_GSO_TCPV4 :case VIRTIO_NET_HDR_GSO_UDP : +case VIRTIO_NET_HDR_GSO_UDP_L4 : skb->protocol =cpu_to_be16 (ETH_P_IP );break ;case VIRTIO_NET_HDR_GSO_TCPV6 :
@@ -69,6 +71,11 @@ static inline int virtio_net_hdr_to_skb(struct sk_buff *skb, ip_proto =IPPROTO_UDP ; thlen =sizeof (struct udphdr );break ; +case VIRTIO_NET_HDR_GSO_UDP_L4 : + gso_type =SKB_GSO_UDP_L4 ; + ip_proto =IPPROTO_UDP ; + thlen =sizeof (struct udphdr ); +break ;default :return -EINVAL ; }
@@ -182,6 +189,8 @@ static inline int virtio_net_hdr_from_skb(const struct sk_buff *skb, hdr->gso_type =VIRTIO_NET_HDR_GSO_TCPV4 ; else if (sinfo->gso_type & SKB_GSO_TCPV6) hdr->gso_type = VIRTIO_NET_HDR_GSO_TCPV6; + else if (sinfo->gso_type & SKB_GSO_UDP_L4) + hdr->gso_type = VIRTIO_NET_HDR_GSO_UDP_L4; else return -EINVAL;if (sinfo->gso_type &SKB_GSO_TCP_ECN )
Signature · 6 lines
2.37.2 _______________________________________________ Virtualization mailing list [email protected] https://lists.linuxfoundation.org/mailman/listinfo/virtualization
PATCH v3 6/6 drivers/net/virtio_net.c: Added USO support.
In reply to Andrew MelnychenkoNow, it possible to enable GSO_UDP_L4("tx-udp-segmentation") for VirtioNet.
--- drivers/net/virtio_net.c | 19 +++++++++++++++---- 1 file changed, 15 insertions(+), 4 deletions(-)
drivers/net/virtio_net.c +15 -4
@@ -60,13 +60,17 @@ static const unsigned long guest_offloads[] = { VIRTIO_NET_F_GUEST_TSO6,VIRTIO_NET_F_GUEST_ECN ,VIRTIO_NET_F_GUEST_UFO , -VIRTIO_NET_F_GUEST_CSUM +VIRTIO_NET_F_GUEST_CSUM , +VIRTIO_NET_F_GUEST_USO4 , +VIRTIO_NET_F_GUEST_USO6 };#define GUEST_OFFLOAD_GRO_HW_MASK ((1ULL << VIRTIO_NET_F_GUEST_TSO4) | \ (1ULL << VIRTIO_NET_F_GUEST_TSO6) | \ (1ULL << VIRTIO_NET_F_GUEST_ECN) | \ - (1ULL << VIRTIO_NET_F_GUEST_UFO)) + (1ULL << VIRTIO_NET_F_GUEST_UFO) | \ + (1ULL << VIRTIO_NET_F_GUEST_USO4) | \ + (1ULL << VIRTIO_NET_F_GUEST_USO6))struct virtnet_stat_desc {char desc[ETH_GSTRING_LEN ];
@@ -3082,7 +3086,9 @@ static int virtnet_xdp_set(struct net_device *dev, struct bpf_prog *prog, virtio_has_feature(vi->vdev, VIRTIO_NET_F_GUEST_TSO6) || virtio_has_feature(vi->vdev, VIRTIO_NET_F_GUEST_ECN) || virtio_has_feature(vi->vdev, VIRTIO_NET_F_GUEST_UFO) || - virtio_has_feature(vi->vdev, VIRTIO_NET_F_GUEST_CSUM))) { + virtio_has_feature(vi->vdev, VIRTIO_NET_F_GUEST_CSUM) || + virtio_has_feature(vi->vdev, VIRTIO_NET_F_GUEST_USO4) || + virtio_has_feature(vi->vdev, VIRTIO_NET_F_GUEST_USO6))) {NL_SET_ERR_MSG_MOD (extack,"Can't set XDP while host is implementing GRO_HW/CSUM, disable GRO_HW/CSUM first" );return -EOPNOTSUPP ; }
@@ -3734,6 +3740,8 @@ static int virtnet_probe(struct virtio_device *vdev) dev->hw_features |=NETIF_F_TSO6 ;if (virtio_has_feature (vdev,VIRTIO_NET_F_HOST_ECN )) dev->hw_features |=NETIF_F_TSO_ECN ; +if (virtio_has_feature (vdev,VIRTIO_NET_F_HOST_USO )) + dev->hw_features |=NETIF_F_GSO_UDP_L4 ; dev->features |=NETIF_F_GSO_ROBUST ;
@@ -3780,7 +3788,9 @@ static int virtnet_probe(struct virtio_device *vdev)if (virtio_has_feature (vdev,VIRTIO_NET_F_GUEST_TSO4 ) ||virtio_has_feature (vdev,VIRTIO_NET_F_GUEST_TSO6 ) ||virtio_has_feature (vdev,VIRTIO_NET_F_GUEST_ECN ) || -virtio_has_feature (vdev,VIRTIO_NET_F_GUEST_UFO )) +virtio_has_feature (vdev,VIRTIO_NET_F_GUEST_UFO ) || + (virtio_has_feature (vdev,VIRTIO_NET_F_GUEST_USO4 ) && +virtio_has_feature (vdev,VIRTIO_NET_F_GUEST_USO6 ))) vi->big_packets = true;if (virtio_has_feature (vdev,VIRTIO_NET_F_MRG_RXBUF ))
@@ -4021,6 +4031,7 @@ static struct virtio_device_id id_table[] = {VIRTIO_NET_F_HOST_TSO4 ,VIRTIO_NET_F_HOST_UFO ,VIRTIO_NET_F_HOST_TSO6 , \VIRTIO_NET_F_HOST_ECN ,VIRTIO_NET_F_GUEST_TSO4 ,VIRTIO_NET_F_GUEST_TSO6 , \VIRTIO_NET_F_GUEST_ECN ,VIRTIO_NET_F_GUEST_UFO , \ +VIRTIO_NET_F_HOST_USO ,VIRTIO_NET_F_GUEST_USO4 ,VIRTIO_NET_F_GUEST_USO6 , \VIRTIO_NET_F_MRG_RXBUF ,VIRTIO_NET_F_STATUS ,VIRTIO_NET_F_CTRL_VQ , \VIRTIO_NET_F_CTRL_RX ,VIRTIO_NET_F_CTRL_VLAN , \ VIRTIO_NET_F_GUEST_ANNOUNCE, VIRTIO_NET_F_MQ, \
Signature · 6 lines
2.37.2 _______________________________________________ Virtualization mailing list [email protected] https://lists.linuxfoundation.org/mailman/listinfo/virtualization
Quoted 4 lines · Andrew Melnychenko
Added new offloads for TUN devices
TUN_F_USO4andTUN_F_USO6. Technically they enableNETIF_F_GSO_UDP_L4
(and only if USO4 & USO6 are set simultaneously). It allows the transmission of large UDP packets.
Please spell out USO at least once in the cover letter to make sure the acronym is clear.
Quoted 4 lines
Different features USO4 and USO6 are required for qemu where Windows guests can enable disable USO receives for IPv4 and IPv6 separately.
On the other side, Linux can't really differentiate USO4 and USO6, for now.
Why is that and what is needed for Linux to differentiate between the 2 protocols?
Quoted 27 lines
For now, to enable USO for TUN it requires enabling USO4 and USO6 together. In the future, there would be a mechanism to control
UDP_L4GSO separately.New types for virtio-net already in virtio-net specification:
https://github.com/oasis-tcs/virtio-spec/issues/120Test it WIP Qemu https://github.com/daynix/qemu/tree/USOv3
Andrew (5):
uapi/linux/if_tun.h: Added new offload types for USO4/6. driver/net/tun: Added features for USO.uapi/linux/virtio_net.h: Added USO types.linux/virtio_net.h: Support USO offload in vnet header.drivers/net/virtio_net.c: Added USO support.Andrew Melnychenko (1):
udp: allow header check for dodgyGSO_UDP_L4packets.drivers/net/tap.c | 10 ++++++++-- drivers/net/tun.c | 8 +++++++- drivers/net/virtio_net.c | 19 +++++++++++++++---- include/linux/virtio_net.h | 9 +++++++++ include/uapi/linux/if_tun.h | 2 ++ include/uapi/linux/virtio_net.h | 5 +++++ net/ipv4/udp_offload.c | 2 +- 7 files changed, 47 insertions(+), 8 deletions(-)
Hi all,
Quoted 9 lines · David Ahern
Quoted 4 lines · Andrew Melnychenko
Added new offloads for TUN devices
TUN_F_USO4andTUN_F_USO6. Technically they enableNETIF_F_GSO_UDP_L4
(and only if USO4 & USO6 are set simultaneously). It allows the transmission of large UDP packets.Please spell out USO at least once in the cover letter to make sure the acronym is clear.
USO - UDP Segmentation Offload. Ability to split UDP packets into several segments.
Allows sending/receiving big UDP packets. At some point, it looks like UFO(fragmentation),
but each segment contains a UDP header.
Quoted 8 lines
Quoted 5 lines
Different features USO4 and USO6 are required for qemu where Windows guests can enable disable USO receives for IPv4 and IPv6 separately.
On the other side, Linux can't really differentiate USO4 and USO6, for now.Why is that and what is needed for Linux to differentiate between the 2 protocols?
Well, this feature requires for Windows VM guest interaction. Windows may have a separate configuration for USO4/USO6. Currently, we support Windows guests with enabled USO4 and USO6 simultaneously.
To implement this on Linux host, will require at least one additional netdev feature
and changes in Linux network stack. Discussion about that will be in the future after
some research.
Quoted 29 lines
Quoted 28 lines
For now, to enable USO for TUN it requires enabling USO4 and USO6 together. In the future, there would be a mechanism to control
UDP_L4GSO separately.New types for virtio-net already in virtio-net specification:
https://github.com/oasis-tcs/virtio-spec/issues/120Test it WIP Qemu https://github.com/daynix/qemu/tree/USOv3
Andrew (5):
uapi/linux/if_tun.h: Added new offload types for USO4/6. driver/net/tun: Added features for USO.uapi/linux/virtio_net.h: Added USO types.linux/virtio_net.h: Support USO offload in vnet header.drivers/net/virtio_net.c: Added USO support.Andrew Melnychenko (1):
udp: allow header check for dodgyGSO_UDP_L4packets.drivers/net/tap.c | 10 ++++++++-- drivers/net/tun.c | 8 +++++++- drivers/net/virtio_net.c | 19 +++++++++++++++---- include/linux/virtio_net.h | 9 +++++++++ include/uapi/linux/if_tun.h | 2 ++ include/uapi/linux/virtio_net.h | 5 +++++ net/ipv4/udp_offload.c | 2 +- 7 files changed, 47 insertions(+), 8 deletions(-)