Skip to contentSkip to conversation

virtio-net: define UDP tunnel offload

PATCH Superseded Paolo Abeni 3 participants 27 messages Jul 29, 2024 – Aug 2, 2024

v6 of 11 Superseded Latest is v11 →

lore
Share & tools
lore
#0 lore raw

v6 cover virtio-net: define UDP tunnel offload

UDP tunnel usage is ubiquitous in container deployment, and the ability to offload UDP encapsulated GSO traffic impacts greatly the performances and the CPU utilization of such use cases.

This series introduces separate features to handle both UDP tunnel segmentation offload (patch 1) and UDP tunnel outer checksum offload (patch 2)

Changes from v5:
- split in 2 patches
- dropped outer_mh_offset field
- many csum related clarification

Paolo Abeni (2):
virtio-net: define UDP tunnel segmentation offload feature
virtio-net: define UDP tunnel checksum offload feature

 device-types/net/description.tex | 227 +++++++++++++++++++++++++++++--
 1 file changed, 218 insertions(+), 9 deletions(-)
Signature · 1 line
2.45.2
#1 lore raw

PATCH v6 1/2 virtio-net: define UDP tunnel segmentation offload feature

The VIRTIO_NET_HDR_GSO_UDP_TUNNEL is a gso_type flag allowing GSO over UDP tunnel. It can be negotiated on both the host and guest sides.

One constraint addressed here is that the virtio side (either device or driver) receiving a UDP tunneled GSO packet must be able to reconstruct completely the inner and outer headers offset - to allow for later GSO.

To accommodate such need, new fields are introduced in the virtio_net
header: outer_th_offset, inner_protocol_type and inner_nh_offset. They map directly to the corresponding header information. The inner transport header is implied by the (inner) checksum offload.

Those fields are required because a virtio device H/W implementation performing segmentation for UDP tunneled packet will need to touch the outer transport protocol (for the UDP length filed), the inner network protocol (for the total length field, in the IPv4 case).

Note that segmentation will additionally need to touch
the outer network protocol and the inner transport protocol. The first is implied/easily found with trivial parsing, the latter is identified by the existing csum_start field.

Note that there is no concept of UDP tunnel type negotiation (e.g. vxlan, geneve, vxlan-gpe, etc.), as a virtio device H/W implementation can perform segmentation for every possible UDP-tunnel given the specified new fields.
In the reverse direction, if a virtio device H/W implementation receives some traffic for an unknown or unsupported UDP tunnel, it will simply not aggregate the wire packet in GSO one.

---
v5 -> v6:
- split in 2 patches
- dropped the unneeded outer_mh_offset field
https://lore.kernel.org/virtio-comment/56ccba136272593d0d2af0303600c06f26bd84a1.1718621522.git.pabeni@redhat.com/

v4 -> v5:
- add separate negotiation for UDP_TUNNEL_CSUM
- much more verbose specification of outer csum handling
- UDP_TUNNEL_CSUM requires GSO_UDP_TUNNEL
https://lore.kernel.org/virtio-comment/cb89d3d6b6a7e4a7f13e60dd3fded5e950727890.1716448766.git.pabeni@redhat.com/

v3 -> v4:
- new virtio_net_hdr fields depends on F_HOST_UDP_TUNNEL_GSO or F_GUEST_UDP_TUNNEL_GSO (Stefano)
- Extended the changelog to answer Jason's questions.
- Clarified outer csum handling (Jason)
https://lore.kernel.org/virtio-dev/f0bf2db203f8061a3ecb50b7a48cf26d8f3c7f68.1716193086.git.pabeni@redhat.com

v2 -> v3:
- UDP_TUNNEL -> UDP_TUNNEL_GSO
- add explicit fields for the inner meta-data
- more verbose changelog
https://lists.oasis-open.org/archives/virtio-dev/202206/msg00026.html

v1 -> v2:
- explicitly state that the outer header probing is mandatory
- explicitly state that GSO_UDP is not allowed with GSO_UDP_TUNNEL
- clarify hdr_len usage
- clarify UDP_TUNNEL_CSUM bit usage
- fix a few typos
https://lists.oasis-open.org/archives/virtio-dev/202205/msg00037.html

---
 device-types/net/description.tex | 118 +++++++++++++++++++++++++++++--
 1 file changed, 112 insertions(+), 6 deletions(-)
device-types/net/description.tex +112 -6
index 76585b0..eda294d 100644
@@ -88,6 +88,12 @@ \subsection{Feature bits}\label{sec:Device Types / Network Device / Feature bits
 \item[VIRTIO_NET_F_CTRL_MAC_ADDR(23)] Set MAC address through control
     channel.
 
+\item[VIRTIO_NET_F_GUEST_UDP_TUNNEL_GSO (47)] Driver can receive GSO packets
+  carried by a UDP tunnel.
+
+\item[VIRTIO_NET_F_HOST_UDP_TUNNEL_GSO (49)] Device can receive GSO packets
+  carried by a UDP tunnel.
+
 \item[VIRTIO_NET_F_HASH_TUNNEL(51)] Device supports inner header hash for encapsulated packets.
 
 \item[VIRTIO_NET_F_VQ_NOTF_COAL(52)] Device supports virtqueue notification coalescing.
@@ -133,12 +139,16 @@ \subsubsection{Feature bit requirements}\label{sec:Device Types / Network Device
 \item[VIRTIO_NET_F_GUEST_UFO] Requires VIRTIO_NET_F_GUEST_CSUM.
 \item[VIRTIO_NET_F_GUEST_USO4] Requires VIRTIO_NET_F_GUEST_CSUM.
 \item[VIRTIO_NET_F_GUEST_USO6] Requires VIRTIO_NET_F_GUEST_CSUM.
+\item[VIRTIO_NET_F_GUEST_UDP_TUNNEL_GSO] Requires both VIRTIO_NET_F_GUEST_TSO4 and VIRTIO_NET_F_GUEST_TSO6, or
+   both VIRTIO_NET_F_GUEST_USO4 and VIRTIO_NET_F_GUEST_USO6.
 
 \item[VIRTIO_NET_F_HOST_TSO4] Requires VIRTIO_NET_F_CSUM.
 \item[VIRTIO_NET_F_HOST_TSO6] Requires VIRTIO_NET_F_CSUM.
 \item[VIRTIO_NET_F_HOST_ECN] Requires VIRTIO_NET_F_HOST_TSO4 or VIRTIO_NET_F_HOST_TSO6.
 \item[VIRTIO_NET_F_HOST_UFO] Requires VIRTIO_NET_F_CSUM.
 \item[VIRTIO_NET_F_HOST_USO] Requires VIRTIO_NET_F_CSUM.
+\item[VIRTIO_NET_F_HOST_UDP_TUNNEL_GSO] Requires both VIRTIO_NET_F_HOST_TSO4 and VIRTIO_NET_F_HOST_TSO6, or
+   both VIRTIO_NET_F_GUEST_USO4 or VIRTIO_NET_F_GUEST_USO6.
 
 \item[VIRTIO_NET_F_CTRL_RX] Requires VIRTIO_NET_F_CTRL_VQ.
 \item[VIRTIO_NET_F_CTRL_VLAN] Requires VIRTIO_NET_F_CTRL_VQ.
@@ -375,6 +385,12 @@ \subsection{Device Initialization}\label{sec:Device Types / Network Device / Dev
   TCP), VIRTIO_NET_F_HOST_TSO6 (IPv6 TCP), VIRTIO_NET_F_HOST_UFO
   (UDP fragmentation) and VIRTIO_NET_F_HOST_USO (UDP segmentation) features.
 
+\item If any TCP or UDP segmentation feature is negotiated, a driver can
+  use TCP segmentation or UDP segmentation on top of UDP encapsulation
+  offload, when the outer header does not require checksumming - e.g.
+  the outer UDP checksum is zero - by negotiating the
+  VIRTIO_NET_F_HOST_UDP_TUNNEL_GSO feature.
+
 \item The converse features are also available: a driver can save
   the virtual device some work by negotiating these features.\note{For example, a network packet transported between two guests on
 the same system might not need checksumming at all, nor segmentation,
@@ -382,8 +398,9 @@ \subsection{Device Initialization}\label{sec:Device Types / Network Device / Dev
    The VIRTIO_NET_F_GUEST_CSUM feature indicates that partially
   checksummed packets can be received, and if it can do that then
   the VIRTIO_NET_F_GUEST_TSO4, VIRTIO_NET_F_GUEST_TSO6,
-  VIRTIO_NET_F_GUEST_UFO, VIRTIO_NET_F_GUEST_ECN, VIRTIO_NET_F_GUEST_USO4
-  and VIRTIO_NET_F_GUEST_USO6 are the input equivalents of the features described above.
+  VIRTIO_NET_F_GUEST_UFO, VIRTIO_NET_F_GUEST_ECN, VIRTIO_NET_F_GUEST_USO4,
+  VIRTIO_NET_F_GUEST_USO6 and VIRTIO_NET_F_GUEST_UDP_TUNNEL_GSO
+  are the input equivalents of the features described above.
   See \ref{sec:Device Types / Network Device / Device Operation /
 Setting Up Receive Buffers}~\nameref{sec:Device Types / Network
 Device / Device Operation / Setting Up Receive Buffers} and
@@ -413,6 +430,7 @@ \subsection{Device Operation}\label{sec:Device Types / Network Device / Device O
 #define VIRTIO_NET_HDR_GSO_UDP         3
 #define VIRTIO_NET_HDR_GSO_TCPV6       4
 #define VIRTIO_NET_HDR_GSO_UDP_L4      5
+#define VIRTIO_NET_HDR_GSO_UDP_TUNNEL 0x40
 #define VIRTIO_NET_HDR_GSO_ECN      0x80
         u8 gso_type;
         le16 hdr_len;
@@ -423,6 +441,9 @@ \subsection{Device Operation}\label{sec:Device Types / Network Device / Device O
         le32 hash_value;        (Only if VIRTIO_NET_F_HASH_REPORT negotiated)
         le16 hash_report;       (Only if VIRTIO_NET_F_HASH_REPORT negotiated)
         le16 padding_reserved;  (Only if VIRTIO_NET_F_HASH_REPORT negotiated)
+        le16 outer_th_offset    (Only if VIRTIO_NET_F_HOST_UDP_TUNNEL_GSO negotiated)
+        le16 inner_protocol_type;(Only if VIRTIO_NET_F_HOST_UDP_TUNNEL_GSO negotiated)
+        le16 inner_nh_offset;   (Only if VIRTIO_NET_F_HOST_UDP_TUNNEL_GSO negotiated)
 };
 \end{lstlisting}
 
@@ -480,6 +501,8 @@ \subsubsection{Packet Transmission}\label{sec:Device Types / Network Device / De
 followed by the TCP header (with the TCP checksum field 16 bytes
 into that header). \field{csum_start} will be 14+20 = 34 (the TCP
 checksum includes the header), and \field{csum_offset} will be 16.
+If the given packet has the VIRTIO_NET_HDR_GSO_UDP_TUNNEL bit set,
+the above checksum fields refer to the inner header checksum.
 \end{note}
 
 \item If the driver negotiated
@@ -516,6 +539,26 @@ \subsubsection{Packet Transmission}\label{sec:Device Types / Network Device / De
 specifically in the protocol.}.
    \end{itemize}
 
+\item If the driver negotiated the VIRTIO_NET_F_HOST_UDP_TUNNEL_GSO feature,
+  the VIRTIO_NET_HDR_GSO_UDP_TUNNEL bit in \field{gso_type} indicates that
+  the GSO protocol is encapsulated in a UDP tunnel.
+  The outer UDP header must not require checksumming, e.g. the outer UDP checksum
+  must be zero.
+  The other tunnel-related fields indicate how to replicate the packet
+  headers to cut it into smaller packets:
+
+  \begin{itemize}
+  \item \field{outer_th_offset} field indicates the outer transport header within
+      the packet. Note that such field differs from \field{csum_start} as the latter
+      points to the inner transport header within the packet.
+
+  \item \field{inner_protocol_type} field indicates the ethernet type of the inner
+      protocol.
+
+  \item \field{inner_nh_offset} field indicates the inner network header within
+      the packet.
+  \end{itemize}
+
 \item \field{num_buffers} is set to zero.  This field is unused on transmitted packets.
 
 \item The header and packet are added as one output descriptor to the
@@ -557,6 +600,18 @@ \subsubsection{Packet Transmission}\label{sec:Device Types / Network Device / De
 driver MUST set the VIRTIO_NET_HDR_GSO_ECN bit in
 \field{gso_type}.
 
+The driver MUST NOT send to the device TCP or UDP GSO packets over UDP tunnel
+requiring segmentation offload, unless the VIRTIO_NET_F_HOST_UDP_TUNNEL_GSO is
+negotiated, in which case the driver MUST set the VIRTIO_NET_HDR_GSO_UDP_TUNNEL
+bit in \field{gso_type}.
+
+The driver MUST NOT send to the device TCP or UDP GSO packets over UDP tunnel
+requiring segmentation and outer UDP checksum offload.
+
+The driver MUST NOT set the VIRTIO_NET_HDR_GSO_UDP_TUNNEL together with
+VIRTIO_NET_HDR_GSO_UDP, as the latter is deprecated in favor of UDP_L4
+and no new features will support it.
+
 If the VIRTIO_NET_F_CSUM feature has been negotiated, the
 driver MAY set the VIRTIO_NET_HDR_F_NEEDS_CSUM bit in
 \field{flags}, if so:
@@ -598,6 +653,28 @@ \subsubsection{Packet Transmission}\label{sec:Device Types / Network Device / De
 	header.
 \end{itemize}
 
+If the VIRTIO_NET_F_HOST_UDP_TUNNEL_GSO option has been negotiated, the
+driver MAY set the VIRTIO_NET_HDR_GSO_UDP_TUNNEL bit in \field{gso_type}, if so:
+\begin{itemize}
+\item the driver MUST set \field{outer_th_offset} to the outer UDP header
+  offset, \field{inner_protocol_type} to the ethernet type of the inner
+  protocol, and \field{inner_nh_offset} to the inner network header offset.
+\end{itemize}
+
+If the bit VIRTIO_NET_HDR_GSO_UDP_TUNNEL in \field{gso_type} is set, the outer
+UDP header MUST NOT require checksumming. That is the, the outer UDP checksum
+value must be 0, or the valid complete checksum for such header.
+
+\begin{note}
+The valid complete checksum of the outer UDP header of individual segments
+can be computed by the driver prior to segmentation only if the GSO packet
+size is a multiple of \field{gso_size}.
+\end{note}
+
+If the VIRTIO_NET_F_HOST_UDP_TUNNEL_GSO option has not
+been negotiated, the driver MUST NOT set the VIRTIO_NET_HDR_F_UDP_TUNNEL bit
+in \field{gso_type}.
+
 The driver SHOULD accept the VIRTIO_NET_F_GUEST_HDRLEN feature if it has
 been offered, and if it's able to provide the exact header length.
 
@@ -633,6 +710,10 @@ \subsubsection{Packet Transmission}\label{sec:Device Types / Network Device / De
 	\end{note}
 \end{itemize}
 
+If VIRTIO_NET_HDR_GSO_UDP_TUNNEL bit in \field{gso_type} is not set, the
+device MUST NOT use the \field{outer_th_offset}, \field{inner_protocol_type}
+ and \field{inner_nh_offset}.
+
 If VIRTIO_NET_HDR_F_NEEDS_CSUM is not set, the device MUST NOT
 rely on the packet checksum being correct.
 \paragraph{Packet Transmission Interrupt}\label{sec:Device Types / Network Device / Device Operation / Packet Transmission / Packet Transmission Interrupt}
@@ -727,8 +808,8 @@ \subsubsection{Processing of Incoming Packets}\label{sec:Device Types / Network
   has been validated.
 \end{enumerate}
 
-Additionally, VIRTIO_NET_F_GUEST_CSUM, TSO4, TSO6, UDP and ECN
-features enable receive checksum, large receive offload and ECN
+Additionally, VIRTIO_NET_F_GUEST_CSUM, TSO4, TSO6, UDP, UDP_TUNNEL
+and ECN features enable receive checksum, large receive offload and ECN
 support which are the input equivalents of the transmit checksum,
 transmit segmentation offloading and ECN features, as described
 in \ref{sec:Device Types / Network Device / Device Operation /
@@ -750,8 +831,14 @@ \subsubsection{Processing of Incoming Packets}\label{sec:Device Types / Network
   from \field{csum_start} and any preceding checksums
   have been validated.  The checksum on the packet is incomplete and
   if bit VIRTIO_NET_HDR_F_RSC_INFO is not set in \field{flags},
-  then \field{csum_start} and \field{csum_offset} indicate how to calculate it
-  (see Packet Transmission point 1).
+  then \field{csum_start} and \field{csum_offset} indicate how to calculate it.
+  If the VIRTIO_NET_HDR_GSO_UDP_TUNNEL bit is set, the \field{csum_start} field
+  refers to the inner transport header offset (see Packet Transmission point 1).
+\item If the VIRTIO_NET_F_GUEST_UDP_TUNNEL_GSO option was negotiated and
+  \field{gso_type} is not VIRTIO_NET_HDR_GSO_NONE, the VIRTIO_NET_HDR_GSO_UDP_TUNNEL
+  bit MAY be set. In such case the \field{outer_th_offset}, \field{inner_protocol_type},
+  and \field{inner_nh_offset} fields indicate the corresponding
+  headers information.
 
 \end{enumerate}
 
@@ -796,6 +883,9 @@ \subsubsection{Processing of Incoming Packets}\label{sec:Device Types / Network
 If none of VIRTIO_NET_F_GUEST_USO4 or VIRTIO_NET_F_GUEST_USO6 have been negotiated,
 the device MUST NOT set \field{gso_type} to VIRTIO_NET_HDR_GSO_UDP_L4.
 
+If VIRTIO_NET_F_GUEST_UDP_TUNNEL_GSO is not negotiated the device MUST NOT set
+the VIRTIO_NET_HDR_GSO_UDP_TUNNEL bit in \field{gso_type}.
+
 The device SHOULD NOT send to the driver TCP packets requiring segmentation offload
 which have the Explicit Congestion Notification bit set, unless the
 VIRTIO_NET_F_GUEST_ECN feature is negotiated, in which case the
@@ -819,6 +909,17 @@ \subsubsection{Processing of Incoming Packets}\label{sec:Device Types / Network
 	fully checksummed packet;
 \end{enumerate}
 
+The device MUST NOT send to the driver TCP or UDP GSO packets encapsulated in UDP
+tunnel and requiring segmentation offload, unless the
+VIRTIO_NET_F_GUEST_UDP_TUNNEL_GSO is negotiated, in which case the device MUST set
+the VIRTIO_NET_HDR_GSO_UDP_TUNNEL bit in \field{gso_type} and MUST set the
+\field{outer_th_offset}, \field{inner_protocol_type} and
+\field{inner_nh_offset} fields to the corresponding header information, and the outer
+UDP header MUST NOT require checksum offload.
+
+The device MUST NOT send the driver TCP or UDP GSO packets encapsulated in UDP
+tunnel and requiring segmentation and outer checksum offload.
+
 If none of the VIRTIO_NET_F_GUEST_TSO4, TSO6, UFO, USO4 or USO6 options have
 been negotiated, the device MUST set \field{gso_type} to
 VIRTIO_NET_HDR_GSO_NONE.
@@ -867,6 +968,10 @@ \subsubsection{Processing of Incoming Packets}\label{sec:Device Types / Network
 VIRTIO_NET_HDR_F_DATA_VALID is set, the driver MUST NOT
 rely on the packet checksum being correct.
 
+If the VIRTIO_NET_HDR_GSO_UDP_TUNNEL_GSO bit in \field{gso_type} is not set,
+the driver MUST NOT use the \field{outer_th_offset}, \field{inner_protocol_type},
+\field{inner_mac_offset} and \field{inner_nh_offset}.
+
 \paragraph{Hash calculation for incoming packets}
 \label{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets}
 
@@ -1624,6 +1729,7 @@ \subsubsection{Control Virtqueue}\label{sec:Device Types / Network Device / Devi
 #define VIRTIO_NET_F_GUEST_TSO6       8
 #define VIRTIO_NET_F_GUEST_ECN        9
 #define VIRTIO_NET_F_GUEST_UFO        10
+#define VIRTIO_NET_F_GUEST_UDP_TUNNEL_GSO  47
 #define VIRTIO_NET_F_GUEST_USO4       54
 #define VIRTIO_NET_F_GUEST_USO6       55
 
Signature · 1 line
2.45.2
#2 lore raw

Re: [PATCH v6 1/2] virtio-net: define UDP tunnel segmentation offload feature

Quoted 32 lines · Paolo Abeni

The VIRTIO_NET_HDR_GSO_UDP_TUNNEL is a gso_type flag allowing GSO over UDP tunnel. It can be negotiated on both the host and guest sides.

One constraint addressed here is that the virtio side (either device or driver) receiving a UDP tunneled GSO packet must be able to reconstruct completely the inner and outer headers offset - to allow for later GSO.

To accommodate such need, new fields are introduced in the virtio_net
header: outer_th_offset, inner_protocol_type and inner_nh_offset. They map directly to the corresponding header information. The inner transport header is implied by the (inner) checksum offload.

Those fields are required because a virtio device H/W implementation performing segmentation for UDP tunneled packet will need to touch the outer transport protocol (for the UDP length filed), the inner network protocol (for the total length field, in the IPv4 case).

Note that segmentation will additionally need to touch
the outer network protocol and the inner transport protocol. The first is implied/easily found with trivial parsing, the latter is identified by the existing csum_start field.

Note that there is no concept of UDP tunnel type negotiation (e.g. vxlan, geneve, vxlan-gpe, etc.), as a virtio device H/W implementation can perform segmentation for every possible UDP-tunnel given the specified new fields.
In the reverse direction, if a virtio device H/W implementation receives some traffic for an unknown or unsupported UDP tunnel, it will simply not aggregate the wire packet in GSO one.

Quoted 1 line

Overall this looks good to me. Thanks for splitting the patches.

You have a working software implementation, right? Will this be sent (as RFC) to the public netdev or so mailing list before this becomes a standard? I assume that there is a standard process for virtio that you are following. I'd just be more confident in a thumbs up if we also review the code that implements the text.

device-types/net/description.tex \subsection{Device Operation}\label{sec:Device Types / Network Device / Device O · line 447

Quoted 8 lines
device-types/net/description.tex +3 -0
@@ -423,6 +441,9 @@ \subsection{Device Operation}\label{sec:Device Types / Network Device / Device O
         le32 hash_value;        (Only if VIRTIO_NET_F_HASH_REPORT negotiated)
         le16 hash_report;       (Only if VIRTIO_NET_F_HASH_REPORT negotiated)
         le16 padding_reserved;  (Only if VIRTIO_NET_F_HASH_REPORT negotiated)
+        le16 outer_th_offset    (Only if VIRTIO_NET_F_HOST_UDP_TUNNEL_GSO negotiated)
+        le16 inner_protocol_type;(Only if VIRTIO_NET_F_HOST_UDP_TUNNEL_GSO negotiated)
+        le16 inner_nh_offset;   (Only if VIRTIO_NET_F_HOST_UDP_TUNNEL_GSO negotiated)
 };

In relation to the above: I recall that expanding the virtio_net_hdr was non-trivial when I last tried to add fields ("[rfc,2/3]
virtio-net: support receive timestamp"). If that is the experience again, it might be worthwhile to expand with enough space to also accommodate a few subsequent extensions. Headroom is not that expensive.

device-types/net/description.tex \subsubsection{Packet Transmission}\label{sec:Device Types / Network Device / De · line 546

Quoted 5 lines
device-types/net/description.tex +5 -0
+\item If the driver negotiated the VIRTIO_NET_F_HOST_UDP_TUNNEL_GSO feature,
+  the VIRTIO_NET_HDR_GSO_UDP_TUNNEL bit in \field{gso_type} indicates that
+  the GSO protocol is encapsulated in a UDP tunnel.
+  The outer UDP header must not require checksumming, e.g. the outer UDP checksum
+  must be zero.

nit: "e.g." should be "i.e.," here

and maybe must be a positive zero (0x0) checksum as defined in UDP RFC 768.

device-types/net/description.tex \subsubsection{Packet Transmission}\label{sec:Device Types / Network Device / De · line 666

Quoted 3 lines
device-types/net/description.tex +3 -0
+If the bit VIRTIO_NET_HDR_GSO_UDP_TUNNEL in \field{gso_type} is set, the outer
+UDP header MUST NOT require checksumming. That is the, the outer UDP checksum
+value must be 0, or the valid complete checksum for such header.

MUST NOT require checksum validation. [..] or the validated complete checksum

(nit-picking at this point, all are suggestions, feel free to ignore).

device-types/net/description.tex \subsubsection{Packet Transmission}\label{sec:Device Types / Network Device / De · line 672

Quoted 6 lines
device-types/net/description.tex +6 -0
+
+\begin{note}
+The valid complete checksum of the outer UDP header of individual segments
+can be computed by the driver prior to segmentation only if the GSO packet
+size is a multiple of \field{gso_size}.
+\end{note}

Might be helpful to explain why. E.g., ", because then all segments have the same size and thus all data included in the outer UDP checksum is the same for every segment. These pre-computed segment length and checksum fields are different from those of the GSO packet."

#3 lore raw

Re: [PATCH v6 1/2] virtio-net: define UDP tunnel segmentation offload feature

Quoted 36 lines · Willem de Bruijn
Quoted 33 lines · Paolo Abeni

The VIRTIO_NET_HDR_GSO_UDP_TUNNEL is a gso_type flag allowing GSO over UDP tunnel. It can be negotiated on both the host and guest sides.

One constraint addressed here is that the virtio side (either device or driver) receiving a UDP tunneled GSO packet must be able to reconstruct completely the inner and outer headers offset - to allow for later GSO.

To accommodate such need, new fields are introduced in the virtio_net
header: outer_th_offset, inner_protocol_type and inner_nh_offset. They map directly to the corresponding header information. The inner transport header is implied by the (inner) checksum offload.

Those fields are required because a virtio device H/W implementation performing segmentation for UDP tunneled packet will need to touch the outer transport protocol (for the UDP length filed), the inner network protocol (for the total length field, in the IPv4 case).

Note that segmentation will additionally need to touch
the outer network protocol and the inner transport protocol. The first is implied/easily found with trivial parsing, the latter is identified by the existing csum_start field.

Note that there is no concept of UDP tunnel type negotiation (e.g. vxlan, geneve, vxlan-gpe, etc.), as a virtio device H/W implementation can perform segmentation for every possible UDP-tunnel given the specified new fields.
In the reverse direction, if a virtio device H/W implementation receives some traffic for an unknown or unsupported UDP tunnel, it will simply not aggregate the wire packet in GSO one.

Quoted 12 lines

device-types/net/description.tex \subsubsection{Packet Transmission}\label{sec:Device Types / Network Device / De · line 672

Quoted 6 lines
device-types/net/description.tex +6 -0
+
+\begin{note}
+The valid complete checksum of the outer UDP header of individual segments
+can be computed by the driver prior to segmentation only if the GSO packet
+size is a multiple of \field{gso_size}.
+\end{note}

Might be helpful to explain why. E.g., ", because then all segments have the same size and thus all data included in the outer UDP checksum is the same for every segment. These pre-computed segment length and checksum fields are different from those of the GSO packet."

Does this part belong in patch 2/2?

#4 lore raw

Re: [PATCH v6 1/2] virtio-net: define UDP tunnel segmentation offload feature

Quoted 50 lines · Willem de Bruijn
Quoted 36 lines · Willem de Bruijn
Quoted 33 lines · Paolo Abeni

The VIRTIO_NET_HDR_GSO_UDP_TUNNEL is a gso_type flag allowing GSO over UDP tunnel. It can be negotiated on both the host and guest sides.

One constraint addressed here is that the virtio side (either device or driver) receiving a UDP tunneled GSO packet must be able to reconstruct completely the inner and outer headers offset - to allow for later GSO.

To accommodate such need, new fields are introduced in the virtio_net
header: outer_th_offset, inner_protocol_type and inner_nh_offset. They map directly to the corresponding header information. The inner transport header is implied by the (inner) checksum offload.

Those fields are required because a virtio device H/W implementation performing segmentation for UDP tunneled packet will need to touch the outer transport protocol (for the UDP length filed), the inner network protocol (for the total length field, in the IPv4 case).

Note that segmentation will additionally need to touch
the outer network protocol and the inner transport protocol. The first is implied/easily found with trivial parsing, the latter is identified by the existing csum_start field.

Note that there is no concept of UDP tunnel type negotiation (e.g. vxlan, geneve, vxlan-gpe, etc.), as a virtio device H/W implementation can perform segmentation for every possible UDP-tunnel given the specified new fields.
In the reverse direction, if a virtio device H/W implementation receives some traffic for an unknown or unsupported UDP tunnel, it will simply not aggregate the wire packet in GSO one.

Quoted 12 lines

device-types/net/description.tex \subsubsection{Packet Transmission}\label{sec:Device Types / Network Device / De · line 672

Quoted 6 lines
device-types/net/description.tex +6 -0
+
+\begin{note}
+The valid complete checksum of the outer UDP header of individual segments
+can be computed by the driver prior to segmentation only if the GSO packet
+size is a multiple of \field{gso_size}.
+\end{note}

Might be helpful to explain why. E.g., ", because then all segments have the same size and thus all data included in the outer UDP checksum is the same for every segment. These pre-computed segment length and checksum fields are different from those of the GSO packet."

Quoted 1 line

Does this part belong in patch 2/2?

Possibly is a bit border-line.
IMHO it could belong to this patch, too. The idea is to specify one of the scenarios what will not require checksum offload.

Thanks,

Paolo

#5 lore raw

Re: [PATCH v6 1/2] virtio-net: define UDP tunnel segmentation offload feature

In reply to Willem de Bruijn
Quoted 5 lines · Willem de Bruijn

You have a working software implementation, right? Will this be sent (as RFC) to the public netdev or so mailing list before this becomes a standard? I assume that there is a standard process for virtio that you are following. I'd just be more confident in a thumbs up if we also review the code that implements the text.

My plan is to write the S/W virtio_net implementation, too, but it's currently not ready. I had a working PoC for the first draft of this changeset, long time ago. Things are changed enough that I'll need a rewrite. I can try to shared such a thing on netdev ASAP (where "soon" here is a very relative terms ;)

Also I don't know the standardization process. I hope for some guidance from Jason and/or Michael.

Quoted 16 lines

device-types/net/description.tex \subsection{Device Operation}\label{sec:Device Types / Network Device / Device O · line 447

Quoted 9 lines
device-types/net/description.tex +3 -0
@@ -423,6 +441,9 @@ \subsection{Device Operation}\label{sec:Device Types / Network Device / Device O
          le32 hash_value;        (Only if VIRTIO_NET_F_HASH_REPORT negotiated)
          le16 hash_report;       (Only if VIRTIO_NET_F_HASH_REPORT negotiated)
          le16 padding_reserved;  (Only if VIRTIO_NET_F_HASH_REPORT negotiated)
+        le16 outer_th_offset    (Only if VIRTIO_NET_F_HOST_UDP_TUNNEL_GSO negotiated)
+        le16 inner_protocol_type;(Only if VIRTIO_NET_F_HOST_UDP_TUNNEL_GSO negotiated)
+        le16 inner_nh_offset;   (Only if VIRTIO_NET_F_HOST_UDP_TUNNEL_GSO negotiated)
  };

In relation to the above: I recall that expanding the virtio_net_hdr was non-trivial when I last tried to add fields ("[rfc,2/3]
virtio-net: support receive timestamp"). If that is the experience again, it might be worthwhile to expand with enough space to also accommodate a few subsequent extensions. Headroom is not that expensive.

Do you mean, by adding 'padding' or 'unused' fields?

Side note: the above is not 32 bit aligned, do we need such aligment?
If we would have VIRTIO_NET_HDR_GSO_UDPv4_L4 and VIRTIO_NET_HDR_GSO_UDPv6_L4 variants, we could avoid entirely 'inner_protocol_type', as the network protocol will be implied by the GSO type itself.

Quoted 8 lines

device-types/net/description.tex \subsubsection{Packet Transmission}\label{sec:Device Types / Network Device / De · line 546

Quoted 5 lines
device-types/net/description.tex +5 -0
+\item If the driver negotiated the VIRTIO_NET_F_HOST_UDP_TUNNEL_GSO feature,
+  the VIRTIO_NET_HDR_GSO_UDP_TUNNEL bit in \field{gso_type} indicates that
+  the GSO protocol is encapsulated in a UDP tunnel.
+  The outer UDP header must not require checksumming, e.g. the outer UDP checksum
+  must be zero.

nit: "e.g." should be "i.e.," here

Quoted 7 lines

and maybe must be a positive zero (0x0) checksum as defined in UDP RFC 768.

device-types/net/description.tex \subsubsection{Packet Transmission}\label{sec:Device Types / Network Device / De · line 666

Quoted 3 lines
device-types/net/description.tex +3 -0
+If the bit VIRTIO_NET_HDR_GSO_UDP_TUNNEL in \field{gso_type} is set, the outer
+UDP header MUST NOT require checksumming. That is the, the outer UDP checksum
+value must be 0, or the valid complete checksum for such header.

MUST NOT require checksum validation. [..] or the validated complete checksum

Ack to all the above.

Thanks!

Paolo

#6 lore raw

Re: [PATCH v6 1/2] virtio-net: define UDP tunnel segmentation offload feature

Quoted 8 lines · Paolo Abeni
Quoted 5 lines · Willem de Bruijn

You have a working software implementation, right? Will this be sent (as RFC) to the public netdev or so mailing list before this becomes a standard? I assume that there is a standard process for virtio that you are following. I'd just be more confident in a thumbs up if we also review the code that implements the text.

Quoted 8 lines

My plan is to write the S/W virtio_net implementation, too, but it's currently not ready. I had a working PoC for the first draft of this changeset, long time ago. Things are changed enough that I'll need a rewrite. I can try to shared such a thing on netdev ASAP (where "soon" here is a very relative terms ;)

Also I don't know the standardization process. I hope for some guidance from Jason and/or Michael.

It should be something like this. When there's no future comment, open an issue https://github.com/oasis-tcs/virtio-spec/issues and ask for a vote. When it passes the voting, it would be merged.

Quoted 18 lines
Quoted 17 lines

device-types/net/description.tex \subsection{Device Operation}\label{sec:Device Types / Network Device / Device O · line 447

Quoted 10 lines
device-types/net/description.tex +3 -0
@@ -423,6 +441,9 @@ \subsection{Device Operation}\label{sec:Device Types / Network Device / Device O
          le32 hash_value;        (Only if VIRTIO_NET_F_HASH_REPORT negotiated)
          le16 hash_report;       (Only if VIRTIO_NET_F_HASH_REPORT negotiated)
          le16 padding_reserved;  (Only if VIRTIO_NET_F_HASH_REPORT negotiated)
+        le16 outer_th_offset    (Only if VIRTIO_NET_F_HOST_UDP_TUNNEL_GSO negotiated)
+        le16 inner_protocol_type;(Only if VIRTIO_NET_F_HOST_UDP_TUNNEL_GSO negotiated)
+        le16 inner_nh_offset;   (Only if VIRTIO_NET_F_HOST_UDP_TUNNEL_GSO negotiated)
  };

In relation to the above: I recall that expanding the virtio_net_hdr was non-trivial when I last tried to add fields ("[rfc,2/3]
virtio-net: support receive timestamp"). If that is the experience again, it might be worthwhile to expand with enough space to also accommodate a few subsequent extensions. Headroom is not that expensive.

Yes, but it looks like an independent issue for this.

Quoted 3 lines

Do you mean, by adding 'padding' or 'unused' fields?

Side note: the above is not 32 bit aligned, do we need such aligment?

Maybe, but it should be an independent topic.

Quoted 4 lines

If we would have VIRTIO_NET_HDR_GSO_UDPv4_L4 and VIRTIO_NET_HDR_GSO_UDPv6_L4 variants, we could avoid entirely 'inner_protocol_type', as the network protocol will be implied by the GSO type itself.

I may miss something, but can it be done by just introducing a new gso_type without vnet header extension?

Quoted 17 lines
Quoted 9 lines

device-types/net/description.tex \subsubsection{Packet Transmission}\label{sec:Device Types / Network Device / De · line 546

Quoted 6 lines
device-types/net/description.tex +5 -0
+\item If the driver negotiated the VIRTIO_NET_F_HOST_UDP_TUNNEL_GSO feature,
+  the VIRTIO_NET_HDR_GSO_UDP_TUNNEL bit in \field{gso_type} indicates that
+  the GSO protocol is encapsulated in a UDP tunnel.
+  The outer UDP header must not require checksumming, e.g. the outer UDP checksum
+  must be zero.

nit: "e.g." should be "i.e.," here

Quoted 7 lines

and maybe must be a positive zero (0x0) checksum as defined in UDP RFC 768.

device-types/net/description.tex \subsubsection{Packet Transmission}\label{sec:Device Types / Network Device / De · line 666

Quoted 3 lines
device-types/net/description.tex +3 -0
+If the bit VIRTIO_NET_HDR_GSO_UDP_TUNNEL in \field{gso_type} is set, the outer
+UDP header MUST NOT require checksumming. That is the, the outer UDP checksum
+value must be 0, or the valid complete checksum for such header.

MUST NOT require checksum validation. [..] or the validated complete checksum

Quoted 6 lines

Ack to all the above.

Thanks!

Paolo

Thanks

#7 lore raw

Re: [PATCH v6 1/2] virtio-net: define UDP tunnel segmentation offload feature

Quoted 10 lines · Jason Wang
Quoted 9 lines · Paolo Abeni
Quoted 8 lines · Willem de Bruijn
Quoted 8 lines
device-types/net/description.tex +3 -0
@@ -423,6 +441,9 @@ \subsection{Device Operation}\label{sec:Device Types / Network Device / Device O
           le32 hash_value;        (Only if VIRTIO_NET_F_HASH_REPORT negotiated)
           le16 hash_report;       (Only if VIRTIO_NET_F_HASH_REPORT negotiated)
           le16 padding_reserved;  (Only if VIRTIO_NET_F_HASH_REPORT negotiated)
+        le16 outer_th_offset    (Only if VIRTIO_NET_F_HOST_UDP_TUNNEL_GSO negotiated)
+        le16 inner_protocol_type;(Only if VIRTIO_NET_F_HOST_UDP_TUNNEL_GSO negotiated)
+        le16 inner_nh_offset;   (Only if VIRTIO_NET_F_HOST_UDP_TUNNEL_GSO negotiated)
   };

[...]

Quoted 7 lines
Quoted 4 lines

If we would have VIRTIO_NET_HDR_GSO_UDPv4_L4 and VIRTIO_NET_HDR_GSO_UDPv6_L4 variants, we could avoid entirely 'inner_protocol_type', as the network protocol will be implied by the GSO type itself.

I may miss something, but can it be done by just introducing a new gso_type without vnet header extension?

Yep, that is what I mean above: we could replace the
'inner_protocol_type' field - which carries a single bit information, inner ipv4 vs ipv6 - with some (inner) GSO-type related info.

Unfortunately that last step does not look 110% 'clean' to me, as the inner GSO type is already specified by VIRTIO_NET_HDR_GSO_TCPV4, VIRTIO_NET_HDR_GSO_TCPV6 or VIRTIO_NET_HDR_GSO_UDP_L4. In case of inner TCP, the inner network protocol is already specified, but it's not in case of UDP.

I'm wondering: why the single VIRTIO_NET_HDR_GSO_UDP_L4 value has been preferred to a pair of ipv4/ipv6 variants? Could we add such variants now? Note that I would like such option very much, as it will make this change even more complex, but looks IMHO quite clean.

Otherwise another option would be to allocate another bit in the GSO type field, set such field when VIRTIO_NET_HDR_GSO_UDP_TUNNEL is set, too, and use such field specify the inner header network protocol. I don't like much even this option because the new bit will carry duplicate information in case of TCP, and it's implementation looks bug prone.

I don't see other options, any suggestions/preferences welcome!

Thanks,

Paolo

Quoted 27 lines
Quoted 18 lines
Quoted 10 lines

device-types/net/description.tex \subsubsection{Packet Transmission}\label{sec:Device Types / Network Device / De · line 546

Quoted 7 lines

+\item If the driver negotiated the VIRTIO_NET_F_HOST_UDP_TUNNEL_GSO feature,

device-types/net/description.tex +4 -0
+  the VIRTIO_NET_HDR_GSO_UDP_TUNNEL bit in \field{gso_type} indicates that
+  the GSO protocol is encapsulated in a UDP tunnel.
+  The outer UDP header must not require checksumming, e.g. the outer UDP checksum
+  must be zero.

nit: "e.g." should be "i.e.," here

Quoted 7 lines

and maybe must be a positive zero (0x0) checksum as defined in UDP RFC 768.

device-types/net/description.tex \subsubsection{Packet Transmission}\label{sec:Device Types / Network Device / De · line 666

Quoted 3 lines
device-types/net/description.tex +3 -0
+If the bit VIRTIO_NET_HDR_GSO_UDP_TUNNEL in \field{gso_type} is set, the outer
+UDP header MUST NOT require checksumming. That is the, the outer UDP checksum
+value must be 0, or the valid complete checksum for such header.

MUST NOT require checksum validation. [..] or the validated complete checksum

Quoted 6 lines

Ack to all the above.

Thanks!

Paolo

Thanks

#8 lore raw

Re: [PATCH v6 1/2] virtio-net: define UDP tunnel segmentation offload feature

Quoted 32 lines · Paolo Abeni
Quoted 10 lines · Jason Wang
Quoted 9 lines · Paolo Abeni
Quoted 8 lines · Willem de Bruijn
Quoted 8 lines
device-types/net/description.tex +3 -0
@@ -423,6 +441,9 @@ \subsection{Device Operation}\label{sec:Device Types / Network Device / Device O
            le32 hash_value;        (Only if VIRTIO_NET_F_HASH_REPORT negotiated)
            le16 hash_report;       (Only if VIRTIO_NET_F_HASH_REPORT negotiated)
            le16 padding_reserved;  (Only if VIRTIO_NET_F_HASH_REPORT negotiated)
+        le16 outer_th_offset    (Only if VIRTIO_NET_F_HOST_UDP_TUNNEL_GSO negotiated)
+        le16 inner_protocol_type;(Only if VIRTIO_NET_F_HOST_UDP_TUNNEL_GSO negotiated)
+        le16 inner_nh_offset;   (Only if VIRTIO_NET_F_HOST_UDP_TUNNEL_GSO negotiated)
    };

[...]

Quoted 7 lines
Quoted 4 lines

If we would have VIRTIO_NET_HDR_GSO_UDPv4_L4 and VIRTIO_NET_HDR_GSO_UDPv6_L4 variants, we could avoid entirely 'inner_protocol_type', as the network protocol will be implied by the GSO type itself.

I may miss something, but can it be done by just introducing a new gso_type without vnet header extension?

Yep, that is what I mean above: we could replace the 'inner_protocol_type' field - which carries a single bit information, inner ipv4 vs ipv6 - with some (inner) GSO-type related info.

Unfortunately that last step does not look 110% 'clean' to me, as the inner GSO type is already specified by VIRTIO_NET_HDR_GSO_TCPV4, VIRTIO_NET_HDR_GSO_TCPV6 or
VIRTIO_NET_HDR_GSO_UDP_L4. In case of inner TCP, the inner network protocol is already specified, but it's not in case of UDP.

Quoted 4 lines

I'm wondering: why the single VIRTIO_NET_HDR_GSO_UDP_L4 value has been preferred to a pair of ipv4/ipv6 variants? Could we add such variants now? Note that I would like such option very much, as it will make this change even more complex, but looks IMHO quite clean.

After more pondering, I think implementing this option would be quite a mess. E.g. we would need a feature bit for VIRTIO_NET_HDR_GSO_UDP{4,6}_L4 different from VIRTIO_NET_F_{GUEST,HOST}_USO{4,6,}...

I guess we are better off excluding such option.

Quoted 6 lines

Otherwise another option would be to allocate another bit in the GSO type field, set such field when VIRTIO_NET_HDR_GSO_UDP_TUNNEL is set, too, and use such field specify the inner header network protocol. I don't like much even this option because the new bit will carry duplicate information in case of TCP, and it's implementation looks bug prone.

In practice it would replace 'inner_protocol_type' with:

#define VIRTIO_NET_HDR_GSO_INNER_IPv6 0x20

And set/clearing such bit depending on the inner network protocol type, for VIRTIO_NET_HDR_GSO_UDP_TUNNEL packets.

Processing the virtio net hdr will require validating such bit value with WRT the carried GSO type, i.e. must be set when

gso_type & (VIRTIO_NET_HDR_GSO_TCPV6 | VIRTIO_NET_HDR_GSO_UDP_TUNNEL) == VIRTIO_NET_HDR_GSO_TCPV6 | VIRTIO_NET_HDR_GSO_UDP_TUNNEL

must be cleared in all the other cases

Cheers,

Paolo

#9 lore raw

Re: [PATCH v6 1/2] virtio-net: define UDP tunnel segmentation offload feature

In reply to Paolo Abeni
Quoted 33 lines · Paolo Abeni
Quoted 10 lines · Jason Wang
Quoted 9 lines · Paolo Abeni
Quoted 8 lines · Willem de Bruijn
Quoted 8 lines
device-types/net/description.tex +3 -0
@@ -423,6 +441,9 @@ \subsection{Device Operation}\label{sec:Device Types / Network Device / Device O
           le32 hash_value;        (Only if VIRTIO_NET_F_HASH_REPORT negotiated)
           le16 hash_report;       (Only if VIRTIO_NET_F_HASH_REPORT negotiated)
           le16 padding_reserved;  (Only if VIRTIO_NET_F_HASH_REPORT negotiated)
+        le16 outer_th_offset    (Only if VIRTIO_NET_F_HOST_UDP_TUNNEL_GSO negotiated)
+        le16 inner_protocol_type;(Only if VIRTIO_NET_F_HOST_UDP_TUNNEL_GSO negotiated)
+        le16 inner_nh_offset;   (Only if VIRTIO_NET_F_HOST_UDP_TUNNEL_GSO negotiated)
   };

[...]

Quoted 7 lines
Quoted 4 lines

If we would have VIRTIO_NET_HDR_GSO_UDPv4_L4 and VIRTIO_NET_HDR_GSO_UDPv6_L4 variants, we could avoid entirely 'inner_protocol_type', as the network protocol will be implied by the GSO type itself.

I may miss something, but can it be done by just introducing a new gso_type without vnet header extension?

Yep, that is what I mean above: we could replace the 'inner_protocol_type' field - which carries a single bit information, inner ipv4 vs ipv6 - with some (inner) GSO-type related info.

Unfortunately that last step does not look 110% 'clean' to me, as the inner GSO type is already specified by VIRTIO_NET_HDR_GSO_TCPV4, VIRTIO_NET_HDR_GSO_TCPV6 or
VIRTIO_NET_HDR_GSO_UDP_L4. In case of inner TCP, the inner network protocol is already specified, but it's not in case of UDP.

Quoted 4 lines

I'm wondering: why the single VIRTIO_NET_HDR_GSO_UDP_L4 value has been preferred to a pair of ipv4/ipv6 variants? Could we add such variants now? Note that I would like such option very much, as it will make this change even more complex, but looks IMHO quite clean.

Actually, I point out that when it is proposed. And I was told that it is because Linux doesn't differ that.

Quoted 8 lines

Otherwise another option would be to allocate another bit in the GSO type field, set such field when VIRTIO_NET_HDR_GSO_UDP_TUNNEL is set, too, and use such field specify the inner header network protocol. I don't like much even this option because the new bit will carry duplicate information in case of TCP, and it's implementation looks bug prone.

Quoted 5 lines

I don't see other options, any suggestions/preferences welcome!

Thanks,

Paolo

Thanks

#10 lore raw

Re: [PATCH v6 1/2] virtio-net: define UDP tunnel segmentation offload feature

In reply to Jason Wang
Quoted 16 lines · Jason Wang
Quoted 14 lines · Paolo Abeni
Quoted 5 lines · Willem de Bruijn

You have a working software implementation, right? Will this be sent (as RFC) to the public netdev or so mailing list before this becomes a standard? I assume that there is a standard process for virtio that you are following. I'd just be more confident in a thumbs up if we also review the code that implements the text.

My plan is to write the S/W virtio_net implementation, too, but it's currently not ready. I had a working PoC for the first draft of this changeset, long time ago. Things are changed enough that I'll need a rewrite. I can try to shared such a thing on netdev ASAP (where "soon" here is a very relative terms ;)

Quoted 6 lines
Quoted 2 lines

Also I don't know the standardization process. I hope for some guidance from Jason and/or Michael.

It should be something like this. When there's no future comment, open an issue https://github.com/oasis-tcs/virtio-spec/issues and ask for a vote. When it passes the voting, it would be merged.

So changes to the spec can be merged, because we know that they can be implemented reasonably?

IETF always requires (two, because interoperable protocol definitions) working implementations.

It sounds a bit risky to me to skip this, as you come across the interesting non-obvious details during implementation.

Since implementation is planned anyway, doing at least a rough proof of concept first does not introduce extra work, but reduces risk that the spec contains errors.

Unless it is easy to update the spec before it is numbered and finalized. Then issues can be fixed up.

Anyway, you have gone through this many times, so whatever the process is, I trust that it works. Just sharing my thought as a somewhat uniformed reader.

Quoted 20 lines
Quoted 19 lines
Quoted 18 lines

device-types/net/description.tex \subsection{Device Operation}\label{sec:Device Types / Network Device / Device O · line 447

Quoted 11 lines
device-types/net/description.tex +3 -0
@@ -423,6 +441,9 @@ \subsection{Device Operation}\label{sec:Device Types / Network Device / Device O
          le32 hash_value;        (Only if VIRTIO_NET_F_HASH_REPORT negotiated)
          le16 hash_report;       (Only if VIRTIO_NET_F_HASH_REPORT negotiated)
          le16 padding_reserved;  (Only if VIRTIO_NET_F_HASH_REPORT negotiated)
+        le16 outer_th_offset    (Only if VIRTIO_NET_F_HOST_UDP_TUNNEL_GSO negotiated)
+        le16 inner_protocol_type;(Only if VIRTIO_NET_F_HOST_UDP_TUNNEL_GSO negotiated)
+        le16 inner_nh_offset;   (Only if VIRTIO_NET_F_HOST_UDP_TUNNEL_GSO negotiated)
  };

In relation to the above: I recall that expanding the virtio_net_hdr was non-trivial when I last tried to add fields ("[rfc,2/3]
virtio-net: support receive timestamp"). If that is the experience again, it might be worthwhile to expand with enough space to also accommodate a few subsequent extensions. Headroom is not that expensive.

Quoted 1 line

Yes, but it looks like an independent issue for this.

If this change requires new headroom, then it depends on headroom expansion.

My suggestion is to then expand it significantly immediately, to minimize the number of times we have to change the virtio_net code to support new lengths.

We already have structs virtio_net_hdr, virtio_net_hdr_mrg_rxbuf, virtio_net_hdr_v1, virtio_net_hdr_v1_hash.

Adding a new struct with every new field added just adds lots of churn in the code.

Just a thought, feel free to skip if the maintainers disagree.

Quoted 2 lines
Quoted 2 lines

Do you mean, by adding 'padding' or 'unused' fields?

Yes.

Quoted 9 lines
Quoted 2 lines

Side note: the above is not 32 bit aligned, do we need such aligment?

Maybe, but it should be an independent topic.

Quoted 4 lines

If we would have VIRTIO_NET_HDR_GSO_UDPv4_L4 and VIRTIO_NET_HDR_GSO_UDPv6_L4 variants, we could avoid entirely 'inner_protocol_type', as the network protocol will be implied by the GSO type itself.

But that's too late now? Probably don't want to make this work depending on such a change.

I don't entirely recall why we did not do that immediately. Perhaps it is related to also only having SKB_GSO_UDP_L4.

#11 lore raw

Re: [PATCH v6 1/2] virtio-net: define UDP tunnel segmentation offload feature

Quoted 26 lines · Willem de Bruijn
Quoted 18 lines · Jason Wang
Quoted 16 lines · Paolo Abeni
Quoted 7 lines · Willem de Bruijn
Quoted 7 lines
device-types/net/description.tex +3 -0
@@ -423,6 +441,9 @@ \subsection{Device Operation}\label{sec:Device Types / Network Device / Device O
           le32 hash_value;        (Only if VIRTIO_NET_F_HASH_REPORT negotiated)
           le16 hash_report;       (Only if VIRTIO_NET_F_HASH_REPORT negotiated)
           le16 padding_reserved;  (Only if VIRTIO_NET_F_HASH_REPORT negotiated)
+        le16 outer_th_offset    (Only if VIRTIO_NET_F_HOST_UDP_TUNNEL_GSO negotiated)
+        le16 inner_protocol_type;(Only if VIRTIO_NET_F_HOST_UDP_TUNNEL_GSO negotiated)
+        le16 inner_nh_offset;   (Only if VIRTIO_NET_F_HOST_UDP_TUNNEL_GSO negotiated)
Quoted 8 lines

Review \subsection{Device Operation}\label{sec:Device Types / Network Device / Device O · line 447

Quoted 1 line
(quoted diff) +0 -0
   };

In relation to the above: I recall that expanding the virtio_net_hdr was non-trivial when I last tried to add fields ("[rfc,2/3]
virtio-net: support receive timestamp"). If that is the experience again, it might be worthwhile to expand with enough space to also accommodate a few subsequent extensions. Headroom is not that expensive.

Quoted 1 line

Yes, but it looks like an independent issue for this.

If this change requires new headroom, then it depends on headroom expansion.

My suggestion is to then expand it significantly immediately, to minimize the number of times we have to change the virtio_net code to support new lengths.

Quoted 7 lines

We already have structs virtio_net_hdr, virtio_net_hdr_mrg_rxbuf, virtio_net_hdr_v1, virtio_net_hdr_v1_hash.

Adding a new struct with every new field added just adds lots of churn in the code.

Just a thought, feel free to skip if the maintainers disagree.

I just read the old thread WRT virtio net timestamping.

If we reserve additional headroom space for timestamping here, such room will be available only if UDP_TUNNEL is negotiated, which sounds a bit counterintuitive to me.

I *think* ideally we want a separate virtio net feature for that?

Regarding the virtio_net_hdr parsing complexity increase, I think we should add to the uAPI a struct containing only the new fields, e.g. for UDP tunnel:

struct virtio_net_udp_tunnel {
	__virtio16 outer_th_offset;
	// eventually __virtio16 inner_protocol_type;
	__virtio16 inner_nh_offset;
};

At feature negotiation time both the drivers and the device compute the offset inside the virtio buffer where such struct is placed:

if VIRTIO_NET_F_HASH_REPORT is negotiated such offset is sizeof(struct virtio_net_hdr_v1_hash) elsewhere is sizeof(struct virtio_net_hdr_v1)

At rx time, struct virtio_net_udp_tunnel() is parsed from the above offset.

With all the above in place, for every additional later feature we add just a struct definition and the virtio_net header parsing complexity scales linearly.

Quoted 10 lines
Quoted 4 lines
Quoted 4 lines

If we would have VIRTIO_NET_HDR_GSO_UDPv4_L4 and VIRTIO_NET_HDR_GSO_UDPv6_L4 variants, we could avoid entirely 'inner_protocol_type', as the network protocol will be implied by the GSO type itself.

But that's too late now? Probably don't want to make this work depending on such a change.

I don't entirely recall why we did not do that immediately. Perhaps it is related to also only having SKB_GSO_UDP_L4.

I reached the same conclusion here:

https://lore.kernel.org/virtio-comment/[email protected]/T/#mac694da6e907451d5529116a97510c940bf2edd9

What about reserving an additional bit in gso_type to specify the inner network header protocol?

Thanks,

Paolo

#12 lore raw

Re: [PATCH v6 1/2] virtio-net: define UDP tunnel segmentation offload feature

Quoted 36 lines · Paolo Abeni
Quoted 26 lines · Willem de Bruijn
Quoted 18 lines · Jason Wang
Quoted 16 lines · Paolo Abeni
Quoted 7 lines · Willem de Bruijn
Quoted 7 lines
device-types/net/description.tex +3 -0
@@ -423,6 +441,9 @@ \subsection{Device Operation}\label{sec:Device Types / Network Device / Device O
           le32 hash_value;        (Only if VIRTIO_NET_F_HASH_REPORT negotiated)
           le16 hash_report;       (Only if VIRTIO_NET_F_HASH_REPORT negotiated)
           le16 padding_reserved;  (Only if VIRTIO_NET_F_HASH_REPORT negotiated)
+        le16 outer_th_offset    (Only if VIRTIO_NET_F_HOST_UDP_TUNNEL_GSO negotiated)
+        le16 inner_protocol_type;(Only if VIRTIO_NET_F_HOST_UDP_TUNNEL_GSO negotiated)
+        le16 inner_nh_offset;   (Only if VIRTIO_NET_F_HOST_UDP_TUNNEL_GSO negotiated)
Quoted 8 lines

Review \subsection{Device Operation}\label{sec:Device Types / Network Device / Device O · line 447

Quoted 1 line
(quoted diff) +0 -0
   };

In relation to the above: I recall that expanding the virtio_net_hdr was non-trivial when I last tried to add fields ("[rfc,2/3]
virtio-net: support receive timestamp"). If that is the experience again, it might be worthwhile to expand with enough space to also accommodate a few subsequent extensions. Headroom is not that expensive.

Quoted 1 line

Yes, but it looks like an independent issue for this.

If this change requires new headroom, then it depends on headroom expansion.

My suggestion is to then expand it significantly immediately, to minimize the number of times we have to change the virtio_net code to support new lengths.

Quoted 7 lines

We already have structs virtio_net_hdr, virtio_net_hdr_mrg_rxbuf, virtio_net_hdr_v1, virtio_net_hdr_v1_hash.

Adding a new struct with every new field added just adds lots of churn in the code.

Just a thought, feel free to skip if the maintainers disagree.

Quoted 7 lines

I just read the old thread WRT virtio net timestamping.

If we reserve additional headroom space for timestamping here, such room will be available only if UDP_TUNNEL is negotiated, which sounds a bit counterintuitive to me.

I *think* ideally we want a separate virtio net feature for that?

Fair point. Agreed that extra headroom should not be dependent on this feature.

Quoted 13 lines

Regarding the virtio_net_hdr parsing complexity increase, I think we should add to the uAPI a struct containing only the new fields, e.g. for UDP tunnel:

struct virtio_net_udp_tunnel {
        __virtio16 outer_th_offset;
        // eventually __virtio16 inner_protocol_type;
        __virtio16 inner_nh_offset;
};

At feature negotiation time both the drivers and the device compute the offset inside the virtio buffer where such struct is placed:

Quoted 8 lines

if VIRTIO_NET_F_HASH_REPORT is negotiated such offset is sizeof(struct virtio_net_hdr_v1_hash) elsewhere is sizeof(struct virtio_net_hdr_v1)

At rx time, struct virtio_net_udp_tunnel() is parsed from the above offset.

With all the above in place, for every additional later feature we add just a struct definition and the virtio_net header parsing complexity scales linearly.

Sounds good to me.

Quoted 11 lines
Quoted 10 lines
Quoted 4 lines
Quoted 4 lines

If we would have VIRTIO_NET_HDR_GSO_UDPv4_L4 and VIRTIO_NET_HDR_GSO_UDPv6_L4 variants, we could avoid entirely 'inner_protocol_type', as the network protocol will be implied by the GSO type itself.

But that's too late now? Probably don't want to make this work depending on such a change.

I don't entirely recall why we did not do that immediately. Perhaps it is related to also only having SKB_GSO_UDP_L4.

Quoted 6 lines

I reached the same conclusion here:

https://lore.kernel.org/virtio-comment/[email protected]/T/#mac694da6e907451d5529116a97510c940bf2edd9

What about reserving an additional bit in gso_type to specify the inner network header protocol?

Similar to VIRTIO_NET_HDR_GSO_ECN?

Is that preferable over VIRTIO_NET_HDR_GSO_UDP_TUNNEL_IP4 and .._UDP_TUNNEL_IP6 variants?

And to parsing just this one byte (or nibble) from packet data using inner_nh_offset?

If it's the best of those options, no objections from me. Definitely no need for a u16 if the only options are IPPROTO_IP and IPPROTO_IPV6.

#13 lore raw

Re: [PATCH v6 1/2] virtio-net: define UDP tunnel segmentation offload feature

Quoted 83 lines · Willem de Bruijn
Quoted 36 lines · Paolo Abeni
Quoted 26 lines · Willem de Bruijn
Quoted 18 lines · Jason Wang
Quoted 16 lines · Paolo Abeni
Quoted 7 lines · Willem de Bruijn
Quoted 7 lines
(quoted diff) +3 -0
@@ -423,6 +441,9 @@ \subsection{Device Operation}\label{sec:Device Types / Network Device / Device O
           le32 hash_value;        (Only if VIRTIO_NET_F_HASH_REPORT negotiated)
           le16 hash_report;       (Only if VIRTIO_NET_F_HASH_REPORT negotiated)
           le16 padding_reserved;  (Only if VIRTIO_NET_F_HASH_REPORT negotiated)
+        le16 outer_th_offset    (Only if VIRTIO_NET_F_HOST_UDP_TUNNEL_GSO negotiated)
+        le16 inner_protocol_type;(Only if VIRTIO_NET_F_HOST_UDP_TUNNEL_GSO negotiated)
+        le16 inner_nh_offset;   (Only if VIRTIO_NET_F_HOST_UDP_TUNNEL_GSO negotiated)
Quoted 8 lines

Review \subsection{Device Operation}\label{sec:Device Types / Network Device / Device O · line 447

Quoted 1 line
(quoted diff) +0 -0
   };

In relation to the above: I recall that expanding the virtio_net_hdr was non-trivial when I last tried to add fields ("[rfc,2/3]
virtio-net: support receive timestamp"). If that is the experience again, it might be worthwhile to expand with enough space to also accommodate a few subsequent extensions. Headroom is not that expensive.

Quoted 1 line

Yes, but it looks like an independent issue for this.

If this change requires new headroom, then it depends on headroom expansion.

My suggestion is to then expand it significantly immediately, to minimize the number of times we have to change the virtio_net code to support new lengths.

Quoted 7 lines

We already have structs virtio_net_hdr, virtio_net_hdr_mrg_rxbuf, virtio_net_hdr_v1, virtio_net_hdr_v1_hash.

Adding a new struct with every new field added just adds lots of churn in the code.

Just a thought, feel free to skip if the maintainers disagree.

Quoted 7 lines

I just read the old thread WRT virtio net timestamping.

If we reserve additional headroom space for timestamping here, such room will be available only if UDP_TUNNEL is negotiated, which sounds a bit counterintuitive to me.

I *think* ideally we want a separate virtio net feature for that?

Fair point. Agreed that extra headroom should not be dependent on this feature.

Quoted 13 lines

Regarding the virtio_net_hdr parsing complexity increase, I think we should add to the uAPI a struct containing only the new fields, e.g. for UDP tunnel:

struct virtio_net_udp_tunnel {
        __virtio16 outer_th_offset;
        // eventually __virtio16 inner_protocol_type;
        __virtio16 inner_nh_offset;
};

At feature negotiation time both the drivers and the device compute the offset inside the virtio buffer where such struct is placed:

Quoted 8 lines

if VIRTIO_NET_F_HASH_REPORT is negotiated such offset is sizeof(struct virtio_net_hdr_v1_hash) elsewhere is sizeof(struct virtio_net_hdr_v1)

At rx time, struct virtio_net_udp_tunnel() is parsed from the above offset.

With all the above in place, for every additional later feature we add just a struct definition and the virtio_net header parsing complexity scales linearly.

Sounds good to me.

Quoted 11 lines
Quoted 10 lines
Quoted 4 lines
Quoted 4 lines

If we would have VIRTIO_NET_HDR_GSO_UDPv4_L4 and VIRTIO_NET_HDR_GSO_UDPv6_L4 variants, we could avoid entirely 'inner_protocol_type', as the network protocol will be implied by the GSO type itself.

But that's too late now? Probably don't want to make this work depending on such a change.

I don't entirely recall why we did not do that immediately. Perhaps it is related to also only having SKB_GSO_UDP_L4.

Quoted 8 lines
Quoted 6 lines

I reached the same conclusion here:

https://lore.kernel.org/virtio-comment/[email protected]/T/#mac694da6e907451d5529116a97510c940bf2edd9

What about reserving an additional bit in gso_type to specify the inner network header protocol?

Similar to VIRTIO_NET_HDR_GSO_ECN?

I think so.

Quoted 10 lines

Is that preferable over VIRTIO_NET_HDR_GSO_UDP_TUNNEL_IP4 and .._UDP_TUNNEL_IP6 variants?

And to parsing just this one byte (or nibble) from packet data using inner_nh_offset?

If it's the best of those options, no objections from me. Definitely no need for a u16 if the only options are IPPROTO_IP and IPPROTO_IPV6.

Exactly.

Thanks

#14 lore raw

Re: [PATCH v6 1/2] virtio-net: define UDP tunnel segmentation offload feature

In reply to Willem de Bruijn
Quoted 8 lines · Willem de Bruijn
Quoted 2 lines · Paolo Abeni

What about reserving an additional bit in gso_type to specify the inner network header protocol?

Similar to VIRTIO_NET_HDR_GSO_ECN?

Is that preferable over VIRTIO_NET_HDR_GSO_UDP_TUNNEL_IP4 and .._UDP_TUNNEL_IP6 variants?

Should be basically the same, as VIRTIO_NET_HDR_GSO_UDP_TUNNEL_IP4 VIRTIO_NET_HDR_GSO_UDP_TUNNEL_IP6 will be 2 separate bits in gso_type, right?

Quoted 2 lines

And to parsing just this one byte (or nibble) from packet data using inner_nh_offset?

I think one of the state goal is to avoid parsing. More importantly, accessing the inner network header content this early will cause possibly avoidable (or at least mitigable via prefetch) cache misses.

Quoted 2 lines

If it's the best of those options, no objections from me. Definitely no need for a u16 if the only options are IPPROTO_IP and IPPROTO_IPV6.

Wrapping all the above I'll go for the additional bit for the inner network header type. If I misread something, please LMK:)

Thanks,

Paolo

#15 lore raw

Re: [PATCH v6 1/2] virtio-net: define UDP tunnel segmentation offload feature

Quoted 6 lines · Paolo Abeni
Quoted 4 lines · Willem de Bruijn
Quoted 2 lines · Paolo Abeni

What about reserving an additional bit in gso_type to specify the inner network header protocol?

Quoted 8 lines
Quoted 4 lines

Similar to VIRTIO_NET_HDR_GSO_ECN?

Is that preferable over VIRTIO_NET_HDR_GSO_UDP_TUNNEL_IP4 and .._UDP_TUNNEL_IP6 variants?

Should be basically the same, as VIRTIO_NET_HDR_GSO_UDP_TUNNEL_IP4 VIRTIO_NET_HDR_GSO_UDP_TUNNEL_IP6 will be 2 separate bits in gso_type, right?

One difference is that we have to update other code to use a bitmask.

I'd prefer the two types for that reason.

#16 lore raw

Re: [PATCH v6 1/2] virtio-net: define UDP tunnel segmentation offload feature

Quoted 12 lines · Willem de Bruijn
Quoted 11 lines · Paolo Abeni
Quoted 8 lines · Willem de Bruijn
Quoted 2 lines · Paolo Abeni

What about reserving an additional bit in gso_type to specify the inner network header protocol?

Similar to VIRTIO_NET_HDR_GSO_ECN?

Is that preferable over VIRTIO_NET_HDR_GSO_UDP_TUNNEL_IP4 and .._UDP_TUNNEL_IP6 variants?

Quoted 7 lines
Quoted 3 lines

Should be basically the same, as VIRTIO_NET_HDR_GSO_UDP_TUNNEL_IP4 VIRTIO_NET_HDR_GSO_UDP_TUNNEL_IP6 will be 2 separate bits in gso_type, right?

One difference is that we have to update other code to use a bitmask.

I'd prefer the two types for that reason.

Please pardon me for the high-frequency spamming. I'd love to have this topic well defined.

I'm unsure how to read your suggestion exactly.

VIRTIO_NET_HDR_GSO_UDP_TUNNEL needs to be "combined" (set together) with VIRTIO_NET_HDR_GSO_TCPV4, VIRTIO_NET_HDR_GSO_TCPV6 or VIRTIO_NET_HDR_GSO_UDP_L4, so it's currently defined as a bit flag:

#define VIRTIO_NET_HDR_GSO_UDP_TUNNEL 0x40

From an implementation perspective this allow us reusing the existing code for to fill inner headers related field, masking such bit in the gso_type before invoking the existing virtio_hdr parsing helper - it will be used unmodified.

Due to all the above, I thought the IPv4 and IPv6 variant would be defined as:

#define VIRTIO_NET_HDR_GSO_UDP_TUNNEL_IP4 0x20
#define VIRTIO_NET_HDR_GSO_UDP_TUNNEL_IP6 0x40

How would you define VIRTIO_NET_HDR_GSO_UDP_TUNNEL_IP4 and VIRTIO_NET_HDR_GSO_UDP_TUNNEL_IP6?

Thanks,

Paolo

#17 lore raw

Re: [PATCH v6 1/2] virtio-net: define UDP tunnel segmentation offload feature

Quoted 38 lines · Paolo Abeni
Quoted 12 lines · Willem de Bruijn
Quoted 11 lines · Paolo Abeni
Quoted 8 lines · Willem de Bruijn
Quoted 2 lines · Paolo Abeni

What about reserving an additional bit in gso_type to specify the inner network header protocol?

Similar to VIRTIO_NET_HDR_GSO_ECN?

Is that preferable over VIRTIO_NET_HDR_GSO_UDP_TUNNEL_IP4 and .._UDP_TUNNEL_IP6 variants?

Quoted 7 lines
Quoted 3 lines

Should be basically the same, as VIRTIO_NET_HDR_GSO_UDP_TUNNEL_IP4 VIRTIO_NET_HDR_GSO_UDP_TUNNEL_IP6 will be 2 separate bits in gso_type, right?

One difference is that we have to update other code to use a bitmask.

I'd prefer the two types for that reason.

Please pardon me for the high-frequency spamming. I'd love to have this topic well defined.

I'm unsure how to read your suggestion exactly.

VIRTIO_NET_HDR_GSO_UDP_TUNNEL needs to be "combined" (set together) with VIRTIO_NET_HDR_GSO_TCPV4, VIRTIO_NET_HDR_GSO_TCPV6 or VIRTIO_NET_HDR_GSO_UDP_L4, so it's currently defined as a bit flag:

#define VIRTIO_NET_HDR_GSO_UDP_TUNNEL 0x40

From an implementation perspective this allow us reusing the existing
code for to fill inner headers related field, masking such bit in the gso_type before invoking the existing virtio_hdr parsing helper - it will be used unmodified.

Quoted 8 lines

Due to all the above, I thought the IPv4 and IPv6 variant would be defined as:

#define VIRTIO_NET_HDR_GSO_UDP_TUNNEL_IP4 0x20
#define VIRTIO_NET_HDR_GSO_UDP_TUNNEL_IP6 0x40

How would you define VIRTIO_NET_HDR_GSO_UDP_TUNNEL_IP4 and VIRTIO_NET_HDR_GSO_UDP_TUNNEL_IP6?

Sorry for the confusion. I agree with your proposal.

I had forgotten that it already adds a bit that will have to be masked out in any branch that only deals with the inner type.

#18 lore raw

Re: [PATCH v6 1/2] virtio-net: define UDP tunnel segmentation offload feature

In reply to Paolo Abeni
Quoted 38 lines · Paolo Abeni
Quoted 12 lines · Willem de Bruijn
Quoted 11 lines · Paolo Abeni
Quoted 8 lines · Willem de Bruijn
Quoted 2 lines · Paolo Abeni

What about reserving an additional bit in gso_type to specify the inner network header protocol?

Similar to VIRTIO_NET_HDR_GSO_ECN?

Is that preferable over VIRTIO_NET_HDR_GSO_UDP_TUNNEL_IP4 and .._UDP_TUNNEL_IP6 variants?

Quoted 7 lines
Quoted 3 lines

Should be basically the same, as VIRTIO_NET_HDR_GSO_UDP_TUNNEL_IP4 VIRTIO_NET_HDR_GSO_UDP_TUNNEL_IP6 will be 2 separate bits in gso_type, right?

One difference is that we have to update other code to use a bitmask.

I'd prefer the two types for that reason.

Please pardon me for the high-frequency spamming. I'd love to have this topic well defined.

I'm unsure how to read your suggestion exactly.

VIRTIO_NET_HDR_GSO_UDP_TUNNEL needs to be "combined" (set together) with VIRTIO_NET_HDR_GSO_TCPV4, VIRTIO_NET_HDR_GSO_TCPV6 or VIRTIO_NET_HDR_GSO_UDP_L4, so it's currently defined as a bit flag:

#define VIRTIO_NET_HDR_GSO_UDP_TUNNEL 0x40

From an implementation perspective this allow us reusing the existing
code for to fill inner headers related field, masking such bit in the gso_type before invoking the existing virtio_hdr parsing helper - it will be used unmodified.

Quoted 5 lines

Due to all the above, I thought the IPv4 and IPv6 variant would be defined as:

#define VIRTIO_NET_HDR_GSO_UDP_TUNNEL_IP4 0x20
#define VIRTIO_NET_HDR_GSO_UDP_TUNNEL_IP6 0x40

Yes, I think this makes sense.

Quoted 8 lines

How would you define VIRTIO_NET_HDR_GSO_UDP_TUNNEL_IP4 and VIRTIO_NET_HDR_GSO_UDP_TUNNEL_IP6?

Thanks,

Paolo

Thanks

#19 lore raw

Re: [PATCH v6 1/2] virtio-net: define UDP tunnel segmentation offload feature

In reply to Willem de Bruijn
Quoted 21 lines · Willem de Bruijn
Quoted 19 lines · Jason Wang
Quoted 16 lines · Paolo Abeni
Quoted 5 lines · Willem de Bruijn

You have a working software implementation, right? Will this be sent (as RFC) to the public netdev or so mailing list before this becomes a standard? I assume that there is a standard process for virtio that you are following. I'd just be more confident in a thumbs up if we also review the code that implements the text.

My plan is to write the S/W virtio_net implementation, too, but it's currently not ready. I had a working PoC for the first draft of this changeset, long time ago. Things are changed enough that I'll need a rewrite. I can try to shared such a thing on netdev ASAP (where "soon" here is a very relative terms ;)

Also I don't know the standardization process. I hope for some guidance from Jason and/or Michael.

Quoted 6 lines
Quoted 3 lines

It should be something like this. When there's no future comment, open an issue https://github.com/oasis-tcs/virtio-spec/issues and ask for a vote. When it passes the voting, it would be merged.

So changes to the spec can be merged, because we know that they can be implemented reasonably?

This seems to be how it works now.

Quoted 10 lines

IETF always requires (two, because interoperable protocol definitions) working implementations.

It sounds a bit risky to me to skip this, as you come across the interesting non-obvious details during implementation.

Since implementation is planned anyway, doing at least a rough proof of concept first does not introduce extra work, but reduces risk that the spec contains errors.

Exactly and we have already had qemu which fits for that purpose. A lot of hardware was prototyped in that way (for example, rocker fbnic etc).

Quoted 7 lines

Unless it is easy to update the spec before it is numbered and finalized. Then issues can be fixed up.

Anyway, you have gone through this many times, so whatever the process is, I trust that it works. Just sharing my thought as a somewhat uniformed reader.

I fully agree with you. In the long debate of the live migration series, I point out we need a prototype for features that requires huge changes. But I don't get sufficient feedback for that.

Quoted 28 lines
Quoted 21 lines
Quoted 4 lines
Quoted 4 lines
Quoted 16 lines
Quoted 7 lines
Quoted 7 lines
device-types/net/description.tex +3 -0
@@ -423,6 +441,9 @@ \subsection{Device Operation}\label{sec:Device Types / Network Device / Device O
          le32 hash_value;        (Only if VIRTIO_NET_F_HASH_REPORT negotiated)
          le16 hash_report;       (Only if VIRTIO_NET_F_HASH_REPORT negotiated)
          le16 padding_reserved;  (Only if VIRTIO_NET_F_HASH_REPORT negotiated)
+        le16 outer_th_offset    (Only if VIRTIO_NET_F_HOST_UDP_TUNNEL_GSO negotiated)
+        le16 inner_protocol_type;(Only if VIRTIO_NET_F_HOST_UDP_TUNNEL_GSO negotiated)
+        le16 inner_nh_offset;   (Only if VIRTIO_NET_F_HOST_UDP_TUNNEL_GSO negotiated)
Quoted 8 lines

Review \subsection{Device Operation}\label{sec:Device Types / Network Device / Device O · line 447

Quoted 1 line
(quoted diff) +0 -0
  };

In relation to the above: I recall that expanding the virtio_net_hdr was non-trivial when I last tried to add fields ("[rfc,2/3]
virtio-net: support receive timestamp"). If that is the experience again, it might be worthwhile to expand with enough space to also accommodate a few subsequent extensions. Headroom is not that expensive.

Quoted 1 line

Yes, but it looks like an independent issue for this.

If this change requires new headroom, then it depends on headroom expansion.

My suggestion is to then expand it significantly immediately, to minimize the number of times we have to change the virtio_net code to support new lengths.

Quoted 8 lines

We already have structs virtio_net_hdr, virtio_net_hdr_mrg_rxbuf, virtio_net_hdr_v1, virtio_net_hdr_v1_hash.

Adding a new struct with every new field added just adds lots of churn in the code.

Just a thought, feel free to skip if the maintainers disagree.

I think I agree with that, but I meant it's better to not make UDP tunnel series depending on that.

Thanks

#20 lore raw

Re: [PATCH v6 1/2] virtio-net: define UDP tunnel segmentation offload feature

In reply to Paolo Abeni
Quoted 86 lines · Paolo Abeni

The VIRTIO_NET_HDR_GSO_UDP_TUNNEL is a gso_type flag allowing GSO over UDP tunnel. It can be negotiated on both the host and guest sides.

One constraint addressed here is that the virtio side (either device or completely the inner and outer headers offset - to allow for later GSO.

To accommodate such need, new fields are introduced in the virtio_net
header: outer_th_offset, inner_protocol_type and inner_nh_offset. They map directly to the corresponding header information. The inner transport header is implied by the (inner) checksum offload.

Those fields are required because a virtio device H/W implementation performing segmentation for UDP tunneled packet will need to touch the outer transport protocol (for the UDP length filed), the inner network protocol (for the total length field, in the IPv4 case).

Note that segmentation will additionally need to touch
the outer network protocol and the inner transport protocol. The first is implied/easily found with trivial parsing, the latter is identified by the existing csum_start field.

Note that there is no concept of UDP tunnel type negotiation (e.g. vxlan, geneve, vxlan-gpe, etc.), as a virtio device H/W implementation can perform segmentation for every possible UDP-tunnel given the specified new fields.
In the reverse direction, if a virtio device H/W implementation receives some traffic for an unknown or unsupported UDP tunnel, it will simply not aggregate the wire packet in GSO one.

---
v5 -> v6:
- split in 2 patches
- dropped the unneeded outer_mh_offset field
https://lore.kernel.org/virtio-comment/56ccba136272593d0d2af0303600c06f26bd84a1.1718621522.git.pabeni@redhat.com/

v4 -> v5:
- add separate negotiation for UDP_TUNNEL_CSUM
- much more verbose specification of outer csum handling
- UDP_TUNNEL_CSUM requires GSO_UDP_TUNNEL
https://lore.kernel.org/virtio-comment/cb89d3d6b6a7e4a7f13e60dd3fded5e950727890.1716448766.git.pabeni@redhat.com/

v3 -> v4:
- new virtio_net_hdr fields depends on F_HOST_UDP_TUNNEL_GSO or F_GUEST_UDP_TUNNEL_GSO (Stefano)
- Extended the changelog to answer Jason's questions.
- Clarified outer csum handling (Jason)
https://lore.kernel.org/virtio-dev/f0bf2db203f8061a3ecb50b7a48cf26d8f3c7f68.1716193086.git.pabeni@redhat.com

v2 -> v3:
- UDP_TUNNEL -> UDP_TUNNEL_GSO
- add explicit fields for the inner meta-data
- more verbose changelog
https://lists.oasis-open.org/archives/virtio-dev/202206/msg00026.html

v1 -> v2:
- explicitly state that the outer header probing is mandatory
- explicitly state that GSO_UDP is not allowed with GSO_UDP_TUNNEL
- clarify hdr_len usage
- clarify UDP_TUNNEL_CSUM bit usage
- fix a few typos
https://lists.oasis-open.org/archives/virtio-dev/202205/msg00037.html

---
 device-types/net/description.tex | 118 +++++++++++++++++++++++++++++--
 1 file changed, 112 insertions(+), 6 deletions(-)
device-types/net/description.tex +6 -0
index 76585b0..eda294d 100644
@@ -88,6 +88,12 @@ \subsection{Feature bits}\label{sec:Device Types / Network Device / Feature bits
 \item[VIRTIO_NET_F_CTRL_MAC_ADDR(23)] Set MAC address through control
     channel.
 
+\item[VIRTIO_NET_F_GUEST_UDP_TUNNEL_GSO (47)] Driver can receive GSO packets
+  carried by a UDP tunnel.
+
+\item[VIRTIO_NET_F_HOST_UDP_TUNNEL_GSO (49)] Device can receive GSO packets
+  carried by a UDP tunnel.
+
 \item[VIRTIO_NET_F_HASH_TUNNEL(51)] Device supports inner header hash for encapsulated packets.
 
 \item[VIRTIO_NET_F_VQ_NOTF_COAL(52)] Device supports virtqueue notification coalescing.

device-types/net/description.tex \subsubsection{Feature bit requirements}\label{sec:Device Types / Network Device · line 143

Quoted 5 lines
device-types/net/description.tex +2 -0
@@ -133,12 +139,16 @@ \subsubsection{Feature bit requirements}\label{sec:Device Types / Network Device
 \item[VIRTIO_NET_F_GUEST_UFO] Requires VIRTIO_NET_F_GUEST_CSUM.
 \item[VIRTIO_NET_F_GUEST_USO4] Requires VIRTIO_NET_F_GUEST_CSUM.
 \item[VIRTIO_NET_F_GUEST_USO6] Requires VIRTIO_NET_F_GUEST_CSUM.
+\item[VIRTIO_NET_F_GUEST_UDP_TUNNEL_GSO] Requires both VIRTIO_NET_F_GUEST_TSO4 and VIRTIO_NET_F_GUEST_TSO6, or
+   both VIRTIO_NET_F_GUEST_USO4 and VIRTIO_NET_F_GUEST_USO6.

I think VIRTIO_NET_F_GUEST_UDP_TUNNEL_GSO requires all of the four?

Quoted 12 lines
device-types/net/description.tex +2 -0
 \item[VIRTIO_NET_F_HOST_TSO4] Requires VIRTIO_NET_F_CSUM.
 \item[VIRTIO_NET_F_HOST_TSO6] Requires VIRTIO_NET_F_CSUM.
 \item[VIRTIO_NET_F_HOST_ECN] Requires VIRTIO_NET_F_HOST_TSO4 or VIRTIO_NET_F_HOST_TSO6.
 \item[VIRTIO_NET_F_HOST_UFO] Requires VIRTIO_NET_F_CSUM.
 \item[VIRTIO_NET_F_HOST_USO] Requires VIRTIO_NET_F_CSUM.
+\item[VIRTIO_NET_F_HOST_UDP_TUNNEL_GSO] Requires both VIRTIO_NET_F_HOST_TSO4 and VIRTIO_NET_F_HOST_TSO6, or
+   both VIRTIO_NET_F_GUEST_USO4 or VIRTIO_NET_F_GUEST_USO6.
 
 \item[VIRTIO_NET_F_CTRL_RX] Requires VIRTIO_NET_F_CTRL_VQ.
 \item[VIRTIO_NET_F_CTRL_VLAN] Requires VIRTIO_NET_F_CTRL_VQ.

device-types/net/description.tex \subsection{Device Initialization}\label{sec:Device Types / Network Device / Dev · line 388

Quoted 4 lines
device-types/net/description.tex +1 -0
@@ -375,6 +385,12 @@ \subsection{Device Initialization}\label{sec:Device Types / Network Device / Dev
   TCP), VIRTIO_NET_F_HOST_TSO6 (IPv6 TCP), VIRTIO_NET_F_HOST_UFO
   (UDP fragmentation) and VIRTIO_NET_F_HOST_USO (UDP segmentation) features.
 
+\item If any TCP or UDP segmentation feature is negotiated,

Not a native speaker but this seems to conflict with the above. For example, does "only VIRTIO_NET_F_GUEST_TSO4 is negotiated" mean any TCP segmentation feature?

Quoted 54 lines

a driver can

device-types/net/description.tex +16 -2
+  use TCP segmentation or UDP segmentation on top of UDP encapsulation
+  offload, when the outer header does not require checksumming - e.g.
+  the outer UDP checksum is zero - by negotiating the
+  VIRTIO_NET_F_HOST_UDP_TUNNEL_GSO feature.
+
 \item The converse features are also available: a driver can save
   the virtual device some work by negotiating these features.\note{For example, a network packet transported between two guests on
 the same system might not need checksumming at all, nor segmentation,
@@ -382,8 +398,9 @@ \subsection{Device Initialization}\label{sec:Device Types / Network Device / Dev
    The VIRTIO_NET_F_GUEST_CSUM feature indicates that partially
   checksummed packets can be received, and if it can do that then
   the VIRTIO_NET_F_GUEST_TSO4, VIRTIO_NET_F_GUEST_TSO6,
-  VIRTIO_NET_F_GUEST_UFO, VIRTIO_NET_F_GUEST_ECN, VIRTIO_NET_F_GUEST_USO4
-  and VIRTIO_NET_F_GUEST_USO6 are the input equivalents of the features described above.
+  VIRTIO_NET_F_GUEST_UFO, VIRTIO_NET_F_GUEST_ECN, VIRTIO_NET_F_GUEST_USO4,
+  VIRTIO_NET_F_GUEST_USO6 and VIRTIO_NET_F_GUEST_UDP_TUNNEL_GSO
+  are the input equivalents of the features described above.
   See \ref{sec:Device Types / Network Device / Device Operation /
 Setting Up Receive Buffers}~\nameref{sec:Device Types / Network
 Device / Device Operation / Setting Up Receive Buffers} and
@@ -413,6 +430,7 @@ \subsection{Device Operation}\label{sec:Device Types / Network Device / Device O
 #define VIRTIO_NET_HDR_GSO_UDP         3
 #define VIRTIO_NET_HDR_GSO_TCPV6       4
 #define VIRTIO_NET_HDR_GSO_UDP_L4      5
+#define VIRTIO_NET_HDR_GSO_UDP_TUNNEL 0x40
 #define VIRTIO_NET_HDR_GSO_ECN      0x80
         u8 gso_type;
         le16 hdr_len;
@@ -423,6 +441,9 @@ \subsection{Device Operation}\label{sec:Device Types / Network Device / Device O
         le32 hash_value;        (Only if VIRTIO_NET_F_HASH_REPORT negotiated)
         le16 hash_report;       (Only if VIRTIO_NET_F_HASH_REPORT negotiated)
         le16 padding_reserved;  (Only if VIRTIO_NET_F_HASH_REPORT negotiated)
+        le16 outer_th_offset    (Only if VIRTIO_NET_F_HOST_UDP_TUNNEL_GSO negotiated)
+        le16 inner_protocol_type;(Only if VIRTIO_NET_F_HOST_UDP_TUNNEL_GSO negotiated)
+        le16 inner_nh_offset;   (Only if VIRTIO_NET_F_HOST_UDP_TUNNEL_GSO negotiated)
 };
 \end{lstlisting}
 
@@ -480,6 +501,8 @@ \subsubsection{Packet Transmission}\label{sec:Device Types / Network Device / De
 followed by the TCP header (with the TCP checksum field 16 bytes
 into that header). \field{csum_start} will be 14+20 = 34 (the TCP
 checksum includes the header), and \field{csum_offset} will be 16.
+If the given packet has the VIRTIO_NET_HDR_GSO_UDP_TUNNEL bit set,
+the above checksum fields refer to the inner header checksum.
 \end{note}
 
 \item If the driver negotiated
@@ -516,6 +539,26 @@ \subsubsection{Packet Transmission}\label{sec:Device Types / Network Device / De
 specifically in the protocol.}.
    \end{itemize}
 
+\item If the driver negotiated the VIRTIO_NET_F_HOST_UDP_TUNNEL_GSO feature,
+  the VIRTIO_NET_HDR_GSO_UDP_TUNNEL bit in \field{gso_type} indicates that

device-types/net/description.tex \subsubsection{Packet Transmission}\label{sec:Device Types / Network Device / De · line 553

Quoted 10 lines
device-types/net/description.tex +10 -0
+  the GSO protocol is encapsulated in a UDP tunnel.
+  The outer UDP header must not require checksumming, e.g. the outer UDP checksum
+  must be zero.
+  The other tunnel-related fields indicate how to replicate the packet
+  headers to cut it into smaller packets:
+
+  \begin{itemize}
+  \item \field{outer_th_offset} field indicates the outer transport header within
+      the packet. Note that such field differs from \field{csum_start} as the latter
+      points to the inner transport header within the packet

I wonder if it's better to stick to outer_csum_start, and say it's offset within the packet to begin checksumming for outer packet. This follows the existing name and could be reused for encapsulation other than UDP tunnel.

Quoted 12 lines
device-types/net/description.tex +8 -0
+
+  \item \field{inner_protocol_type} field indicates the ethernet type of the inner
+      protocol.
+
+  \item \field{inner_nh_offset} field indicates the inner network header within
+      the packet.
+  \end{itemize}
+
 \item \field{num_buffers} is set to zero.  This field is unused on transmitted packets.
 
 \item The header and packet are added as one output descriptor to the

device-types/net/description.tex \subsubsection{Packet Transmission}\label{sec:Device Types / Network Device / De · line 606

Quoted 7 lines
device-types/net/description.tex +4 -0
@@ -557,6 +600,18 @@ \subsubsection{Packet Transmission}\label{sec:Device Types / Network Device / De
 driver MUST set the VIRTIO_NET_HDR_GSO_ECN bit in
 \field{gso_type}.
 
+The driver MUST NOT send to the device TCP or UDP GSO packets over UDP tunnel
+requiring segmentation offload, unless the VIRTIO_NET_F_HOST_UDP_TUNNEL_GSO is
+negotiated, in which case the driver MUST set the VIRTIO_NET_HDR_GSO_UDP_TUNNEL
+bit in \field{gso_type}.

Is this better to follow the existing style of the driver normative? For example, we had

"If VIRTIO_NET_F_HOST_TSO6 is negotiated, the driver MAY set gso_type to VIRTIO_NET_HDR_GSO_TCPV6 to request TCPv6 segmentation, otherwise the driver MUST NOT set gso_type to VIRTIO_NET_HDR_GSO_TCPV6."

We could do the same

"If VIRTIO_NET_F_HOST_UDP_TUNNEL_GSO is negotiated, the driver MAY ... otherwise the driver MUST NOT ..."

Quoted 12 lines
device-types/net/description.tex +8 -0
+
+The driver MUST NOT send to the device TCP or UDP GSO packets over UDP tunnel
+requiring segmentation and outer UDP checksum offload.
+
+The driver MUST NOT set the VIRTIO_NET_HDR_GSO_UDP_TUNNEL together with
+VIRTIO_NET_HDR_GSO_UDP, as the latter is deprecated in favor of UDP_L4
+and no new features will support it.
+
 If the VIRTIO_NET_F_CSUM feature has been negotiated, the
 driver MAY set the VIRTIO_NET_HDR_F_NEEDS_CSUM bit in
 \field{flags}, if so:

device-types/net/description.tex \subsubsection{Packet Transmission}\label{sec:Device Types / Network Device / De · line 662

Quoted 10 lines
device-types/net/description.tex +7 -0
@@ -598,6 +653,28 @@ \subsubsection{Packet Transmission}\label{sec:Device Types / Network Device / De
        header.
 \end{itemize}
 
+If the VIRTIO_NET_F_HOST_UDP_TUNNEL_GSO option has been negotiated, the
+driver MAY set the VIRTIO_NET_HDR_GSO_UDP_TUNNEL bit in \field{gso_type}, if so:
+\begin{itemize}
+\item the driver MUST set \field{outer_th_offset} to the outer UDP header
+  offset, \field{inner_protocol_type} to the ethernet type of the inner
+  protocol, and \field{inner_nh_offset} to the inner network header offset.
+\end{itemize}

Do we need to say then csum_start and csum_offset is for the inner packet here?

Quoted 68 lines
device-types/net/description.tex +32 -4
+
+If the bit VIRTIO_NET_HDR_GSO_UDP_TUNNEL in \field{gso_type} is set, the outer
+UDP header MUST NOT require checksumming. That is the, the outer UDP checksum
+value must be 0, or the valid complete checksum for such header.
+
+\begin{note}
+The valid complete checksum of the outer UDP header of individual segments
+can be computed by the driver prior to segmentation only if the GSO packet
+size is a multiple of \field{gso_size}.
+\end{note}
+
+If the VIRTIO_NET_F_HOST_UDP_TUNNEL_GSO option has not
+been negotiated, the driver MUST NOT set the VIRTIO_NET_HDR_F_UDP_TUNNEL bit
+in \field{gso_type}.
+
 The driver SHOULD accept the VIRTIO_NET_F_GUEST_HDRLEN feature if it has
 been offered, and if it's able to provide the exact header length.
 
@@ -633,6 +710,10 @@ \subsubsection{Packet Transmission}\label{sec:Device Types / Network Device / De
        \end{note}
 \end{itemize}
 
+If VIRTIO_NET_HDR_GSO_UDP_TUNNEL bit in \field{gso_type} is not set, the
+device MUST NOT use the \field{outer_th_offset}, \field{inner_protocol_type}
+ and \field{inner_nh_offset}.
+
 If VIRTIO_NET_HDR_F_NEEDS_CSUM is not set, the device MUST NOT
 rely on the packet checksum being correct.
 \paragraph{Packet Transmission Interrupt}\label{sec:Device Types / Network Device / Device Operation / Packet Transmission / Packet Transmission Interrupt}
@@ -727,8 +808,8 @@ \subsubsection{Processing of Incoming Packets}\label{sec:Device Types / Network
   has been validated.
 \end{enumerate}
 
-Additionally, VIRTIO_NET_F_GUEST_CSUM, TSO4, TSO6, UDP and ECN
-features enable receive checksum, large receive offload and ECN
+Additionally, VIRTIO_NET_F_GUEST_CSUM, TSO4, TSO6, UDP, UDP_TUNNEL
+and ECN features enable receive checksum, large receive offload and ECN
 support which are the input equivalents of the transmit checksum,
 transmit segmentation offloading and ECN features, as described
 in \ref{sec:Device Types / Network Device / Device Operation /
@@ -750,8 +831,14 @@ \subsubsection{Processing of Incoming Packets}\label{sec:Device Types / Network
   from \field{csum_start} and any preceding checksums
   have been validated.  The checksum on the packet is incomplete and
   if bit VIRTIO_NET_HDR_F_RSC_INFO is not set in \field{flags},
-  then \field{csum_start} and \field{csum_offset} indicate how to calculate it
-  (see Packet Transmission point 1).
+  then \field{csum_start} and \field{csum_offset} indicate how to calculate it.
+  If the VIRTIO_NET_HDR_GSO_UDP_TUNNEL bit is set, the \field{csum_start} field
+  refers to the inner transport header offset (see Packet Transmission point 1).
+\item If the VIRTIO_NET_F_GUEST_UDP_TUNNEL_GSO option was negotiated and
+  \field{gso_type} is not VIRTIO_NET_HDR_GSO_NONE, the VIRTIO_NET_HDR_GSO_UDP_TUNNEL
+  bit MAY be set. In such case the \field{outer_th_offset}, \field{inner_protocol_type},
+  and \field{inner_nh_offset} fields indicate the corresponding
+  headers information.
 
 \end{enumerate}
 
@@ -796,6 +883,9 @@ \subsubsection{Processing of Incoming Packets}\label{sec:Device Types / Network
 If none of VIRTIO_NET_F_GUEST_USO4 or VIRTIO_NET_F_GUEST_USO6 have been negotiated,
 the device MUST NOT set \field{gso_type} to VIRTIO_NET_HDR_GSO_UDP_L4.
 
+If VIRTIO_NET_F_GUEST_UDP_TUNNEL_GSO is not negotiated the device MUST NOT set
+the VIRTIO_NET_HDR_GSO_UDP_TUNNEL bit in \field{gso_type}.
+
 The device SHOULD NOT send to the driver TCP packets requiring segmentation offload
 which have the Explicit Congestion Notification bit set, unless the
 VIRTIO_NET_F_GUEST_ECN feature is negotiated, in which case the

device-types/net/description.tex \subsubsection{Processing of Incoming Packets}\label{sec:Device Types / Network · line 918

Quoted 10 lines
device-types/net/description.tex +7 -0
@@ -819,6 +909,17 @@ \subsubsection{Processing of Incoming Packets}\label{sec:Device Types / Network
        fully checksummed packet;
 \end{enumerate}
 
+The device MUST NOT send to the driver TCP or UDP GSO packets encapsulated in UDP
+tunnel and requiring segmentation offload, unless the
+VIRTIO_NET_F_GUEST_UDP_TUNNEL_GSO is negotiated, in which case the device MUST set
+the VIRTIO_NET_HDR_GSO_UDP_TUNNEL bit in \field{gso_type} and MUST set the
+\field{outer_th_offset}, \field{inner_protocol_type} and
+\field{inner_nh_offset} fields to the corresponding header information, and the outer
+UDP header MUST NOT require checksum offload.

Similarly, do we need to say csum_start|offset is for inner?

Quoted 27 lines
device-types/net/description.tex +9 -1
+
+The device MUST NOT send the driver TCP or UDP GSO packets encapsulated in UDP
+tunnel and requiring segmentation and outer checksum offload.
+
 If none of the VIRTIO_NET_F_GUEST_TSO4, TSO6, UFO, USO4 or USO6 options have
 been negotiated, the device MUST set \field{gso_type} to
 VIRTIO_NET_HDR_GSO_NONE.
@@ -867,6 +968,10 @@ \subsubsection{Processing of Incoming Packets}\label{sec:Device Types / Network
 VIRTIO_NET_HDR_F_DATA_VALID is set, the driver MUST NOT
 rely on the packet checksum being correct.
 
+If the VIRTIO_NET_HDR_GSO_UDP_TUNNEL_GSO bit in \field{gso_type} is not set,
+the driver MUST NOT use the \field{outer_th_offset}, \field{inner_protocol_type},
+\field{inner_mac_offset} and \field{inner_nh_offset}.
+
 \paragraph{Hash calculation for incoming packets}
 \label{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets}
 
@@ -1624,6 +1729,7 @@ \subsubsection{Control Virtqueue}\label{sec:Device Types / Network Device / Devi
 #define VIRTIO_NET_F_GUEST_TSO6       8
 #define VIRTIO_NET_F_GUEST_ECN        9
 #define VIRTIO_NET_F_GUEST_UFO        10
+#define VIRTIO_NET_F_GUEST_UDP_TUNNEL_GSO  47
 #define VIRTIO_NET_F_GUEST_USO4       54
 #define VIRTIO_NET_F_GUEST_USO6       55
 
--
Quoted 2 lines

2.45.2

Thanks

#21 lore raw

Re: [PATCH v6 1/2] virtio-net: define UDP tunnel segmentation offload feature

Quoted 8 lines · Jason Wang

device-types/net/description.tex \subsubsection{Feature bit requirements}\label{sec:Device Types / Network Device · line 143

Quoted 6 lines
device-types/net/description.tex +2 -0
@@ -133,12 +139,16 @@ \subsubsection{Feature bit requirements}\label{sec:Device Types / Network Device
  \item[VIRTIO_NET_F_GUEST_UFO] Requires VIRTIO_NET_F_GUEST_CSUM.
  \item[VIRTIO_NET_F_GUEST_USO4] Requires VIRTIO_NET_F_GUEST_CSUM.
  \item[VIRTIO_NET_F_GUEST_USO6] Requires VIRTIO_NET_F_GUEST_CSUM.
+\item[VIRTIO_NET_F_GUEST_UDP_TUNNEL_GSO] Requires both VIRTIO_NET_F_GUEST_TSO4 and VIRTIO_NET_F_GUEST_TSO6, or
+   both VIRTIO_NET_F_GUEST_USO4 and VIRTIO_NET_F_GUEST_USO6.

I think VIRTIO_NET_F_GUEST_UDP_TUNNEL_GSO requires all of the four?

Functionally speaking is not strictly necessary: the UDP tunnel feature needs just a single inner GSO type to assemble and process an UDP tunneled packet.

On the flip side I think is reasonable that modern implementations support all the mentioned inner GSO type and requiring all of them would possibly make the implementation simpler and/or less bug prone.

So I guess I can change the requirement as you say above.

Quoted 14 lines
Quoted 14 lines
device-types/net/description.tex +2 -0
  \item[VIRTIO_NET_F_HOST_TSO4] Requires VIRTIO_NET_F_CSUM.
  \item[VIRTIO_NET_F_HOST_TSO6] Requires VIRTIO_NET_F_CSUM.
  \item[VIRTIO_NET_F_HOST_ECN] Requires VIRTIO_NET_F_HOST_TSO4 or VIRTIO_NET_F_HOST_TSO6.
  \item[VIRTIO_NET_F_HOST_UFO] Requires VIRTIO_NET_F_CSUM.
  \item[VIRTIO_NET_F_HOST_USO] Requires VIRTIO_NET_F_CSUM.
+\item[VIRTIO_NET_F_HOST_UDP_TUNNEL_GSO] Requires both VIRTIO_NET_F_HOST_TSO4 and VIRTIO_NET_F_HOST_TSO6, or
+   both VIRTIO_NET_F_GUEST_USO4 or VIRTIO_NET_F_GUEST_USO6.
 
  \item[VIRTIO_NET_F_CTRL_RX] Requires VIRTIO_NET_F_CTRL_VQ.
  \item[VIRTIO_NET_F_CTRL_VLAN] Requires VIRTIO_NET_F_CTRL_VQ.
@@ -375,6 +385,12 @@ \subsection{Device Initialization}\label{sec:Device Types / Network Device / Dev
    TCP), VIRTIO_NET_F_HOST_TSO6 (IPv6 TCP), VIRTIO_NET_F_HOST_UFO
    (UDP fragmentation) and VIRTIO_NET_F_HOST_USO (UDP segmentation) features.
 
Quoted 5 lines

Review \subsection{Device Initialization}\label{sec:Device Types / Network Device / Dev · line 388

Quoted 1 line
(quoted diff) +1 -0
+\item If any TCP or UDP segmentation feature is negotiated,

Not a native speaker but this seems to conflict with the above. For example, does "only VIRTIO_NET_F_GUEST_TSO4 is negotiated" mean any TCP segmentation feature?

As per the previois comment, I'll change this in the next revision to:

"""
If the VIRTIO_NET_F_HOST_TSO6, VIRTIO_NET_F_HOST_TSO4 and VIRTIO_NET_HDR_GSO_UDP_L4 features are negotiated """>> @@ -516,6 +539,26 @@ \subsubsection{Packet Transmission}\label{sec:Device Types / Network Device / De

Quoted 16 lines
Quoted 5 lines
device-types/net/description.tex +2 -0
  specifically in the protocol.}.
     \end{itemize}
 
+\item If the driver negotiated the VIRTIO_NET_F_HOST_UDP_TUNNEL_GSO feature,
+  the VIRTIO_NET_HDR_GSO_UDP_TUNNEL bit in \field{gso_type} indicates that

device-types/net/description.tex \subsubsection{Packet Transmission}\label{sec:Device Types / Network Device / De

Quoted 10 lines
device-types/net/description.tex +10 -0
+  the GSO protocol is encapsulated in a UDP tunnel.
+  The outer UDP header must not require checksumming, e.g. the outer UDP checksum
+  must be zero.
+  The other tunnel-related fields indicate how to replicate the packet
+  headers to cut it into smaller packets:
+
+  \begin{itemize}
+  \item \field{outer_th_offset} field indicates the outer transport header within
+      the packet. Note that such field differs from \field{csum_start} as the latter
+      points to the inner transport header within the packet
Quoted 4 lines

I wonder if it's better to stick to outer_csum_start, and say it's offset within the packet to begin checksumming for outer packet. This follows the existing name and could be reused for encapsulation other than UDP tunnel.

I thought we agreed on this name in the previous iteration?!? It feels strange to introduce a csum related field in a change NOT introducing a checksum related feature. And we need this filed here, for plain segmentation.

Quoted 12 lines

device-types/net/description.tex \subsubsection{Packet Transmission}\label{sec:Device Types / Network Device / De · line 606

Quoted 8 lines
device-types/net/description.tex +4 -0
@@ -557,6 +600,18 @@ \subsubsection{Packet Transmission}\label{sec:Device Types / Network Device / De
  driver MUST set the VIRTIO_NET_HDR_GSO_ECN bit in
  \field{gso_type}.
 
+The driver MUST NOT send to the device TCP or UDP GSO packets over UDP tunnel
+requiring segmentation offload, unless the VIRTIO_NET_F_HOST_UDP_TUNNEL_GSO is
+negotiated, in which case the driver MUST set the VIRTIO_NET_HDR_GSO_UDP_TUNNEL
+bit in \field{gso_type}.

Is this better to follow the existing style of the driver normative? For example, we had

Quoted 8 lines

"If VIRTIO_NET_F_HOST_TSO6 is negotiated, the driver MAY set gso_type to VIRTIO_NET_HDR_GSO_TCPV6 to request TCPv6 segmentation, otherwise the driver MUST NOT set gso_type to VIRTIO_NET_HDR_GSO_TCPV6."

We could do the same

"If VIRTIO_NET_F_HOST_UDP_TUNNEL_GSO is negotiated, the driver MAY ... otherwise the driver MUST NOT ..."

ok, I'll do that in the next revision

Quoted 4 lines
Quoted 4 lines
device-types/net/description.tex +0 -0
@@ -598,6 +653,28 @@ \subsubsection{Packet Transmission}\label{sec:Device Types / Network Device / De
         header.
  \end{itemize}
 
Quoted 9 lines

device-types/net/description.tex \subsubsection{Packet Transmission}\label{sec:Device Types / Network Device / De · line 662

Quoted 7 lines
device-types/net/description.tex +7 -0
+If the VIRTIO_NET_F_HOST_UDP_TUNNEL_GSO option has been negotiated, the
+driver MAY set the VIRTIO_NET_HDR_GSO_UDP_TUNNEL bit in \field{gso_type}, if so:
+\begin{itemize}
+\item the driver MUST set \field{outer_th_offset} to the outer UDP header
+  offset, \field{inner_protocol_type} to the ethernet type of the inner
+  protocol, and \field{inner_nh_offset} to the inner network header offset.
+\end{itemize}

Do we need to say then csum_start and csum_offset is for the inner packet here?

I thought such statement was already everywhere, but no objection to add it here, too.

Somewhat related, I have a doubt WRT the csum_start/csum_offset fields "presence" with plain GSO packets. AFAICS, the specs require the csum offload feature as a pre-req for the plain segmentation feature, but also allow the reception of a GSO packet without the VIRTIO_NET_HDR_F_NEEDS_CSUM bit set, see the recent linux patch from Willem:

https://lore.kernel.org/netdev/[email protected]/

I think it will make the thing simpler if we always require such bit to be set for GSO packets. I guess we can't change the current status for the existing GSO types, but for UDP tunnel we could add that requirement from the start, both for the driver -> device direction and for the device -> driver direction.

Even when the device sets the VIRTIO_NET_HDR_F_DATA_VALID, so can always expect a valid csum_start/csum_offset pair, and make things hopefully more robust. When VIRTIO_NET_HDR_F_DATA_VALID is set, the pair will NOT be used to validated the csum, just to point to the inner transport header and avoid parsing. WDYT?

Thanks,

Paolo

#22 lore raw

Re: [PATCH v6 1/2] virtio-net: define UDP tunnel segmentation offload feature

Quoted 21 lines · Paolo Abeni
Quoted 14 lines · Jason Wang
Quoted 14 lines

\item[VIRTIO_NET_F_HOST_TSO4] Requires VIRTIO_NET_F_CSUM.
\item[VIRTIO_NET_F_HOST_TSO6] Requires VIRTIO_NET_F_CSUM.
\item[VIRTIO_NET_F_HOST_ECN] Requires VIRTIO_NET_F_HOST_TSO4 or VIRTIO_NET_F_HOST_TSO6. \item[VIRTIO_NET_F_HOST_UFO] Requires VIRTIO_NET_F_CSUM.
\item[VIRTIO_NET_F_HOST_USO] Requires VIRTIO_NET_F_CSUM.
+\item[VIRTIO_NET_F_HOST_UDP_TUNNEL_GSO] Requires both VIRTIO_NET_F_HOST_TSO4 and VIRTIO_NET_F_HOST_TSO6, or

device-types/net/description.tex +1 -0
+   both VIRTIO_NET_F_GUEST_USO4 or VIRTIO_NET_F_GUEST_USO6.
 
   \item[VIRTIO_NET_F_CTRL_RX] Requires VIRTIO_NET_F_CTRL_VQ.
   \item[VIRTIO_NET_F_CTRL_VLAN] Requires VIRTIO_NET_F_CTRL_VQ.
@@ -375,6 +385,12 @@ \subsection{Device Initialization}\label{sec:Device Types / Network Device / Dev
     TCP), VIRTIO_NET_F_HOST_TSO6 (IPv6 TCP), VIRTIO_NET_F_HOST_UFO
     (UDP fragmentation) and VIRTIO_NET_F_HOST_USO (UDP segmentation) features.
 
Quoted 5 lines

Review \subsection{Device Initialization}\label{sec:Device Types / Network Device / Dev · line 388

Quoted 1 line
(quoted diff) +1 -0
+\item If any TCP or UDP segmentation feature is negotiated,

Not a native speaker but this seems to conflict with the above. For example, does "only VIRTIO_NET_F_GUEST_TSO4 is negotiated" mean any TCP segmentation feature?

Quoted 6 lines

As per the previois comment, I'll change this in the next revision to:

"""
If the VIRTIO_NET_F_HOST_TSO6, VIRTIO_NET_F_HOST_TSO4 and VIRTIO_NET_HDR_GSO_UDP_L4 features are negotiated """

Typos above, should be:

"""
If the VIRTIO_NET_F_HOST_TSO6, VIRTIO_NET_F_HOST_TSO4, VIRTIO_NET_F_HOST_USO features are negotiated """

Cheers,

Paolo

#23 lore raw

Re: [PATCH v6 1/2] virtio-net: define UDP tunnel segmentation offload feature

In reply to Paolo Abeni
Quoted 11 lines · Paolo Abeni
Quoted 8 lines · Jason Wang

device-types/net/description.tex \subsubsection{Feature bit requirements}\label{sec:Device Types / Network Device · line 143

Quoted 6 lines
device-types/net/description.tex +2 -0
@@ -133,12 +139,16 @@ \subsubsection{Feature bit requirements}\label{sec:Device Types / Network Device
  \item[VIRTIO_NET_F_GUEST_UFO] Requires VIRTIO_NET_F_GUEST_CSUM.
  \item[VIRTIO_NET_F_GUEST_USO4] Requires VIRTIO_NET_F_GUEST_CSUM.
  \item[VIRTIO_NET_F_GUEST_USO6] Requires VIRTIO_NET_F_GUEST_CSUM.
+\item[VIRTIO_NET_F_GUEST_UDP_TUNNEL_GSO] Requires both VIRTIO_NET_F_GUEST_TSO4 and VIRTIO_NET_F_GUEST_TSO6, or
+   both VIRTIO_NET_F_GUEST_USO4 and VIRTIO_NET_F_GUEST_USO6.

I think VIRTIO_NET_F_GUEST_UDP_TUNNEL_GSO requires all of the four?

Quoted 7 lines

Functionally speaking is not strictly necessary: the UDP tunnel feature needs just a single inner GSO type to assemble and process an UDP tunneled packet.

On the flip side I think is reasonable that modern implementations support all the mentioned inner GSO type and requiring all of them would possibly make the implementation simpler and/or less bug prone.

Yes.

Quoted 2 lines

So I guess I can change the requirement as you say above.

That would be great.

Quoted 21 lines
Quoted 15 lines
Quoted 15 lines
device-types/net/description.tex +2 -0
  \item[VIRTIO_NET_F_HOST_TSO4] Requires VIRTIO_NET_F_CSUM.
  \item[VIRTIO_NET_F_HOST_TSO6] Requires VIRTIO_NET_F_CSUM.
  \item[VIRTIO_NET_F_HOST_ECN] Requires VIRTIO_NET_F_HOST_TSO4 or VIRTIO_NET_F_HOST_TSO6.
  \item[VIRTIO_NET_F_HOST_UFO] Requires VIRTIO_NET_F_CSUM.
  \item[VIRTIO_NET_F_HOST_USO] Requires VIRTIO_NET_F_CSUM.
+\item[VIRTIO_NET_F_HOST_UDP_TUNNEL_GSO] Requires both VIRTIO_NET_F_HOST_TSO4 and VIRTIO_NET_F_HOST_TSO6, or
+   both VIRTIO_NET_F_GUEST_USO4 or VIRTIO_NET_F_GUEST_USO6.
 
  \item[VIRTIO_NET_F_CTRL_RX] Requires VIRTIO_NET_F_CTRL_VQ.
  \item[VIRTIO_NET_F_CTRL_VLAN] Requires VIRTIO_NET_F_CTRL_VQ.
@@ -375,6 +385,12 @@ \subsection{Device Initialization}\label{sec:Device Types / Network Device / Dev
    TCP), VIRTIO_NET_F_HOST_TSO6 (IPv6 TCP), VIRTIO_NET_F_HOST_UFO
    (UDP fragmentation) and VIRTIO_NET_F_HOST_USO (UDP segmentation) features.
 
Quoted 5 lines

Review \subsection{Device Initialization}\label{sec:Device Types / Network Device / Dev · line 388

Quoted 1 line
(quoted diff) +1 -0
+\item If any TCP or UDP segmentation feature is negotiated,

Not a native speaker but this seems to conflict with the above. For example, does "only VIRTIO_NET_F_GUEST_TSO4 is negotiated" mean any TCP segmentation feature?

Quoted 6 lines

As per the previois comment, I'll change this in the next revision to:

"""
If the VIRTIO_NET_F_HOST_TSO6, VIRTIO_NET_F_HOST_TSO4 and VIRTIO_NET_HDR_GSO_UDP_L4 features are negotiated """

Ok.

Quoted 18 lines

Review \subsubsection{Packet

Quoted 1 line
(quoted diff) +0 -0
@@ -516,6 +539,26 @@ \subsubsection{Packet

Transmission}\label{sec:Device Types / Network Device / De

Quoted 16 lines
Quoted 15 lines
device-types/net/description.tex +12 -0
  specifically in the protocol.}.
     \end{itemize}
 
+\item If the driver negotiated the VIRTIO_NET_F_HOST_UDP_TUNNEL_GSO feature,
+  the VIRTIO_NET_HDR_GSO_UDP_TUNNEL bit in \field{gso_type} indicates that
+  the GSO protocol is encapsulated in a UDP tunnel.
+  The outer UDP header must not require checksumming, e.g. the outer UDP checksum
+  must be zero.
+  The other tunnel-related fields indicate how to replicate the packet
+  headers to cut it into smaller packets:
+
+  \begin{itemize}
+  \item \field{outer_th_offset} field indicates the outer transport header within
+      the packet. Note that such field differs from \field{csum_start} as the latter
+      points to the inner transport header within the packet
Quoted 6 lines
Quoted 4 lines

I wonder if it's better to stick to outer_csum_start, and say it's offset within the packet to begin checksumming for outer packet. This follows the existing name and could be reused for encapsulation other than UDP tunnel.

I thought we agreed on this name in the previous iteration?!?

Sorry if I do that, the switch loses some context :(

Quoted 4 lines

It feels
strange to introduce a csum related field in a change NOT introducing a checksum related feature. And we need this filed here, for plain segmentation.

Ok, fine.

Quoted 38 lines
Quoted 13 lines

device-types/net/description.tex \subsubsection{Packet Transmission}\label{sec:Device Types / Network Device / De · line 606

Quoted 9 lines
device-types/net/description.tex +4 -0
@@ -557,6 +600,18 @@ \subsubsection{Packet Transmission}\label{sec:Device Types / Network Device / De
  driver MUST set the VIRTIO_NET_HDR_GSO_ECN bit in
  \field{gso_type}.
 
+The driver MUST NOT send to the device TCP or UDP GSO packets over UDP tunnel
+requiring segmentation offload, unless the VIRTIO_NET_F_HOST_UDP_TUNNEL_GSO is
+negotiated, in which case the driver MUST set the VIRTIO_NET_HDR_GSO_UDP_TUNNEL
+bit in \field{gso_type}.

Is this better to follow the existing style of the driver normative? For example, we had

Quoted 8 lines

"If VIRTIO_NET_F_HOST_TSO6 is negotiated, the driver MAY set gso_type to VIRTIO_NET_HDR_GSO_TCPV6 to request TCPv6 segmentation, otherwise the driver MUST NOT set gso_type to VIRTIO_NET_HDR_GSO_TCPV6."

We could do the same

"If VIRTIO_NET_F_HOST_UDP_TUNNEL_GSO is negotiated, the driver MAY ... otherwise the driver MUST NOT ..."

ok, I'll do that in the next revision

Quoted 4 lines
Quoted 4 lines
device-types/net/description.tex +0 -0
@@ -598,6 +653,28 @@ \subsubsection{Packet Transmission}\label{sec:Device Types / Network Device / De
         header.
  \end{itemize}
 
Quoted 9 lines

device-types/net/description.tex \subsubsection{Packet Transmission}\label{sec:Device Types / Network Device / De · line 662

Quoted 7 lines
device-types/net/description.tex +7 -0
+If the VIRTIO_NET_F_HOST_UDP_TUNNEL_GSO option has been negotiated, the
+driver MAY set the VIRTIO_NET_HDR_GSO_UDP_TUNNEL bit in \field{gso_type}, if so:
+\begin{itemize}
+\item the driver MUST set \field{outer_th_offset} to the outer UDP header
+  offset, \field{inner_protocol_type} to the ethernet type of the inner
+  protocol, and \field{inner_nh_offset} to the inner network header offset.
+\end{itemize}

Do we need to say then csum_start and csum_offset is for the inner packet here?

Quoted 2 lines

I thought such statement was already everywhere, but no objection to add it here, too.

I think we need some tweaks. E.g with this patch applied, in "Packet Transmission" subsection, we still have something which is not clear:

"""
\item \field{csum_start} is set to the offset within the packet to
begin checksumming,
and

\item \field{csum_offset} indicates how many bytes after the csum_start the
new (16 bit ones' complement) checksum is placed by the device.
"""

I wonder if it's more clear to just describe the outer and inner metadata there?

And in the end of this description

"""
\item If the driver negotiated the VIRTIO_NET_F_HOST_UDP_TUNNEL_GSO feature,
the VIRTIO_NET_HDR_GSO_UDP_TUNNEL bit in \field{gso_type} indicates that the GSO protocol is encapsulated in a UDP tunnel.
The outer UDP header must not require checksumming, e.g. the outer
UDP checksum
must be zero.
The other tunnel-related fields indicate how to replicate the packet headers to cut it into smaller packets:

\begin{itemize}
\item \field{outer_th_offset} field indicates the outer transport
header within

      the packet. Note that such field differs from \field{csum_start}

as the latter
points to the inner transport header within the packet.

\item \field{inner_protocol_type} field indicates the ethernet type
of the inner
protocol.

\item \field{inner_nh_offset} field indicates the inner network header within
the packet.
\end{itemize}
"""

And it would be even nicer if an example could be give there, as we have a non tunnel example for setting metatdata:

"""
For example, consider a partially checksummed TCP (IPv4) packet. It will have a 14 byte ethernet header and 20 byte IP header followed by the TCP header (with the TCP checksum field 16 bytes into that header). \field{csum_start} will be 14+20 = 34 (the TCP checksum includes the header), and \field{csum_offset} will be 16. """

Quoted 11 lines

Somewhat related, I have a doubt WRT the csum_start/csum_offset fields "presence" with plain GSO packets. AFAICS, the specs require the csum offload feature as a pre-req for the plain segmentation feature, but also allow the reception of a GSO packet without the VIRTIO_NET_HDR_F_NEEDS_CSUM bit set, see the recent linux patch from Willem:

https://lore.kernel.org/netdev/[email protected]/

I think it will make the thing simpler if we always require such bit to be set for GSO packets.

Did you mean forbid VIRTIO_NET_HDR_F_DATA_VALID? I think it means we will prevent vendors from implementing GRO_HW for UDP tunnels.

Quoted 8 lines

I guess we can't change the current status for
the existing GSO types, but for UDP tunnel we could add that requirement from the start, both for the driver -> device direction and for the device -> driver direction.

Even when the device sets the VIRTIO_NET_HDR_F_DATA_VALID, so can always expect a valid csum_start/csum_offset pair, and make things hopefully more robust.

Ok, so you meant you want to mandate csum_start/csum_offset even in the case of VIRTIO_NET_HDR_F_DATA_VALID.

Quoted 3 lines

When VIRTIO_NET_HDR_F_DATA_VALID is set, the pair will NOT be used to validated the csum, just to point to the inner transport header and avoid parsing. WDYT?

It should be fine, but I think it's better to be an independent feature (e.g a new feature flag as it is not specific to tunnels).

If we add too many dependencies for UDP tunnel, it would take too long to be converged.

Thanks

Quoted 5 lines

Thanks,

Paolo

#24 lore raw

PATCH v6 2/2 virtio-net: define UDP tunnel checksum offload feature

In reply to Paolo Abeni

This complements the previous change, additionally introducing the UDP tunnel checksum offload feature.

Differently from the plain checksum offload feature, this depends on UDP tunnel segmentation being available, as outer checksum computation for non GSO packets is cheap and H/W implementation of such feature is complex.

UDP tunnel checksum offload does not introduce additional fields, instead leverages the outer transport offset introduced by the UDP tunnel segmentation feature to locate the outer checksum inside the packet.

When UDP tunnel checksum offload is enabled, the VIRTIO_NET_HDR_F_DATA_VALID bit semantic is extended, in the device -> driver direction, to cover both the inner and the outer checksum validation.

---
 device-types/net/description.tex | 123 ++++++++++++++++++++++++++++---
 1 file changed, 113 insertions(+), 10 deletions(-)
device-types/net/description.tex +113 -10
index eda294d..1225b50 100644
@@ -91,9 +91,15 @@ \subsection{Feature bits}\label{sec:Device Types / Network Device / Feature bits
 \item[VIRTIO_NET_F_GUEST_UDP_TUNNEL_GSO (47)] Driver can receive GSO packets
   carried by a UDP tunnel.
 
+\item[VIRTIO_NET_F_GUEST_UDP_TUNNEL_CSUM (48)] Driver handles packets
+  carried by a UDP tunnel with partial csum for the outer header.
+
 \item[VIRTIO_NET_F_HOST_UDP_TUNNEL_GSO (49)] Device can receive GSO packets
   carried by a UDP tunnel.
 
+\item[VIRTIO_NET_F_HOST_UDP_TUNNEL_CSUM (50)] Device handles packets
+  carried by a UDP tunnel with partial csum for the outer header.
+
 \item[VIRTIO_NET_F_HASH_TUNNEL(51)] Device supports inner header hash for encapsulated packets.
 
 \item[VIRTIO_NET_F_VQ_NOTF_COAL(52)] Device supports virtqueue notification coalescing.
@@ -141,6 +147,7 @@ \subsubsection{Feature bit requirements}\label{sec:Device Types / Network Device
 \item[VIRTIO_NET_F_GUEST_USO6] Requires VIRTIO_NET_F_GUEST_CSUM.
 \item[VIRTIO_NET_F_GUEST_UDP_TUNNEL_GSO] Requires both VIRTIO_NET_F_GUEST_TSO4 and VIRTIO_NET_F_GUEST_TSO6, or
    both VIRTIO_NET_F_GUEST_USO4 and VIRTIO_NET_F_GUEST_USO6.
+\item[VIRTIO_NET_F_GUEST_UDP_TUNNEL_CSUM] Requires VIRTIO_NET_F_GUEST_UDP_TUNNEL_GSO
 
 \item[VIRTIO_NET_F_HOST_TSO4] Requires VIRTIO_NET_F_CSUM.
 \item[VIRTIO_NET_F_HOST_TSO6] Requires VIRTIO_NET_F_CSUM.
@@ -149,6 +156,7 @@ \subsubsection{Feature bit requirements}\label{sec:Device Types / Network Device
 \item[VIRTIO_NET_F_HOST_USO] Requires VIRTIO_NET_F_CSUM.
 \item[VIRTIO_NET_F_HOST_UDP_TUNNEL_GSO] Requires both VIRTIO_NET_F_HOST_TSO4 and VIRTIO_NET_F_HOST_TSO6, or
    both VIRTIO_NET_F_GUEST_USO4 or VIRTIO_NET_F_GUEST_USO6.
++\item[VIRTIO_NET_F_HOST_UDP_TUNNEL_CSUM] Requires VIRTIO_NET_F_HOST_UDP_TUNNEL_GSO
 
 \item[VIRTIO_NET_F_CTRL_RX] Requires VIRTIO_NET_F_CTRL_VQ.
 \item[VIRTIO_NET_F_CTRL_VLAN] Requires VIRTIO_NET_F_CTRL_VQ.
@@ -162,6 +170,13 @@ \subsubsection{Feature bit requirements}\label{sec:Device Types / Network Device
 \item[VIRTIO_NET_F_HASH_TUNNEL] Requires VIRTIO_NET_F_CTRL_VQ along with VIRTIO_NET_F_RSS or VIRTIO_NET_F_HASH_REPORT.
 \end{description}
 
+\begin{note}
+The dependency between UDP_TUNNEL_CSUM and UDP_TUNNEL_GSO is intentionally
+in the opposite direction with respect to the plain GSO features and the plain
+checksum offload because UDP tunnel checksum offload gives very little gain
+for non GSO packets is quite complex to implement in H/W.
+\end{note}
+
 \subsubsection{Legacy Interface: Feature bits}\label{sec:Device Types / Network Device / Feature bits / Legacy Interface: Feature bits}
 \begin{description}
 \item[VIRTIO_NET_F_GSO (6)] Device handles packets with any GSO type. This was supposed to indicate segmentation offload support, but
@@ -391,6 +406,11 @@ \subsection{Device Initialization}\label{sec:Device Types / Network Device / Dev
   the outer UDP checksum is zero - by negotiating the
   VIRTIO_NET_F_HOST_UDP_TUNNEL_GSO feature.
 
+\item If VIRTIO_NET_F_HOST_UDP_TUNNEL_GSO is negotiated, a driver can
+  additionally use TCP segmentation or UDP segmentation on top of UDP
+  encapsulation with the outer header requiring checksum offload,
+  negotiating the VIRTIO_NET_F_HOST_UDP_TUNNEL_CSUM feature.
+
 \item The converse features are also available: a driver can save
   the virtual device some work by negotiating these features.\note{For example, a network packet transported between two guests on
 the same system might not need checksumming at all, nor segmentation,
@@ -399,8 +419,9 @@ \subsection{Device Initialization}\label{sec:Device Types / Network Device / Dev
   checksummed packets can be received, and if it can do that then
   the VIRTIO_NET_F_GUEST_TSO4, VIRTIO_NET_F_GUEST_TSO6,
   VIRTIO_NET_F_GUEST_UFO, VIRTIO_NET_F_GUEST_ECN, VIRTIO_NET_F_GUEST_USO4,
-  VIRTIO_NET_F_GUEST_USO6 and VIRTIO_NET_F_GUEST_UDP_TUNNEL_GSO
-  are the input equivalents of the features described above.
+  VIRTIO_NET_F_GUEST_USO6 VIRTIO_NET_F_GUEST_UDP_TUNNEL_GSO and
+  VIRTIO_NET_F_GUEST_UDP_TUNNEL_CSUM are the input equivalents of
+  the features described above.
   See \ref{sec:Device Types / Network Device / Device Operation /
 Setting Up Receive Buffers}~\nameref{sec:Device Types / Network
 Device / Device Operation / Setting Up Receive Buffers} and
@@ -424,6 +445,7 @@ \subsection{Device Operation}\label{sec:Device Types / Network Device / Device O
 #define VIRTIO_NET_HDR_F_NEEDS_CSUM    1
 #define VIRTIO_NET_HDR_F_DATA_VALID    2
 #define VIRTIO_NET_HDR_F_RSC_INFO      4
+#define VIRTIO_NET_HDR_F_UDP_TUNNEL_CSUM 8
         u8 flags;
 #define VIRTIO_NET_HDR_GSO_NONE        0
 #define VIRTIO_NET_HDR_GSO_TCPV4       1
@@ -490,7 +512,7 @@ \subsubsection{Packet Transmission}\label{sec:Device Types / Network Device / De
     new (16 bit ones' complement) checksum is placed by the device.
 
   \item The TCP checksum field in the packet is set to the sum
-    of the TCP pseudo header, so that replacing it by the ones'
+    of the TCP pseudo header, so that replacing it with the ones'
     complement checksum of the TCP header and body will give the
     correct result.
   \end{itemize}
@@ -542,8 +564,11 @@ \subsubsection{Packet Transmission}\label{sec:Device Types / Network Device / De
 \item If the driver negotiated the VIRTIO_NET_F_HOST_UDP_TUNNEL_GSO feature,
   the VIRTIO_NET_HDR_GSO_UDP_TUNNEL bit in \field{gso_type} indicates that
   the GSO protocol is encapsulated in a UDP tunnel.
-  The outer UDP header must not require checksumming, e.g. the outer UDP checksum
-  must be zero.
+  If the outer UDP header requires checksumming, the driver must have
+  additionally negotiated the VIRTIO_NET_F_HOST_UDP_TUNNEL_CSUM feature
+  and offloaded the outer checksum accordingly, otherwise
+  the outer UDP header must not require checksumming, e.g. the outer
+  UDP checksum must be zero.
   The other tunnel-related fields indicate how to replicate the packet
   headers to cut it into smaller packets:
 
@@ -559,6 +584,20 @@ \subsubsection{Packet Transmission}\label{sec:Device Types / Network Device / De
       the packet.
   \end{itemize}
 
+\item If the driver negotiated the VIRTIO_NET_F_HOST_UDP_TUNNEL_CSUM feature,
+  the transmitted packet is a GSO one encapsulated in a UDP tunnel, and
+  the outer UDP header requires checksumming, the driver can skip checksumming
+  the outer header:
+
+  \begin{itemize}
+  \item \field{flags} has the VIRTIO_NET_HDR_F_UDP_TUNNEL_CSUM set,
+
+  \item The outer UDP checksum field in the packet is set to the sum
+    of the UDP pseudo header, so that replacing it by the ones'
+    complement checksum of the outer UDP header and payload will give the
+    correct result.
+  \end{itemize}
+
 \item \field{num_buffers} is set to zero.  This field is unused on transmitted packets.
 
 \item The header and packet are added as one output descriptor to the
@@ -606,12 +645,23 @@ \subsubsection{Packet Transmission}\label{sec:Device Types / Network Device / De
 bit in \field{gso_type}.
 
 The driver MUST NOT send to the device TCP or UDP GSO packets over UDP tunnel
+requiring segmentation and outer UDP checksum offload, unless both the
+VIRTIO_NET_F_HOST_UDP_TUNNEL_GSO and VIRTIO_NET_F_HOST_UDP_TUNNEL_CSUM features
+are negotiated, in which case the driver MUST set the VIRTIO_NET_HDR_GSO_UDP_TUNNEL
+bit in the \field{gso_type} and the VIRTIO_NET_HDR_F_UDP_TUNNEL_CSUM bit in the \field{flags}.
+
+If VIRTIO_NET_F_HOST_UDP_TUNNEL_CSUM is not negotiated, the driver MUST not set
+the VIRTIO_NET_HDR_F_UDP_TUNNEL_CSUM bit in the \field{flags} and
+MUST NOT send to the device TCP or UDP GSO packets over UDP tunnel
 requiring segmentation and outer UDP checksum offload.
 
 The driver MUST NOT set the VIRTIO_NET_HDR_GSO_UDP_TUNNEL together with
 VIRTIO_NET_HDR_GSO_UDP, as the latter is deprecated in favor of UDP_L4
 and no new features will support it.
 
+The driver MUST NOT set the VIRTIO_NET_HDR_F_UDP_TUNNEL_CSUM bit \field{flags}
+without setting the VIRTIO_NET_F_HOST_UDP_TUNNEL_GSO bit in \field{gso_type}.
+
 If the VIRTIO_NET_F_CSUM feature has been negotiated, the
 driver MAY set the VIRTIO_NET_HDR_F_NEEDS_CSUM bit in
 \field{flags}, if so:
@@ -661,8 +711,27 @@ \subsubsection{Packet Transmission}\label{sec:Device Types / Network Device / De
   protocol, and \field{inner_nh_offset} to the inner network header offset.
 \end{itemize}
 
-If the bit VIRTIO_NET_HDR_GSO_UDP_TUNNEL in \field{gso_type} is set, the outer
-UDP header MUST NOT require checksumming. That is the, the outer UDP checksum
+If the VIRTIO_NET_F_HOST_UDP_TUNNEL_CSUM feature has been negotiated, and the
+bit VIRTIO_NET_HDR_GSO_UDP_TUNNEL in \field{gso_type} is set, the
+driver MAY set the VIRTIO_NET_HDR_F_UDP_TUNNEL_CSUM bit in
+\field{flags}, if so:
+\begin{enumerate}
+\item the driver MUST validate the packet checksum at
+	offset 6 from \field{outer_th_offset} as well as all
+	preceding offsets;
+\item the driver MUST set the packet outer UDP header checksum
+      to the outer UDP pseudo header;
+\end{enumerate}
+
+\begin{note}
+calculating a ones' complement checksum from \field{outer_th_offset}
+up until the end of the packet and storing the result at offset 6
+from \field{outer_th_offset} will result in a fully checksummed outer UDP packet;
+\end{note}
+
+If the bit VIRTIO_NET_HDR_GSO_UDP_TUNNEL in \field{gso_type} is set
+and the VIRTIO_NET_F_HOST_UDP_TUNNEL_CSUM feature has not been negotiated, the outer
+UDP header MUST NOT require checksumming. That is, the outer UDP checksum
 value must be 0, or the valid complete checksum for such header.
 
 \begin{note}
@@ -675,6 +744,10 @@ \subsubsection{Packet Transmission}\label{sec:Device Types / Network Device / De
 been negotiated, the driver MUST NOT set the VIRTIO_NET_HDR_F_UDP_TUNNEL bit
 in \field{gso_type}.
 
+If the VIRTIO_NET_F_HOST_UDP_TUNNEL_CSUM option has not been negotiated,
+the driver MUST NOT set the VIRTIO_NET_HDR_F_UDP_TUNNEL_CSUM bit
+in \field{flags}.
+
 The driver SHOULD accept the VIRTIO_NET_F_GUEST_HDRLEN feature if it has
 been offered, and if it's able to provide the exact header length.
 
@@ -839,6 +912,12 @@ \subsubsection{Processing of Incoming Packets}\label{sec:Device Types / Network
   bit MAY be set. In such case the \field{outer_th_offset}, \field{inner_protocol_type},
   and \field{inner_nh_offset} fields indicate the corresponding
   headers information.
+\item If the VIRTIO_NET_F_GUEST_UDP_TUNNEL_CSUM feature was negotiated, and
+  the VIRTIO_NET_HDR_GSO_UDP_TUNNEL bit is set in \field{gso_type}, the
+  VIRTIO_NET_HDR_F_UDP_TUNNEL_CSUM bit in the \field{flags} can be set: if so,
+  the outer UDP header checksum at offset 6 from \field{outer_th_offset} have
+  been validated. The checksum on the packet is incomplete and the
+  \field{outer_th_offset} indicates how to calculate it.
 
 \end{enumerate}
 
@@ -886,6 +965,9 @@ \subsubsection{Processing of Incoming Packets}\label{sec:Device Types / Network
 If VIRTIO_NET_F_GUEST_UDP_TUNNEL_GSO is not negotiated the device MUST NOT set
 the VIRTIO_NET_HDR_GSO_UDP_TUNNEL bit in \field{gso_type}.
 
+If VIRTIO_NET_F_GUEST_UDP_TUNNEL_CSUM is not negotiated the device MUST NOT set
+the VIRTIO_NET_HDR_F_UDP_TUNNEL_CSUM bit in \field{flags}.
+
 The device SHOULD NOT send to the driver TCP packets requiring segmentation offload
 which have the Explicit Congestion Notification bit set, unless the
 VIRTIO_NET_F_GUEST_ECN feature is negotiated, in which case the
@@ -917,7 +999,8 @@ \subsubsection{Processing of Incoming Packets}\label{sec:Device Types / Network
 \field{inner_nh_offset} fields to the corresponding header information, and the outer
 UDP header MUST NOT require checksum offload.
 
-The device MUST NOT send the driver TCP or UDP GSO packets encapsulated in UDP
+If the VIRTIO_NET_F_GUEST_UDP_TUNNEL_CSUM feature has not been negotiated,
+the device MUST NOT send the driver TCP or UDP GSO packets encapsulated in UDP
 tunnel and requiring segmentation and outer checksum offload.
 
 If none of the VIRTIO_NET_F_GUEST_TSO4, TSO6, UFO, USO4 or USO6 options have
@@ -943,8 +1026,27 @@ \subsubsection{Processing of Incoming Packets}\label{sec:Device Types / Network
 If the VIRTIO_NET_F_GUEST_CSUM feature has been negotiated, the
 device MAY set the VIRTIO_NET_HDR_F_DATA_VALID bit in
 \field{flags}, if so, the device MUST validate the packet
-checksum (in case of multiple encapsulated protocols, one level
-of checksums is validated).
+checksum. If the VIRTIO_NET_F_GUEST_UDP_TUNNEL_CSUM feature has been negotiated,
+and the VIRTIO_NET_HDR_F_UDP_TUNNEL bit set, both the outer UDP checksum
+and the inner transport checksum have been
+validated, otherwise only one level of checksums (the inner one in case
+of tunnels) has been validated.
+
+If the VIRTIO_NET_F_GUEST_UDP_TUNNEL_CSUM feature has been negotiated,
+the bit VIRTIO_NET_HDR_GSO_UDP_TUNNEL is set in \field{gso_type}
+device MAY set the VIRTIO_NET_HDR_F_UDP_TUNNEL_CSUM bit in
+\field{flags}, if so:
+\begin{enumerate}
+\item the device MUST validate the packet checksum at
+	offset 6 from \field{outer_th_offset};
+\item the device MUST set the packet checksum stored in the
+	receive buffer to the outer UDP pseudo header;
+\end{enumerate}
+
+Otherwise, if the VIRTIO_NET_HDR_GSO_UDP_TUNNEL is set in
+\field{gso_type} and the bit VIRTIO_NET_HDR_F_UDP_TUNNEL_CSUM is cleared in
+\field{flags}, the device MUST either provide a zero outer UDP header
+checksum or a fully checksummed outer UDP header.
 
 \drivernormative{\paragraph}{Processing of Incoming
 Packets}{Device Types / Network Device / Device Operation /
@@ -1730,6 +1832,7 @@ \subsubsection{Control Virtqueue}\label{sec:Device Types / Network Device / Devi
 #define VIRTIO_NET_F_GUEST_ECN        9
 #define VIRTIO_NET_F_GUEST_UFO        10
 #define VIRTIO_NET_F_GUEST_UDP_TUNNEL_GSO  47
+#define VIRTIO_NET_F_GUEST_UDP_TUNNEL_CSUM 48
 #define VIRTIO_NET_F_GUEST_USO4       54
 #define VIRTIO_NET_F_GUEST_USO6       55
 
Signature · 1 line
2.45.2
#25 lore raw

Re: [PATCH v6 2/2] virtio-net: define UDP tunnel checksum offload feature

Quoted 26 lines · Paolo Abeni

This complements the previous change, additionally introducing the UDP tunnel checksum offload feature.

Differently from the plain checksum offload feature, this depends on UDP tunnel segmentation being available, as outer checksum computation for non GSO packets is cheap and H/W implementation of such feature is complex.

UDP tunnel checksum offload does not introduce additional fields, instead leverages the outer transport offset introduced by the UDP tunnel segmentation feature to locate the outer checksum inside the packet.

When UDP tunnel checksum offload is enabled, the VIRTIO_NET_HDR_F_DATA_VALID bit semantic is extended, in the device -> driver direction, to cover both the inner and the outer checksum validation.

+\item[VIRTIO_NET_F_GUEST_UDP_TUNNEL_CSUM (48)] Driver handles packets

device-types/net/description.tex +3 -0
+  carried by a UDP tunnel with partial csum for the outer header.
+
 
 \item[VIRTIO_NET_F_HOST_UDP_TUNNEL_GSO (49)] Device can receive GSO packets
   carried by a UDP tunnel.
 
+\item[VIRTIO_NET_F_HOST_UDP_TUNNEL_CSUM (50)] Device handles packets

device-types/net/description.tex \subsection{Feature bits}\label{sec:Device Types / Network Device / Feature bits

Quoted 10 lines
device-types/net/description.tex +8 -0
+  carried by a UDP tunnel with partial csum for the outer header.
+
 \item[VIRTIO_NET_F_HASH_TUNNEL(51)] Device supports inner header hash for encapsulated packets.
 
+\begin{note}
+The dependency between UDP_TUNNEL_CSUM and UDP_TUNNEL_GSO is intentionally
+in the opposite direction with respect to the plain GSO features and the plain
+checksum offload because UDP tunnel checksum offload gives very little gain
+for non GSO packets is quite complex to implement in H/W.
+\end{note}

Plan was to call this UDP_TUNNEL_GSO_CSUM, to make this implicit dependency more explicit? It really only is checksum offload for the outer checksum that this feature covers.

device-types/net/description.tex \subsection{Device Initialization}\label{sec:Device Types / Network Device / Dev · line 424

Quoted 12 lines

\item The converse features are also available: a driver can save
the virtual device some work by negotiating these features.\note{For example, a network packet transported between two guests on
the same system might not need checksumming at all, nor segmentation,

device-types/net/description.tex +3 -2
@@ -399,8 +419,9 @@ \subsection{Device Initialization}\label{sec:Device Types / Network Device / Dev
   checksummed packets can be received, and if it can do that then
   the VIRTIO_NET_F_GUEST_TSO4, VIRTIO_NET_F_GUEST_TSO6,
   VIRTIO_NET_F_GUEST_UFO, VIRTIO_NET_F_GUEST_ECN, VIRTIO_NET_F_GUEST_USO4,
-  VIRTIO_NET_F_GUEST_USO6 and VIRTIO_NET_F_GUEST_UDP_TUNNEL_GSO
-  are the input equivalents of the features described above.
+  VIRTIO_NET_F_GUEST_USO6 VIRTIO_NET_F_GUEST_UDP_TUNNEL_GSO and
+  VIRTIO_NET_F_GUEST_UDP_TUNNEL_CSUM are the input equivalents of
+  the features described above.

Minor, not specific to this feature and more to the spec maintainers:

Does it make sense to put each constant on its own line? To avoid having to edit existing lines when expanding. This is source text, so that does not show in the generated output anyway:

and if it can do that then the VIRTIO_NET_F_GUEST_TSO4,
VIRTIO_NET_F_GUEST_TSO6,
VIRTIO_NET_F_GUEST_UFO,
VIRTIO_NET_F_GUEST_ECN,
VIRTIO_NET_F_GUEST_USO4,
VIRTIO_NET_F_GUEST_USO6,
+ VIRTIO_NET_F_GUEST_UDP_TUNNEL_GSO,
are the input equivalents of the features described above.

The trailing comma is admittedly a bit ugly. If fixing that up, the diff is still cleaner with one constant per line:

- VIRTIO_NET_F_GUEST_USO6
+ VIRTIO_NET_F_GUEST_USO6,
+ VIRTIO_NET_F_GUEST_UDP_TUNNEL_GSO

device-types/net/description.tex \subsubsection{Packet Transmission}\label{sec:Device Types / Network Device / De · line 515

Quoted 6 lines
device-types/net/description.tex +1 -1
@@ -490,7 +512,7 @@ \subsubsection{Packet Transmission}\label{sec:Device Types / Network Device / De
     new (16 bit ones' complement) checksum is placed by the device.
 
   \item The TCP checksum field in the packet is set to the sum
-    of the TCP pseudo header, so that replacing it by the ones'
+    of the TCP pseudo header, so that replacing it with the ones'

Unintentional change?

device-types/net/description.tex \subsubsection{Packet Transmission}\label{sec:Device Types / Network Device / De · line 599

Quoted 8 lines
device-types/net/description.tex +8 -0
+  \begin{itemize}
+  \item \field{flags} has the VIRTIO_NET_HDR_F_UDP_TUNNEL_CSUM set,
+
+  \item The outer UDP checksum field in the packet is set to the sum
+    of the UDP pseudo header, so that replacing it by the ones'
+    complement checksum of the outer UDP header and payload will give the
+    correct result.
+  \end{itemize}

"set to the sum of the UDP pseudo header [of each segment]"?

as it will not be the UDP length of the gso packet.

#26 lore raw

Re: [PATCH v6 2/2] virtio-net: define UDP tunnel checksum offload feature

Quoted 11 lines · Willem de Bruijn

device-types/net/description.tex \subsubsection{Feature bit requirements}\label{sec:Device Types / Network Device · line 178

Quoted 6 lines · Paolo Abeni

+\begin{note}

device-types/net/description.tex +5 -0
+The dependency between UDP_TUNNEL_CSUM and UDP_TUNNEL_GSO is intentionally
+in the opposite direction with respect to the plain GSO features and the plain
+checksum offload because UDP tunnel checksum offload gives very little gain
+for non GSO packets is quite complex to implement in H/W.
+\end{note}

Plan was to call this UDP_TUNNEL_GSO_CSUM, to make this implicit dependency more explicit? It really only is checksum offload for the outer checksum that this feature covers.

Indeed. I lost that change in the split. I'll fix in the next revision.

Quoted 36 lines

device-types/net/description.tex \subsection{Device Initialization}\label{sec:Device Types / Network Device / Dev · line 424

Quoted 12 lines

\item The converse features are also available: a driver can save
the virtual device some work by negotiating these features.\note{For example, a network packet transported between two guests on
the same system might not need checksumming at all, nor segmentation,

device-types/net/description.tex +3 -2
@@ -399,8 +419,9 @@ \subsection{Device Initialization}\label{sec:Device Types / Network Device / Dev
    checksummed packets can be received, and if it can do that then
    the VIRTIO_NET_F_GUEST_TSO4, VIRTIO_NET_F_GUEST_TSO6,
    VIRTIO_NET_F_GUEST_UFO, VIRTIO_NET_F_GUEST_ECN, VIRTIO_NET_F_GUEST_USO4,
-  VIRTIO_NET_F_GUEST_USO6 and VIRTIO_NET_F_GUEST_UDP_TUNNEL_GSO
-  are the input equivalents of the features described above.
+  VIRTIO_NET_F_GUEST_USO6 VIRTIO_NET_F_GUEST_UDP_TUNNEL_GSO and
+  VIRTIO_NET_F_GUEST_UDP_TUNNEL_CSUM are the input equivalents of
+  the features described above.

Minor, not specific to this feature and more to the spec maintainers:

Does it make sense to put each constant on its own line? To avoid having to edit existing lines when expanding. This is source text, so that does not show in the generated output anyway:

and if it can do that then the VIRTIO_NET_F_GUEST_TSO4,
VIRTIO_NET_F_GUEST_TSO6,
VIRTIO_NET_F_GUEST_UFO,
VIRTIO_NET_F_GUEST_ECN,
VIRTIO_NET_F_GUEST_USO4,
VIRTIO_NET_F_GUEST_USO6,
+ VIRTIO_NET_F_GUEST_UDP_TUNNEL_GSO,
are the input equivalents of the features described above.

The trailing comma is admittedly a bit ugly. If fixing that up, the diff is still cleaner with one constant per line:

- VIRTIO_NET_F_GUEST_USO6
+ VIRTIO_NET_F_GUEST_USO6,
+ VIRTIO_NET_F_GUEST_UDP_TUNNEL_GSO

Quoted 8 lines

device-types/net/description.tex \subsubsection{Packet Transmission}\label{sec:Device Types / Network Device / De · line 515

Quoted 6 lines
device-types/net/description.tex +1 -1
@@ -490,7 +512,7 @@ \subsubsection{Packet Transmission}\label{sec:Device Types / Network Device / De
      new (16 bit ones' complement) checksum is placed by the device.
 
    \item The TCP checksum field in the packet is set to the sum
-    of the TCP pseudo header, so that replacing it by the ones'
+    of the TCP pseudo header, so that replacing it with the ones'

Unintentional change?

Yep, I piped the patch in a grammar checker and while reflecting the suggestions on the patch itself I unintentionally touched the context, too.

Quoted 9 lines

device-types/net/description.tex \subsubsection{Packet Transmission}\label{sec:Device Types / Network Device / De · line 599

Quoted 8 lines
device-types/net/description.tex +8 -0
+  \begin{itemize}
+  \item \field{flags} has the VIRTIO_NET_HDR_F_UDP_TUNNEL_CSUM set,
+
+  \item The outer UDP checksum field in the packet is set to the sum
+    of the UDP pseudo header, so that replacing it by the ones'
+    complement checksum of the outer UDP header and payload will give the
+    correct result.
+  \end{itemize}
Quoted 3 lines

"set to the sum of the UDP pseudo header [of each segment]"?

as it will not be the UDP length of the gso packet.

AFAICS the Linux tunnel implementation uses the GSO length:

<each UDP tunnel> -> udp_tunnel_xmit_skb -> udp_set_csum

https://elixir.bootlin.com/linux/v6.10.2/source/net/ipv4/udp_tunnel_core.c#L170
https://elixir.bootlin.com/linux/v6.10.2/source/net/ipv4/udp.c#L890

Regardless of the Linux implementation, using the individual segment len will makes things difficult for the last segment (len < gso_size). The caller will have to touch the individual segment headers, which could be quite hard/costily.

Am I missing something?

Thanks,

Paolo