Skip to contentSkip to conversation

virtio-net: Introduce a new field to indicate outer network header offset

PATCH Superseded Kommula Shiva Shankar 2 participants 4 messages Mar 4, 2025 – Mar 28, 2025

v4 of 5 Superseded Latest is v5 →

lore
Share & tools
lore
#0 lore raw

PATCH v4 1/1 virtio-net: Introduce a new field to indicate outer network header offset

This patch introduces a new field in the virtio_net_header called outer_nh_offset, along with a new net device feature, VIRTIO_NET_F_OUT_NET_HEADER.

Currently, drivers lack a dedicated field to signal the start of the network header to the device when performing checksum offload and segmentation offload. This requires reading packet, which significantly affects performance in data path. Additionally, some hardware implementations require knowledge of the outer L3 offset (aka L2 length) for inline IPsec hardware acceleration.

To address this limitation, we propose to introduce a new field in the virtio_net_header called outer_nh_offset

The outer_nh_offset represents the start byte offset of the outer network header from the beginning of the packet

This issue was briefly discussed on the mailing list in a different thread, which can be found here
https://lore.kernel.org/all/DM4PR18MB4269FAAC3CFC7E57E25DFBD2DF8B2@DM4PR18MB4269.namprd18.prod.outlook.com/

v3 -> v4
- Removed the union of new flag with existing flags. Added as a separate field in the virtio net header
- Renamed out_nh_offset to outer_nh_offset to maintain consistency with other fields
- Spellchecks in commit message description
v3: https://lore.kernel.org/all/[email protected]

v2 -> v3:
- Rebase to virtio-1.4
- Addressed pending review comments related to wording.
v2: https://lore.kernel.org/all/[email protected]/

v1 -> v2:
- explicitly state that the out_nh_offset can be set only when a valid network header is present.
- updated out_nh_offset usage in the RX direction.
- minor word cleanup.
v1: https://lore.kernel.org/virtio-comment/[email protected]/

---
 device-types/net/description.tex | 31 +++++++++++++++++++++++++++++++
 1 file changed, 31 insertions(+)
device-types/net/description.tex +31 -0
index 8d30673..ca1242f 100644
@@ -136,6 +136,10 @@ \subsection{Feature bits}\label{sec:Device Types / Network Device / Feature bits
 
 \item[VIRTIO_NET_F_HOST_UDP_TUNNEL_GSO_CSUM (68)] Device handles packets
   carried by a UDP tunnel with partial csum for the outer header.
+
+\item[VIRTIO_NET_F_OUT_NET_HEADER(69)] Driver can provide the start of \field{outer_nh_offset}
+ value. Device gains advantage by not reading packet to calculate outer network
+ header offset
 \end{description}
 
 \subsubsection{Feature bit requirements}\label{sec:Device Types / Network Device / Feature bits / Feature bit requirements}
@@ -599,6 +603,7 @@ \subsection{Device Operation}\label{sec:Device Types / Network Device / Device O
         le16 padding_reserved;  (Only if VIRTIO_NET_F_HASH_REPORT negotiated)
         le16 outer_th_offset    (Only if VIRTIO_NET_F_HOST_UDP_TUNNEL_GSO or VIRTIO_NET_F_GUEST_UDP_TUNNEL_GSO negotiated)
         le16 inner_nh_offset;   (Only if VIRTIO_NET_F_HOST_UDP_TUNNEL_GSO or VIRTIO_NET_F_GUEST_UDP_TUNNEL_GSO negotiated)
+        le16 outer_nh_offset;   /* Only if VIRTIO_NET_F_OUT_NET_HEADER negotiated */
 };
 \end{lstlisting}
 
@@ -633,6 +638,11 @@ \subsubsection{Packet Transmission}\label{sec:Device Types / Network Device / De
 \item The driver can send a completely checksummed packet.  In this case,
   \field{flags} will be zero, and \field{gso_type} will be VIRTIO_NET_HDR_GSO_NONE.
 
+\item When VIRTIO_NET_F_OUT_NET_HEADER is negotiated, the driver MAY optionally provide
+   the \field{outer_nh_offset} value. A nonzero value of \field{outer_nh_offset} indicates
+   a valid outer network header with in the packet, and specifies the offset in bytes from
+   the beginning of the packet. Otherwise \field{outer_nh_offset} MUST not be used.
+
 \item If the driver negotiated VIRTIO_NET_F_CSUM, it can skip
   checksumming the packet:
   \begin{itemize}
@@ -767,6 +777,11 @@ \subsubsection{Packet Transmission}\label{sec:Device Types / Network Device / De
 \field{flags} to zero and SHOULD supply a fully checksummed
 packet to the device.
 
+If the VIRTIO_NET_F_OUT_NET_HEADER feature has been negotiated,
+the driver MAY set \field{outer_nh_offset} to indicate the start of the
+outer network header offset, if the packet contains a valid network header.
+Otherwise, \field{outer_nh_offset} is not used.
+
 If VIRTIO_NET_F_HOST_TSO4 is negotiated, the driver MAY set
 \field{gso_type} to VIRTIO_NET_HDR_GSO_TCPV4 to request TCPv4
 segmentation, otherwise the driver MUST NOT set
@@ -948,6 +963,11 @@ \subsubsection{Packet Transmission}\label{sec:Device Types / Network Device / De
 If VIRTIO_NET_HDR_F_NEEDS_CSUM bit in \field{flags} is not set, the
 device MUST NOT use the \field{csum_start} and \field{csum_offset}.
 
+If the VIRTIO_NET_F_OUT_NET_HEADER feature has been negotiated,
+and \field{outer_nh_offset} is not zero, the device MAY use \field{outer_nh_offset}
+as the outer network header offset. Otherwise, device MUST NOT use
+the \field{outer_nh_offset}.
+
 If one of the VIRTIO_NET_F_HOST_TSO4, TSO6, USO or UFO options have
 been negotiated:
 \begin{itemize}
@@ -1102,6 +1122,9 @@ \subsubsection{Processing of Incoming Packets}\label{sec:Device Types / Network
   both the outer UDP checksum and the inner transport checksum
   have been validated, otherwise only one level of checksums (the outer one
   in case of tunnels) has been validated.
+\item If the VIRTIO_NET_F_OUT_NET_HEADER has been negotiated, and if the packet
+  contains a valid network header, \field{outer_nh_offset} MAY be set to indicate the
+  outer network header offset in packet.
 \end{enumerate}
 
 Additionally, VIRTIO_NET_F_GUEST_CSUM, TSO4, TSO6, UDP, UDP_TUNNEL
@@ -1215,6 +1238,10 @@ \subsubsection{Processing of Incoming Packets}\label{sec:Device Types / Network
 device MUST set the VIRTIO_NET_HDR_GSO_ECN bit in
 \field{gso_type}.
 
+If VIRTIO_NET_F_OUT_NET_HEADER has been negotiated, the device MAY
+set the \field{outer_nh_offset} to indicate outer network header offset, if packet contains
+a valid network header. Otherwise, the device MUST not use \field{outer_nh_offset}.
+
 If the VIRTIO_NET_F_GUEST_CSUM feature has been negotiated, the
 device MAY set the VIRTIO_NET_HDR_F_NEEDS_CSUM bit in
 \field{flags}, if so:
@@ -1302,6 +1329,10 @@ \subsubsection{Processing of Incoming Packets}\label{sec:Device Types / Network
 
 The driver MUST ignore \field{flag} bits that it does not recognize.
 
+If VIRTIO_NET_F_OUT_NET_HEADER has been negotiated, and if \field{outer_nh_offset}
+is nonzero, the driver MAY use \field{outer_nh_offset} as outer network header
+offset. Otherwise, the driver MUST not use the \field{outer_nh_offset}.
+
 If VIRTIO_NET_HDR_F_NEEDS_CSUM bit in \field{flags} is not set or
 if VIRTIO_NET_HDR_F_RSC_INFO bit \field{flags} is set, the
 driver MUST NOT use the \field{csum_start} and \field{csum_offset}.
Signature · 1 line
2.43.0
#1 lore raw
Quoted 5 lines

From: Kommula Shiva Shankar <[email protected]>
Sent: Tuesday, March 4, 2025 1:30 PM

This patch introduces a new field in the virtio_net_header called

s/virtio_net_header/virtio_net_hdr

Quoted 7 lines

outer_nh_offset, along with a new net device feature, VIRTIO_NET_F_OUT_NET_HEADER.

Currently, drivers lack a dedicated field to signal the start of the network header to the device when performing checksum offload and segmentation offload. This requires reading packet, which significantly affects performance in data path.

s/This requires reading packet/This requires the device to read the packet in data path ..

Quoted 5 lines

Additionally, some hardware implementations require knowledge of the outer L3 offset (aka L2 length) for inline IPsec hardware acceleration.

To address this limitation, we propose to introduce a new field in the virtio_net_header called outer_nh_offset

s/virtio_net_header/virtio_net_hdr

Quoted 3 lines

The outer_nh_offset represents the start byte offset of the outer network header from the beginning of the packet

Add full stop at the end of the sentence.

Quoted 1 line

[..]

Quoted 17 lines
---
 device-types/net/description.tex | 31 +++++++++++++++++++++++++++++++
 1 file changed, 31 insertions(+)
device- renamed from device-types/net/description.tex +0 -0

types/net/description.tex
index 8d30673..ca1242f 100644

device-types/net/description.tex +0 -0
@@ -136,6 +136,10 @@ \subsection{Feature bits}\label{sec:Device Types /

Network Device / Feature bits

\item[VIRTIO_NET_F_HOST_UDP_TUNNEL_GSO_CSUM (68)] Device handles
packets
carried by a UDP tunnel with partial csum for the outer header.

device-types/net/description.tex

Quoted 4 lines
device-types/net/description.tex +4 -0
+
+\item[VIRTIO_NET_F_OUT_NET_HEADER(69)] Driver can provide the start of
+\field{outer_nh_offset}  value. Device gains advantage by not reading
+packet to calculate outer network  header offset

Please add the full stop at the end of above sentence.

Quoted 39 lines

\end{description}

\subsubsection{Feature bit requirements}\label{sec:Device Types / Network
Device / Feature bits / Feature bit requirements} @@ -599,6 +603,7 @@ \subsection{Device Operation}\label{sec:Device Types / Network Device / Device O
le16 padding_reserved; (Only if VIRTIO_NET_F_HASH_REPORT
negotiated)
le16 outer_th_offset (Only if VIRTIO_NET_F_HOST_UDP_TUNNEL_GSO
or VIRTIO_NET_F_GUEST_UDP_TUNNEL_GSO negotiated)
le16 inner_nh_offset; (Only if VIRTIO_NET_F_HOST_UDP_TUNNEL_GSO
or VIRTIO_NET_F_GUEST_UDP_TUNNEL_GSO negotiated)
+ le16 outer_nh_offset; /* Only if VIRTIO_NET_F_OUT_NET_HEADER
negotiated */
};
\end{lstlisting}

device-types/net/description.tex +0 -0
@@ -633,6 +638,11 @@ \subsubsection{Packet

Transmission}\label{sec:Device Types / Network Device / De \item The driver can send a completely checksummed packet. In this case,
\field{flags} will be zero, and \field{gso_type} will be
VIRTIO_NET_HDR_GSO_NONE.

+\item When VIRTIO_NET_F_OUT_NET_HEADER is negotiated, the driver MAY optionally provide
+ the \field{outer_nh_offset} value. A nonzero value of
\field{outer_nh_offset} indicates
+ a valid outer network header with in the packet, and specifies the offset in
bytes from
+ the beginning of the packet. Otherwise \field{outer_nh_offset} MUST not
be used.

device-types/net/description.tex +1 -0
+
 \item If the driver negotiated VIRTIO_NET_F_CSUM, it can skip
   checksumming the packet:
   \begin{itemize}
@@ -767,6 +777,11 @@ \subsubsection{Packet

Transmission}\label{sec:Device Types / Network Device / De \field{flags} to zero and SHOULD supply a fully checksummed packet to the device.

device-types/net/description.tex \subsubsection{Packet Transmission}\label{sec:Device Types / Network Device / De · line 784

Quoted 5 lines
device-types/net/description.tex +5 -0
+If the VIRTIO_NET_F_OUT_NET_HEADER feature has been negotiated, the
+driver MAY set \field{outer_nh_offset} to indicate the start of the
+outer network header offset, if the packet contains a valid network header.
+Otherwise, \field{outer_nh_offset} is not used.
+

s/MAY be set to/MAY be set to non zero value/

Here and few more places below.

Quoted 45 lines

If VIRTIO_NET_F_HOST_TSO4 is negotiated, the driver MAY set
\field{gso_type} to VIRTIO_NET_HDR_GSO_TCPV4 to request TCPv4 segmentation, otherwise the driver MUST NOT set @@ -948,6 +963,11 @@ \subsubsection{Packet Transmission}\label{sec:Device Types / Network Device / De If VIRTIO_NET_HDR_F_NEEDS_CSUM bit in \field{flags} is not set, the device MUST NOT use the \field{csum_start} and \field{csum_offset}.

device-types/net/description.tex +5 -0
+If the VIRTIO_NET_F_OUT_NET_HEADER feature has been negotiated, and
+\field{outer_nh_offset} is not zero, the device MAY use
+\field{outer_nh_offset} as the outer network header offset. Otherwise,
+device MUST NOT use the \field{outer_nh_offset}.
+
 If one of the VIRTIO_NET_F_HOST_TSO4, TSO6, USO or UFO options have

been negotiated:
\begin{itemize}

device-types/net/description.tex +0 -0
@@ -1102,6 +1122,9 @@ \subsubsection{Processing of Incoming

Packets}\label{sec:Device Types / Network
both the outer UDP checksum and the inner transport checksum have been validated, otherwise only one level of checksums (the outer one in case of tunnels) has been validated.
+\item If the VIRTIO_NET_F_OUT_NET_HEADER has been negotiated, and if

device-types/net/description.tex +4 -0
+the packet
+  contains a valid network header, \field{outer_nh_offset} MAY be set
+to indicate the
+  outer network header offset in packet.
 \end{enumerate}
 
 Additionally, VIRTIO_NET_F_GUEST_CSUM, TSO4, TSO6, UDP, UDP_TUNNEL
@@ -1215,6 +1238,10 @@ \subsubsection{Processing of Incoming

Packets}\label{sec:Device Types / Network device MUST set the VIRTIO_NET_HDR_GSO_ECN bit in \field{gso_type}.

+If VIRTIO_NET_F_OUT_NET_HEADER has been negotiated, the device MAY set

device-types/net/description.tex +2 -0
+the \field{outer_nh_offset} to indicate outer network header offset, if
+packet contains a valid network header. Otherwise, the device MUST not use

\field{outer_nh_offset}.
+
If the VIRTIO_NET_F_GUEST_CSUM feature has been negotiated, the device
MAY set the VIRTIO_NET_HDR_F_NEEDS_CSUM bit in \field{flags}, if so:

device-types/net/description.tex +0 -0
@@ -1302,6 +1329,10 @@ \subsubsection{Processing of Incoming

Packets}\label{sec:Device Types / Network

The driver MUST ignore \field{flag} bits that it does not recognize.

Quoted 10 lines

+If VIRTIO_NET_F_OUT_NET_HEADER has been negotiated, and if +\field{outer_nh_offset} is nonzero, the driver MAY use +\field{outer_nh_offset} as outer network header offset. Otherwise, the driver MUST not use the \field{outer_nh_offset}.
+
If VIRTIO_NET_HDR_F_NEEDS_CSUM bit in \field{flags} is not set or if
VIRTIO_NET_HDR_F_RSC_INFO bit \field{flags} is set, the driver MUST NOT use the \field{csum_start} and \field{csum_offset}.
--
2.43.0

Can you please also describe the description and requirements for the virtio_net_hdr size to indicate that when this feature is negotiated, the vnet hdr size is all the way upto this field. i.e. when HASH_REPORT is not negotiated and NET_HEADER is negotiated, this field does not move up etc.

Please also add 6 bytes of reserved to have the vnet hdr aligned to 64-bit address.

#2 lore raw
Quoted 179 lines

-----Original Message-----
From: Parav Pandit <[email protected]>
Sent: Sunday, March 9, 2025 10:02 PM
To: Shiva Shankar Kommula <[email protected]>; [email protected]; [email protected]; [email protected]; [email protected]
Cc: Nithin Kumar Dabilpuram <[email protected]>; Jerin Jacob <[email protected]>; Rahul Bhansali <[email protected]>
Subject: [EXTERNAL] RE: [PATCH v4] virtio-net: Introduce a new field to indicate outer network header offset

Quoted 10 lines

From: Kommula Shiva Shankar <kshankar@ marvell. com> > Sent: Tuesday, March 4, 2025 1: 30 PM > > This patch introduces a new field in the virtio_net_header called s/virtio_net_header/virtio_net_hdr > outer_nh_offset, along

From: Kommula Shiva Shankar <[email protected]>
Sent: Tuesday, March 4, 2025 1:30 PM

This patch introduces a new field in the virtio_net_header called

s/virtio_net_header/virtio_net_hdr

Quoted 7 lines

outer_nh_offset, along with a new net device feature, VIRTIO_NET_F_OUT_NET_HEADER.

Currently, drivers lack a dedicated field to signal the start of the network header to the device when performing checksum offload and segmentation offload. This requires reading packet, which significantly affects performance in data path.

s/This requires reading packet/This requires the device to read the packet in data path ..

Quoted 6 lines

Additionally, some hardware implementations require knowledge of the outer
L3 offset (aka L2 length) for inline IPsec hardware acceleration.

To address this limitation, we propose to introduce a new field in the virtio_net_header called outer_nh_offset

s/virtio_net_header/virtio_net_hdr

Quoted 3 lines

The outer_nh_offset represents the start byte offset of the outer network header from the beginning of the packet

Add full stop at the end of the sentence.

Quoted 1 line

[..]

Quoted 16 lines
---
 device-types/net/description.tex | 31 +++++++++++++++++++++++++++++++
 1 file changed, 31 insertions(+)
device- renamed from device-types/net/description.tex +0 -0

types/net/description.tex index 8d30673..ca1242f 100644

device-types/net/description.tex +0 -0
@@ -136,6 +136,10 @@ \subsection{Feature bits}\label{sec:Device Types

/ Network Device / Feature bits

\item[VIRTIO_NET_F_HOST_UDP_TUNNEL_GSO_CSUM (68)] Device handles
packets
carried by a UDP tunnel with partial csum for the outer header.

device-types/net/description.tex

Quoted 4 lines
device-types/net/description.tex +4 -0
+
+\item[VIRTIO_NET_F_OUT_NET_HEADER(69)] Driver can provide the start
+of \field{outer_nh_offset}  value. Device gains advantage by not
+reading packet to calculate outer network  header offset

Please add the full stop at the end of above sentence.

Quoted 23 lines

\end{description}

\subsubsection{Feature bit requirements}\label{sec:Device Types /
Network Device / Feature bits / Feature bit requirements} @@ -599,6 +603,7 @@ \subsection{Device Operation}\label{sec:Device Types / Network Device / Device O
le16 padding_reserved; (Only if VIRTIO_NET_F_HASH_REPORT
negotiated)
le16 outer_th_offset (Only if VIRTIO_NET_F_HOST_UDP_TUNNEL_GSO
or VIRTIO_NET_F_GUEST_UDP_TUNNEL_GSO negotiated)
le16 inner_nh_offset; (Only if VIRTIO_NET_F_HOST_UDP_TUNNEL_GSO
or VIRTIO_NET_F_GUEST_UDP_TUNNEL_GSO negotiated)
+ le16 outer_nh_offset; /* Only if VIRTIO_NET_F_OUT_NET_HEADER
negotiated */
};
\end{lstlisting}

device-types/net/description.tex +0 -0
@@ -633,6 +638,11 @@ \subsubsection{Packet

Transmission}\label{sec:Device Types / Network Device / De \item The driver can send a completely checksummed packet. In this case,
\field{flags} will be zero, and \field{gso_type} will be
VIRTIO_NET_HDR_GSO_NONE.

Quoted 1 line

+\item When VIRTIO_NET_F_OUT_NET_HEADER is negotiated, the driver

MAY

Quoted 14 lines

optionally provide
+ the \field{outer_nh_offset} value. A nonzero value of
\field{outer_nh_offset} indicates
+ a valid outer network header with in the packet, and specifies the
+ offset in
bytes from
+ the beginning of the packet. Otherwise \field{outer_nh_offset}
+ MUST not
be used.

device-types/net/description.tex +1 -0
+
 \item If the driver negotiated VIRTIO_NET_F_CSUM, it can skip
   checksumming the packet:
   \begin{itemize}
@@ -767,6 +777,11 @@ \subsubsection{Packet
Quoted 2 lines

Transmission}\label{sec:Device Types / Network Device / De \field{flags} to zero and SHOULD supply a fully checksummed packet to the

device.

device-types/net/description.tex \subsubsection{Packet Transmission}\label{sec:Device Types / Network Device / De · line 784

Quoted 6 lines
device-types/net/description.tex +5 -0
+If the VIRTIO_NET_F_OUT_NET_HEADER feature has been negotiated, the
+driver MAY set \field{outer_nh_offset} to indicate the start of the
+outer network header offset, if the packet contains a valid network header.
+Otherwise, \field{outer_nh_offset} is not used.
+

s/MAY be set to/MAY be set to non zero value/

Here and few more places below.

Quoted 6 lines

If VIRTIO_NET_F_HOST_TSO4 is negotiated, the driver MAY set
\field{gso_type} to VIRTIO_NET_HDR_GSO_TCPV4 to request TCPv4 segmentation, otherwise the driver MUST NOT set @@ -948,6 +963,11 @@ \subsubsection{Packet Transmission}\label{sec:Device Types / Network Device / De If VIRTIO_NET_HDR_F_NEEDS_CSUM bit in \field{flags} is not set, the device MUST NOT use the \field{csum_start} and

\field{csum_offset}.

Quoted 15 lines
device-types/net/description.tex +5 -0
+If the VIRTIO_NET_F_OUT_NET_HEADER feature has been negotiated, and
+\field{outer_nh_offset} is not zero, the device MAY use
+\field{outer_nh_offset} as the outer network header offset.
+Otherwise, device MUST NOT use the \field{outer_nh_offset}.
+
 If one of the VIRTIO_NET_F_HOST_TSO4, TSO6, USO or UFO options have

been negotiated:
\begin{itemize}

device-types/net/description.tex +0 -0
@@ -1102,6 +1122,9 @@ \subsubsection{Processing of Incoming

Packets}\label{sec:Device Types / Network
both the outer UDP checksum and the inner transport checksum have been validated, otherwise only one level of checksums (the outer one in case of tunnels) has been validated.
+\item If the VIRTIO_NET_F_OUT_NET_HEADER has been negotiated, and if

device-types/net/description.tex \subsubsection{Processing of Incoming Packets}\label{sec:Device Types / Network · line 1130

Quoted 7 lines
device-types/net/description.tex +4 -0
+the packet
+  contains a valid network header, \field{outer_nh_offset} MAY be set
+to indicate the
+  outer network header offset in packet.
 \end{enumerate}
 
 Additionally, VIRTIO_NET_F_GUEST_CSUM, TSO4, TSO6, UDP, UDP_TUNNEL

@@

Quoted 9 lines

-1215,6 +1238,10 @@ \subsubsection{Processing of Incoming Packets}\label{sec:Device Types / Network device MUST set the VIRTIO_NET_HDR_GSO_ECN bit in \field{gso_type}.

+If VIRTIO_NET_F_OUT_NET_HEADER has been negotiated, the device MAY set

device-types/net/description.tex +3 -0
+the \field{outer_nh_offset} to indicate outer network header offset,
+if packet contains a valid network header. Otherwise, the device MUST
+not use
Quoted 4 lines

\field{outer_nh_offset}.
+
If the VIRTIO_NET_F_GUEST_CSUM feature has been negotiated, the
device MAY set the VIRTIO_NET_HDR_F_NEEDS_CSUM bit in \field{flags}, if

so:

Quoted 14 lines
device-types/net/description.tex +0 -0
@@ -1302,6 +1329,10 @@ \subsubsection{Processing of Incoming

Packets}\label{sec:Device Types / Network

The driver MUST ignore \field{flag} bits that it does not recognize.

device-types/net/description.tex +4 -0
+If VIRTIO_NET_F_OUT_NET_HEADER has been negotiated, and if
+\field{outer_nh_offset} is nonzero, the driver MAY use
+\field{outer_nh_offset} as outer network header offset. Otherwise,
+the

driver MUST not use the \field{outer_nh_offset}.
+
If VIRTIO_NET_HDR_F_NEEDS_CSUM bit in \field{flags} is not set or if
VIRTIO_NET_HDR_F_RSC_INFO bit \field{flags} is set, the driver MUST NOT use the \field{csum_start} and \field{csum_offset}.

Quoted 8 lines
Quoted 2 lines

--
2.43.0

Can you please also describe the description and requirements for the virtio_net_hdr size to indicate that when this feature is negotiated, the vnet hdr size is all the way upto this field.
i.e. when HASH_REPORT is not negotiated and NET_HEADER is negotiated, this field does not move up etc.

Could you please add more details ?
NET_HEADER can be an independent feature without requiring HASH_REPORT or the recently added feature UDP_TUNNEL_GSO. Should we document virtio_net_hdr size for each such combination ?

Quoted 3 lines

Please also add 6 bytes of reserved to have the vnet hdr aligned to 64-bit address.

#3 lore raw
Quoted 207 lines

From: Shiva Shankar Kommula <[email protected]>
Sent: Monday, March 24, 2025 11:20 PM

Quoted 55 lines

-----Original Message-----
From: Parav Pandit <[email protected]>
Sent: Sunday, March 9, 2025 10:02 PM
To: Shiva Shankar Kommula <[email protected]>; [email protected]; [email protected]; [email protected]; [email protected]
Cc: Nithin Kumar Dabilpuram <[email protected]>; Jerin Jacob <[email protected]>; Rahul Bhansali <[email protected]>
Subject: [EXTERNAL] RE: [PATCH v4] virtio-net: Introduce a new field to indicate outer network header offset

Quoted 10 lines

From: Kommula Shiva Shankar <kshankar@ marvell. com> > Sent:
Tuesday, March 4, 2025 1: 30 PM > > This patch introduces a new field in the virtio_net_header called s/virtio_net_header/virtio_net_hdr > outer_nh_offset, along

From: Kommula Shiva Shankar <[email protected]>
Sent: Tuesday, March 4, 2025 1:30 PM

This patch introduces a new field in the virtio_net_header called

s/virtio_net_header/virtio_net_hdr

Quoted 7 lines

outer_nh_offset, along with a new net device feature, VIRTIO_NET_F_OUT_NET_HEADER.

Currently, drivers lack a dedicated field to signal the start of the network header to the device when performing checksum offload and segmentation offload. This requires reading packet, which significantly affects performance in data path.

s/This requires reading packet/This requires the device to read the packet in data path ..

Quoted 6 lines

Additionally, some hardware implementations require knowledge of the outer
L3 offset (aka L2 length) for inline IPsec hardware acceleration.

To address this limitation, we propose to introduce a new field in the virtio_net_header called outer_nh_offset

s/virtio_net_header/virtio_net_hdr

Quoted 3 lines

The outer_nh_offset represents the start byte offset of the outer network header from the beginning of the packet

Add full stop at the end of the sentence.

Quoted 1 line

[..]

Quoted 7 lines
---
 device-types/net/description.tex | 31
+++++++++++++++++++++++++++++++
 1 file changed, 31 insertions(+)
Quoted 8 lines
Quoted 8 lines
device- renamed from device-types/net/description.tex +0 -0

types/net/description.tex index 8d30673..ca1242f 100644

device-types/net/description.tex +0 -0
@@ -136,6 +136,10 @@ \subsection{Feature bits}\label{sec:Device

Types / Network Device / Feature bits

\item[VIRTIO_NET_F_HOST_UDP_TUNNEL_GSO_CSUM (68)] Device

handles

Quoted 4 lines
Quoted 4 lines

packets
carried by a UDP tunnel with partial csum for the outer header.
+
+\item[VIRTIO_NET_F_OUT_NET_HEADER(69)] Driver can provide the

start

Quoted 13 lines

device-types/net/description.tex

Quoted 2 lines
device-types/net/description.tex +2 -0
+of \field{outer_nh_offset}  value. Device gains advantage by not
+reading packet to calculate outer network  header offset

Please add the full stop at the end of above sentence.

Quoted 9 lines

\end{description}

\subsubsection{Feature bit requirements}\label{sec:Device Types /
Network Device / Feature bits / Feature bit requirements} @@ -599,6 +603,7 @@ \subsection{Device Operation}\label{sec:Device Types / Network Device / Device O
le16 padding_reserved; (Only if VIRTIO_NET_F_HASH_REPORT
negotiated)
le16 outer_th_offset (Only if

VIRTIO_NET_F_HOST_UDP_TUNNEL_GSO

Quoted 2 lines
Quoted 2 lines

or VIRTIO_NET_F_GUEST_UDP_TUNNEL_GSO negotiated)
le16 inner_nh_offset; (Only if

VIRTIO_NET_F_HOST_UDP_TUNNEL_GSO

Quoted 24 lines
Quoted 12 lines

or VIRTIO_NET_F_GUEST_UDP_TUNNEL_GSO negotiated)
+ le16 outer_nh_offset; /* Only if VIRTIO_NET_F_OUT_NET_HEADER
negotiated */
};
\end{lstlisting}

device-types/net/description.tex +0 -0
@@ -633,6 +638,11 @@ \subsubsection{Packet

Transmission}\label{sec:Device Types / Network Device / De \item The driver can send a completely checksummed packet. In this case,
\field{flags} will be zero, and \field{gso_type} will be
VIRTIO_NET_HDR_GSO_NONE.

Quoted 1 line

+\item When VIRTIO_NET_F_OUT_NET_HEADER is negotiated, the driver

MAY

Quoted 10 lines

optionally provide
+ the \field{outer_nh_offset} value. A nonzero value of
\field{outer_nh_offset} indicates
+ a valid outer network header with in the packet, and specifies
+ the offset in
bytes from
+ the beginning of the packet. Otherwise \field{outer_nh_offset}
+ MUST not
be used.
+

Quoted 10 lines
Quoted 7 lines

\item If the driver negotiated VIRTIO_NET_F_CSUM, it can skip
checksumming the packet:

   \begin{itemize}
device-types/net/description.tex +0 -0
@@ -767,6 +777,11 @@ \subsubsection{Packet

Transmission}\label{sec:Device Types / Network Device / De \field{flags} to zero and SHOULD supply a fully checksummed packet to the

device.

Quoted 2 lines

+If the VIRTIO_NET_F_OUT_NET_HEADER feature has been negotiated,

the

Quoted 2 lines

device-types/net/description.tex

Quoted 2 lines
device-types/net/description.tex +2 -0
+driver MAY set \field{outer_nh_offset} to indicate the start of the
+outer network header offset, if the packet contains a valid network

header.

Quoted 9 lines

device-types/net/description.tex \subsubsection{Packet Transmission}\label{sec:Device Types / Network Device / De · line 784

Quoted 2 lines
device-types/net/description.tex +2 -0
+Otherwise, \field{outer_nh_offset} is not used.
+

s/MAY be set to/MAY be set to non zero value/

Here and few more places below.

Quoted 3 lines

If VIRTIO_NET_F_HOST_TSO4 is negotiated, the driver MAY set
\field{gso_type} to VIRTIO_NET_HDR_GSO_TCPV4 to request TCPv4 segmentation, otherwise the driver MUST NOT set @@ -948,6 +963,11

@@

Quoted 6 lines
Quoted 3 lines

\subsubsection{Packet Transmission}\label{sec:Device Types / Network Device / De If VIRTIO_NET_HDR_F_NEEDS_CSUM bit in \field{flags} is not set, the device MUST NOT use the \field{csum_start} and

\field{csum_offset}.

Quoted 2 lines

+If the VIRTIO_NET_F_OUT_NET_HEADER feature has been negotiated,

and

Quoted 11 lines
Quoted 11 lines

+\field{outer_nh_offset} is not zero, the device MAY use +\field{outer_nh_offset} as the outer network header offset.

device-types/net/description.tex +2 -0
+Otherwise, device MUST NOT use the \field{outer_nh_offset}.
+
 If one of the VIRTIO_NET_F_HOST_TSO4, TSO6, USO or UFO options have

been negotiated:
\begin{itemize}

device-types/net/description.tex +0 -0
@@ -1102,6 +1122,9 @@ \subsubsection{Processing of Incoming

Packets}\label{sec:Device Types / Network
both the outer UDP checksum and the inner transport checksum have been validated, otherwise only one level of checksums (the outer

one

Quoted 9 lines

device-types/net/description.tex

Quoted 9 lines

in case of tunnels) has been validated.
+\item If the VIRTIO_NET_F_OUT_NET_HEADER has been negotiated, and

device-types/net/description.tex +4 -0
+if the packet
+  contains a valid network header, \field{outer_nh_offset} MAY be
+set to indicate the
+  outer network header offset in packet.
 \end{enumerate}
 
 Additionally, VIRTIO_NET_F_GUEST_CSUM, TSO4, TSO6, UDP,

UDP_TUNNEL

Quoted 6 lines

@@

Quoted 5 lines

-1215,6 +1238,10 @@ \subsubsection{Processing of Incoming Packets}\label{sec:Device Types / Network device MUST set the VIRTIO_NET_HDR_GSO_ECN bit in \field{gso_type}.

+If VIRTIO_NET_F_OUT_NET_HEADER has been negotiated, the device

MAY

Quoted 24 lines
Quoted 9 lines

set

device-types/net/description.tex +3 -0
+the \field{outer_nh_offset} to indicate outer network header
+offset, if packet contains a valid network header. Otherwise, the
+device MUST not use

\field{outer_nh_offset}.
+
If the VIRTIO_NET_F_GUEST_CSUM feature has been negotiated, the
device MAY set the VIRTIO_NET_HDR_F_NEEDS_CSUM bit in \field{flags}, if

so:

Quoted 14 lines
device-types/net/description.tex +0 -0
@@ -1302,6 +1329,10 @@ \subsubsection{Processing of Incoming

Packets}\label{sec:Device Types / Network

The driver MUST ignore \field{flag} bits that it does not recognize.

device-types/net/description.tex +4 -0
+If VIRTIO_NET_F_OUT_NET_HEADER has been negotiated, and if
+\field{outer_nh_offset} is nonzero, the driver MAY use
+\field{outer_nh_offset} as outer network header offset. Otherwise,
+the

driver MUST not use the \field{outer_nh_offset}.
+
If VIRTIO_NET_HDR_F_NEEDS_CSUM bit in \field{flags} is not set or
if VIRTIO_NET_HDR_F_RSC_INFO bit \field{flags} is set, the driver MUST NOT use the \field{csum_start} and \field{csum_offset}.

Quoted 8 lines
Quoted 2 lines

--
2.43.0

Can you please also describe the description and requirements for the virtio_net_hdr size to indicate that when this feature is negotiated, the vnet hdr size is all the way upto this field.
i.e. when HASH_REPORT is not negotiated and NET_HEADER is negotiated, this field does not move up etc.

Quoted 5 lines

Could you please add more details ?

NET_HEADER can be an independent feature without requiring HASH_REPORT or the recently added feature UDP_TUNNEL_GSO. Should we document virtio_net_hdr size for each such combination ?

NET_HEADER can be independent feature.
However, the field we are adding in the structure is at the end.
So when HASH_REPORT and UDP_TUNNEL_GSO is not negotiated, but OUT_NET_HDR is negotiated,
The virtio_net_hdr size is still 24 bytes (upto including field inner_nh_offset) + 8 bytes of your extension. I considered 8B extension to align to 64-bit width.

So yeah, we don’t need to write all combinations.
Just write something like,

When OUT_NET_HDR is negotiated, vnet header size is up to including field outer_nh_offset.

Quoted 3 lines
Quoted 3 lines

Please also add 6 bytes of reserved to have the vnet hdr aligned to 64-bit address.