Skip to contentSkip to conversation

virtio-net: Introduce a new field to indicate outer network header offset

PATCH Superseded Kommula Shiva Shankar 3 participants 10 messages Feb 17, 2025 – Feb 28, 2025

v3 of 5 Superseded Latest is v5 →

lore
Share & tools
lore
#0 lore raw

PATCH v3 1/1 virtio-net: Introduce a new field to indicate outer network header offset

This patch introduces a new field in the virtio_net_header called out_nh_offset, along with a new net device feature, VIRTIO_NET_F_OUT_NET_HEADER.

Currently, drivers lack a dedicated field to signal the start of the network header to the device when performing checksum offload and segmentation offload. This requires reading packet, which significantly affects performance in datapath. Additionally, some hardware implementations requrie knowledege of the outer L3 offset (aka L2 length) for inline IPSec hardware acceleration.

To address this limitation, we propose splitting the csum_offset field into two 8-bit fields named csum_offset and out_nh_offset. The csum_offset indicates the offset value from the csum_start and may not exceed 256B bits(2^8) for protocols that use a 16-bit one's complement checksum

Following table lists such protocols and their checksum offset fields within their headers

+-----+--------+
|Proto|csum_off|
+-----+--------+
| IPV4| 10B    |
| ICMP| 2B     |
| IGMP| 2B     |
|  TCP| 16B    |
|  UDP| 6B     |
+-----+--------+

The out_nh_offset represents the start byte offset of the outer network header from the beginning of the packet

This issue was briefly discussed on the mailing list in a different thread, which can be found here
https://lore.kernel.org/all/DM4PR18MB4269FAAC3CFC7E57E25DFBD2DF8B2@DM4PR18MB4269.namprd18.prod.outlook.com/

v2 -> v3:
- Rebase to virtio-1.4
- Addressed pending review commnets related to wording.

v2 - https://lore.kernel.org/all/[email protected]/

v1 -> v2:
- explicitly state that the out_nh_offset can be set only when a valid network header is present.
- updated out_nh_offset usage in the RX direction.
- minor word cleanup.

v1 - https://lore.kernel.org/virtio-comment/[email protected]/

---
 device-types/net/description.tex | 37 ++++++++++++++++++++++++++++++++
 1 file changed, 37 insertions(+)
device-types/net/description.tex +37 -0
index b546cd7..2a4aa66 100644
@@ -136,6 +136,10 @@ \subsection{Feature bits}\label{sec:Device Types / Network Device / Feature bits
 
 \item[VIRTIO_NET_F_HOST_UDP_TUNNEL_GSO_CSUM (68)] Device handles packets
   carried by a UDP tunnel with partial csum for the outer header.
+
+\item[VIRTIO_NET_F_OUT_NET_HEADER(69)] Driver can provide the start of \field{out_nh_offset}
+ value. Device gains advantage by not reading packet to calculate outer network
+ header offset
 \end{description}
 
 \subsubsection{Feature bit requirements}\label{sec:Device Types / Network Device / Feature bits / Feature bit requirements}
@@ -592,6 +596,13 @@ \subsection{Device Operation}\label{sec:Device Types / Network Device / Device O
         le16 hdr_len;
         le16 gso_size;
         le16 csum_start;
+        union {
+                le16 csum_offset;
+                struct {        /* Only if VIRTIO_NET_F_OUT_NET_HEADER negotiated */
+                        le8 csum_offset;
+                        le8 out_nh_offset;
+                };
+        };
         le16 csum_offset;
         le16 num_buffers;
         le32 hash_value;        (Only if VIRTIO_NET_F_HASH_REPORT negotiated)
@@ -633,6 +644,11 @@ \subsubsection{Packet Transmission}\label{sec:Device Types / Network Device / De
 \item The driver can send a completely checksummed packet.  In this case,
   \field{flags} will be zero, and \field{gso_type} will be VIRTIO_NET_HDR_GSO_NONE.
 
+\item When VIRTIO_NET_F_OUT_NET_HEADER is negotiated, the driver MAY optionally provide
+   the \field{out_nh_offset} value. A nonzero value of \field{out_nh_offset} indicates
+   a valid outer network header with in the packet, and specifies the offset in bytes from
+   the beginning of the packet. Otherwise \field{out_nh_offset} MUST not be used.
+
 \item If the driver negotiated VIRTIO_NET_F_CSUM, it can skip
   checksumming the packet:
   \begin{itemize}
@@ -767,6 +783,11 @@ \subsubsection{Packet Transmission}\label{sec:Device Types / Network Device / De
 \field{flags} to zero and SHOULD supply a fully checksummed
 packet to the device.
 
+If the VIRTIO_NET_F_OUT_NET_HEADER feature has been negotiated,
+the driver MAY set \field{out_nh_offset} to indicate the start of the
+outer network header offset, if the packet contains a valid network header.
+Otherwise, \field{out_nh_offset} is not used.
+
 If VIRTIO_NET_F_HOST_TSO4 is negotiated, the driver MAY set
 \field{gso_type} to VIRTIO_NET_HDR_GSO_TCPV4 to request TCPv4
 segmentation, otherwise the driver MUST NOT set
@@ -948,6 +969,11 @@ \subsubsection{Packet Transmission}\label{sec:Device Types / Network Device / De
 If VIRTIO_NET_HDR_F_NEEDS_CSUM bit in \field{flags} is not set, the
 device MUST NOT use the \field{csum_start} and \field{csum_offset}.
 
+If the VIRTIO_NET_F_OUT_NET_HEADER feature has been negotiated,
+and \field{out_nh_offset} is not zero, the device MAY use \field{out_nh_offset}
+as the outer network header offset. Otherwise, device MUST NOT use
+the \field{out_nh_offset}.
+
 If one of the VIRTIO_NET_F_HOST_TSO4, TSO6, USO or UFO options have
 been negotiated:
 \begin{itemize}
@@ -1102,6 +1128,9 @@ \subsubsection{Processing of Incoming Packets}\label{sec:Device Types / Network
   both the outer UDP checksum and the inner transport checksum
   have been validated, otherwise only one level of checksums (the outer one
   in case of tunnels) has been validated.
+\item If the VIRTIO_NET_F_OUT_NET_HEADER has been negotiated, and if the packet
+  contains a valid network header, \field{out_nh_offset} MAY be set to indicate the
+  outer network header offset in packet.
 \end{enumerate}
 
 Additionally, VIRTIO_NET_F_GUEST_CSUM, TSO4, TSO6, UDP, UDP_TUNNEL
@@ -1215,6 +1244,10 @@ \subsubsection{Processing of Incoming Packets}\label{sec:Device Types / Network
 device MUST set the VIRTIO_NET_HDR_GSO_ECN bit in
 \field{gso_type}.
 
+If VIRTIO_NET_F_OUT_NET_HEADER has been negotiated, the device MAY
+set the \field{out_nh_offset} to indicate outer network header offset, if packet contains
+a valid network header. Otherwise, the device MUST not use \field{out_nh_offset}.
+
 If the VIRTIO_NET_F_GUEST_CSUM feature has been negotiated, the
 device MAY set the VIRTIO_NET_HDR_F_NEEDS_CSUM bit in
 \field{flags}, if so:
@@ -1302,6 +1335,10 @@ \subsubsection{Processing of Incoming Packets}\label{sec:Device Types / Network
 
 The driver MUST ignore \field{flag} bits that it does not recognize.
 
+If VIRTIO_NET_F_OUT_NET_HEADER has been negotiated, and if \field{out_nh_offset}
+is nonzero, the driver MAY use \field{out_nh_offset} as outer network header
+offset. Otherwise, the driver MUST not use the \field{out_nh_offset}.
+
 If VIRTIO_NET_HDR_F_NEEDS_CSUM bit in \field{flags} is not set or
 if VIRTIO_NET_HDR_F_RSC_INFO bit \field{flags} is set, the
 driver MUST NOT use the \field{csum_start} and \field{csum_offset}.
Signature · 1 line
2.43.0
#1 lore raw
Quoted 61 lines · Kommula Shiva Shankar

This patch introduces a new field in the virtio_net_header called out_nh_offset, along with a new net device feature, VIRTIO_NET_F_OUT_NET_HEADER.

Currently, drivers lack a dedicated field to signal the start of the network header to the device when performing checksum offload and segmentation offload. This requires reading packet, which significantly affects performance in datapath. Additionally, some hardware implementations requrie knowledege of the outer L3 offset (aka L2 length) for inline IPSec hardware acceleration.

To address this limitation, we propose splitting the csum_offset field into two 8-bit fields named csum_offset and out_nh_offset. The csum_offset indicates the offset value from the csum_start and may not exceed 256B bits(2^8) for protocols that use a 16-bit one's complement checksum

Following table lists such protocols and their checksum offset fields within their headers

+-----+--------+
|Proto|csum_off|
+-----+--------+
| IPV4| 10B    |
| ICMP| 2B     |
| IGMP| 2B     |
|  TCP| 16B    |
|  UDP| 6B     |
+-----+--------+

The out_nh_offset represents the start byte offset of the outer network header from the beginning of the packet

This issue was briefly discussed on the mailing list in a different thread, which can be found here
https://lore.kernel.org/all/DM4PR18MB4269FAAC3CFC7E57E25DFBD2DF8B2@DM4PR18MB4269.namprd18.prod.outlook.com/

v2 -> v3:
- Rebase to virtio-1.4
- Addressed pending review commnets related to wording.

v2 - https://lore.kernel.org/all/[email protected]/

v1 -> v2:
- explicitly state that the out_nh_offset can be set only when a valid network header is present.
- updated out_nh_offset usage in the RX direction.
- minor word cleanup.

v1 - https://lore.kernel.org/virtio-comment/[email protected]/

---
 device-types/net/description.tex | 37 ++++++++++++++++++++++++++++++++
 1 file changed, 37 insertions(+)
device-types/net/description.tex +4 -0
index b546cd7..2a4aa66 100644
@@ -136,6 +136,10 @@ \subsection{Feature bits}\label{sec:Device Types / Network Device / Feature bits
 
 \item[VIRTIO_NET_F_HOST_UDP_TUNNEL_GSO_CSUM (68)] Device handles packets
   carried by a UDP tunnel with partial csum for the outer header.
+
+\item[VIRTIO_NET_F_OUT_NET_HEADER(69)] Driver can provide the start of \field{out_nh_offset}
+ value. Device gains advantage by not reading packet to calculate outer network
+ header offset
 \end{description}
 
 \subsubsection{Feature bit requirements}\label{sec:Device Types / Network Device / Feature bits / Feature bit requirements}

device-types/net/description.tex \subsection{Device Operation}\label{sec:Device Types / Network Device / Device O · line 603

Quoted 8 lines
device-types/net/description.tex +5 -0
@@ -592,6 +596,13 @@ \subsection{Device Operation}\label{sec:Device Types / Network Device / Device O
         le16 hdr_len;
         le16 gso_size;
         le16 csum_start;
+        union {
+                le16 csum_offset;
+                struct {        /* Only if VIRTIO_NET_F_OUT_NET_HEADER negotiated */
+                        le8 csum_offset;
+                        le8 out_nh_offset;

Need some explanation why reusing csum_offset is better than introducing new fields.

device-types/net/description.tex \subsection{Device Operation}\label{sec:Device Types / Network Device / Device O · line 606

Quoted 3 lines
device-types/net/description.tex +2 -0
+                };
+        };
         le16 csum_offset;

It seems like it needs to be removed.

device-types/net/description.tex \subsection{Device Operation}\label{sec:Device Types / Network Device / Device O · line 608

Quoted 2 lines
device-types/net/description.tex +0 -0
         le16 num_buffers;
         le32 hash_value;        (Only if VIRTIO_NET_F_HASH_REPORT negotiated)

Thanks

#2 lore raw
Quoted 131 lines

-----Original Message-----
From: Jason Wang <[email protected]>
Sent: Wednesday, February 19, 2025 9:00 AM
To: Shiva Shankar Kommula <[email protected]>
Cc: [email protected]; [email protected]; [email protected]; virtio- [email protected]; Nithin Kumar Dabilpuram <[email protected]>; Jerin Jacob <[email protected]>; Rahul Bhansali <[email protected]>
Subject: [EXTERNAL] Re: [PATCH v3] virtio-net: Introduce a new field to indicate outer network header offset

On Tue, Feb 18, 2025 at 1: 25 AM Kommula Shiva Shankar <kshankar@ marvell. com> wrote: > > This patch introduces a new field in the virtio_net_header called out_nh_offset, along with a new net device feature, VIRTIO_NET_F_OUT_NET_HEADER.

Quoted 2 lines · Kommula Shiva Shankar

This patch introduces a new field in the virtio_net_header called

out_nh_offset, along with a new net device feature, VIRTIO_NET_F_OUT_NET_HEADER.

Quoted 3 lines

Currently, drivers lack a dedicated field to signal the start of the network header to the device when performing checksum offload and

segmentation offload. This requires reading packet, which significantly affects performance in datapath.

Quoted 1 line

Additionally, some hardware implementations requrie knowledege of the

outer L3 offset (aka L2 length) for inline IPSec hardware acceleration.

Quoted 2 lines

To address this limitation, we propose splitting the csum_offset field into

two 8-bit fields named csum_offset and out_nh_offset.

Quoted 17 lines

The csum_offset indicates the offset value from the csum_start and may not exceed 256B bits(2^8) for protocols that use a 16-bit one's complement checksum

Following table lists such protocols and their checksum offset fields within their headers

+-----+--------+
|Proto|csum_off|
+-----+--------+
| IPV4| 10B    |
| ICMP| 2B     |
| IGMP| 2B     |
|  TCP| 16B    |
|  UDP| 6B     |
+-----+--------+
Quoted 7 lines

The out_nh_offset represents the start byte offset of the outer network header from the beginning of the packet

This issue was briefly discussed on the mailing list in a different thread, which can be found here
https://urldefense.proofpoint.com/v2/url?u=https-3A__lore.kernel.org_a
ll_DM4PR18MB4269FAAC3CFC7E57E25DFBD2DF8B2-

40DM4PR18MB4269.namprd18.pro

Quoted 1 line

d.outlook.com_&d=DwIFaQ&c=nKjWec2b6R0mOyPaz7xtfQ&r=XP728rR_yI3b CCyFq0M

Quoted 1 line

EWPS2176D_oSryc7cD-

0TRPs&m=jajIHiJb18aASNFHlkOtvANrA_06lDHp6zG9AEFU8u_

Quoted 1 line

nHyksxDbHI3UT0OFedi-b&s=MR9AX0Md9PFbZjQM-

bwJAmO13ZwETRZJCm9qFIYDy6E&e=

Quoted 8 lines

v2 -> v3:
- Rebase to virtio-1.4
- Addressed pending review commnets related to wording.

v2 -
https://urldefense.proofpoint.com/v2/url?u=https-3A__lore.kernel.org_a
ll_20250128142152.3662988-2D1-2Dkshankar-

40marvell.com_&d=DwIFaQ&c=nKj

Quoted 1 line

Wec2b6R0mOyPaz7xtfQ&r=XP728rR_yI3bCCyFq0MEWPS2176D_oSryc7cD-

0TRPs&m=ja

Quoted 1 line

jIHiJb18aASNFHlkOtvANrA_06lDHp6zG9AEFU8u_nHyksxDbHI3UT0OFedi-

b&s=jvYZB

Quoted 4 lines

QtOwWyydI5lFvCFz4hF94jDmU70nROTU44Ma0M&e=

v1 -> v2:
- explicitly state that the out_nh_offset can be set only when a valid

network header is present.

Quoted 6 lines

- updated out_nh_offset usage in the RX direction.
- minor word cleanup.

v1 -
https://urldefense.proofpoint.com/v2/url?u=https-3A__lore.kernel.org_v
irtio-2Dcomment_20250114171636.3175670-2D1-2Dkshankar-

40marvell.com_&d

Quoted 1 line

=DwIFaQ&c=nKjWec2b6R0mOyPaz7xtfQ&r=XP728rR_yI3bCCyFq0MEWPS217 6D_oSryc7

Quoted 1 line

cD-

0TRPs&m=jajIHiJb18aASNFHlkOtvANrA_06lDHp6zG9AEFU8u_nHyksxDbHI3UT 0OF

Quoted 16 lines

edi-b&s=rzyoQrmZTznG5V9N03u7bpPbG1uQC0WPFmypJJKS9r8&e=

---
 device-types/net/description.tex | 37
++++++++++++++++++++++++++++++++
 1 file changed, 37 insertions(+)
device-types/net/description.tex +0 -0
index b546cd7..2a4aa66 100644
@@ -136,6 +136,10 @@ \subsection{Feature bits}\label{sec:Device Types

/ Network Device / Feature bits

Quoted 1 line

\item[VIRTIO_NET_F_HOST_UDP_TUNNEL_GSO_CSUM (68)] Device

handles packets

Quoted 9 lines

carried by a UDP tunnel with partial csum for the outer header.

device-types/net/description.tex +4 -0
+
+\item[VIRTIO_NET_F_OUT_NET_HEADER(69)] Driver can provide the start
+of \field{out_nh_offset}  value. Device gains advantage by not
+reading packet to calculate outer network  header offset
 \end{description}
 
 \subsubsection{Feature bit requirements}\label{sec:Device Types /

Network Device / Feature bits / Feature bit requirements} @@ -592,6

+596,13 @@ \subsection{Device Operation}\label{sec:Device Types / Network Device / Device O

device-types/net/description.tex \subsection{Device Operation}\label{sec:Device Types / Network Device / Device O

Quoted 6 lines
         le16 hdr_len;
         le16 gso_size;
         le16 csum_start;
device-types/net/description.tex +3 -0
+        union {
+                le16 csum_offset;
+                struct {        /* Only if VIRTIO_NET_F_OUT_NET_HEADER
Quoted 6 lines

negotiated */

device-types/net/description.tex \subsection{Device Operation}\label{sec:Device Types / Network Device / Device O · line 603

Quoted 2 lines
device-types/net/description.tex +2 -0
+                        le8 csum_offset;
+                        le8 out_nh_offset;

Need some explanation why reusing csum_offset is better than introducing new fields.

Our objective is to preserve the current size of the virtio_net_hdr structure, to ensure compatibility with the existing header size. (unlike other features such as hash reporting). Given that the csum_offset represents an offset within the header and may not exceed 2^8, we propose utilizing the existing space to accommodate this newly introduced field.

Would it be more prudent to add a new field for this feature ? Do you foresee any issues with this approach ?

Quoted 6 lines

device-types/net/description.tex \subsection{Device Operation}\label{sec:Device Types / Network Device / Device O · line 606

Quoted 4 lines
device-types/net/description.tex +2 -0
+                };
+        };
         le16 csum_offset;

It seems like it needs to be removed.

ACK .
Inadvertently included in version 3. Will fix up and send version4

Quoted 6 lines
Quoted 3 lines

le16 num_buffers;
le32 hash_value; (Only if VIRTIO_NET_F_HASH_REPORT

negotiated)

Thanks

#3 lore raw
Quoted 136 lines · Shiva Shankar Kommula
Quoted 136 lines

-----Original Message-----
From: Jason Wang <[email protected]>
Sent: Wednesday, February 19, 2025 9:00 AM
To: Shiva Shankar Kommula <[email protected]>
Cc: [email protected]; [email protected]; [email protected]; virtio- [email protected]; Nithin Kumar Dabilpuram <[email protected]>; Jerin Jacob <[email protected]>; Rahul Bhansali <[email protected]>
Subject: [EXTERNAL] Re: [PATCH v3] virtio-net: Introduce a new field to indicate outer network header offset

On Tue, Feb 18, 2025 at 1: 25 AM Kommula Shiva Shankar <kshankar@ marvell. com> wrote: > > This patch introduces a new field in the virtio_net_header called out_nh_offset, along with a new net device feature, VIRTIO_NET_F_OUT_NET_HEADER.

Quoted 2 lines · Kommula Shiva Shankar

This patch introduces a new field in the virtio_net_header called

out_nh_offset, along with a new net device feature, VIRTIO_NET_F_OUT_NET_HEADER.

Quoted 3 lines

Currently, drivers lack a dedicated field to signal the start of the network header to the device when performing checksum offload and

segmentation offload. This requires reading packet, which significantly affects performance in datapath.

Quoted 1 line

Additionally, some hardware implementations requrie knowledege of the

outer L3 offset (aka L2 length) for inline IPSec hardware acceleration.

Quoted 2 lines

To address this limitation, we propose splitting the csum_offset field into

two 8-bit fields named csum_offset and out_nh_offset.

Quoted 17 lines

The csum_offset indicates the offset value from the csum_start and may not exceed 256B bits(2^8) for protocols that use a 16-bit one's complement checksum

Following table lists such protocols and their checksum offset fields within their headers

+-----+--------+
|Proto|csum_off|
+-----+--------+
| IPV4| 10B    |
| ICMP| 2B     |
| IGMP| 2B     |
|  TCP| 16B    |
|  UDP| 6B     |
+-----+--------+
Quoted 7 lines

The out_nh_offset represents the start byte offset of the outer network header from the beginning of the packet

This issue was briefly discussed on the mailing list in a different thread, which can be found here
https://urldefense.proofpoint.com/v2/url?u=https-3A__lore.kernel.org_a
ll_DM4PR18MB4269FAAC3CFC7E57E25DFBD2DF8B2-

40DM4PR18MB4269.namprd18.pro

Quoted 1 line

d.outlook.com_&d=DwIFaQ&c=nKjWec2b6R0mOyPaz7xtfQ&r=XP728rR_yI3b CCyFq0M

Quoted 1 line

EWPS2176D_oSryc7cD-

0TRPs&m=jajIHiJb18aASNFHlkOtvANrA_06lDHp6zG9AEFU8u_

Quoted 1 line

nHyksxDbHI3UT0OFedi-b&s=MR9AX0Md9PFbZjQM-

bwJAmO13ZwETRZJCm9qFIYDy6E&e=

Quoted 8 lines

v2 -> v3:
- Rebase to virtio-1.4
- Addressed pending review commnets related to wording.

v2 -
https://urldefense.proofpoint.com/v2/url?u=https-3A__lore.kernel.org_a
ll_20250128142152.3662988-2D1-2Dkshankar-

40marvell.com_&d=DwIFaQ&c=nKj

Quoted 1 line

Wec2b6R0mOyPaz7xtfQ&r=XP728rR_yI3bCCyFq0MEWPS2176D_oSryc7cD-

0TRPs&m=ja

Quoted 1 line

jIHiJb18aASNFHlkOtvANrA_06lDHp6zG9AEFU8u_nHyksxDbHI3UT0OFedi-

b&s=jvYZB

Quoted 4 lines

QtOwWyydI5lFvCFz4hF94jDmU70nROTU44Ma0M&e=

v1 -> v2:
- explicitly state that the out_nh_offset can be set only when a valid

network header is present.

Quoted 6 lines

- updated out_nh_offset usage in the RX direction.
- minor word cleanup.

v1 -
https://urldefense.proofpoint.com/v2/url?u=https-3A__lore.kernel.org_v
irtio-2Dcomment_20250114171636.3175670-2D1-2Dkshankar-

40marvell.com_&d

Quoted 1 line

=DwIFaQ&c=nKjWec2b6R0mOyPaz7xtfQ&r=XP728rR_yI3bCCyFq0MEWPS217 6D_oSryc7

Quoted 1 line

cD-

0TRPs&m=jajIHiJb18aASNFHlkOtvANrA_06lDHp6zG9AEFU8u_nHyksxDbHI3UT 0OF

Quoted 16 lines

edi-b&s=rzyoQrmZTznG5V9N03u7bpPbG1uQC0WPFmypJJKS9r8&e=

---
 device-types/net/description.tex | 37
++++++++++++++++++++++++++++++++
 1 file changed, 37 insertions(+)
device-types/net/description.tex +0 -0
index b546cd7..2a4aa66 100644
@@ -136,6 +136,10 @@ \subsection{Feature bits}\label{sec:Device Types

/ Network Device / Feature bits

Quoted 1 line

\item[VIRTIO_NET_F_HOST_UDP_TUNNEL_GSO_CSUM (68)] Device

handles packets

Quoted 9 lines

carried by a UDP tunnel with partial csum for the outer header.

device-types/net/description.tex +4 -0
+
+\item[VIRTIO_NET_F_OUT_NET_HEADER(69)] Driver can provide the start
+of \field{out_nh_offset}  value. Device gains advantage by not
+reading packet to calculate outer network  header offset
 \end{description}
 
 \subsubsection{Feature bit requirements}\label{sec:Device Types /

Network Device / Feature bits / Feature bit requirements} @@ -592,6

+596,13 @@ \subsection{Device Operation}\label{sec:Device Types / Network Device / Device O

device-types/net/description.tex \subsection{Device Operation}\label{sec:Device Types / Network Device / Device O

Quoted 6 lines
         le16 hdr_len;
         le16 gso_size;
         le16 csum_start;
device-types/net/description.tex +3 -0
+        union {
+                le16 csum_offset;
+                struct {        /* Only if VIRTIO_NET_F_OUT_NET_HEADER

negotiated */

Quoted 2 lines
device-types/net/description.tex +2 -0
+                        le8 csum_offset;
+                        le8 out_nh_offset;
Quoted 7 lines
Quoted 2 lines

Need some explanation why reusing csum_offset is better than introducing new fields.

Our objective is to preserve the current size of the virtio_net_hdr structure, to ensure compatibility with the existing header size. (unlike other features such as hash reporting). Given that the csum_offset represents an offset within the header and may not exceed 2^8, we propose utilizing the existing space to accommodate this newly introduced field.

Would it be more prudent to add a new field for this feature ? Do you foresee any issues with this approach ?

It might have some compatibility issue. For example in the current Linux implementation, csum_offset were 1:1 mapped to skb->csum_offset which is a u16.

I haven't checked all the protocols (especially the tunnels) but if one day there's a protocol that uses a csum_offset >= 256, it would be problematic.

Thanks

Quoted 15 lines
Quoted 6 lines

device-types/net/description.tex \subsection{Device Operation}\label{sec:Device Types / Network Device / Device O · line 606

Quoted 4 lines
device-types/net/description.tex +2 -0
+                };
+        };
         le16 csum_offset;

It seems like it needs to be removed.

ACK .
Inadvertently included in version 3. Will fix up and send version4

Quoted 6 lines
Quoted 3 lines

le16 num_buffers;
le32 hash_value; (Only if VIRTIO_NET_F_HASH_REPORT

negotiated)

Thanks

#4 lore raw
Quoted 180 lines

-----Original Message-----
From: Jason Wang <[email protected]>
Sent: Thursday, February 20, 2025 7:20 AM
To: Shiva Shankar Kommula <[email protected]>
Cc: [email protected]; [email protected]; [email protected]; virtio- [email protected]; Nithin Kumar Dabilpuram <[email protected]>; Jerin Jacob <[email protected]>; Rahul Bhansali <[email protected]>
Subject: Re: [EXTERNAL] Re: [PATCH v3] virtio-net: Introduce a new field to indicate outer network header offset

On Wed, Feb 19, 2025 at 2: 40 PM Shiva Shankar Kommula <kshankar@ marvell. com> wrote: > > > > > -----Original Message----- > >
From: Jason Wang <jasowang@ redhat. com> > > Sent: Wednesday, February 19, 2025

Quoted 57 lines · Shiva Shankar Kommula
Quoted 57 lines

-----Original Message-----
From: Jason Wang <[email protected]>
Sent: Wednesday, February 19, 2025 9:00 AM
To: Shiva Shankar Kommula <[email protected]>
Cc: [email protected]; [email protected]; [email protected]; virtio- [email protected]; Nithin Kumar Dabilpuram <[email protected]>; Jerin Jacob <[email protected]>; Rahul Bhansali <[email protected]>
Subject: [EXTERNAL] Re: [PATCH v3] virtio-net: Introduce a new field to indicate outer network header offset

On Tue, Feb 18, 2025 at 1: 25 AM Kommula Shiva Shankar <kshankar@ marvell. com> wrote: > > This patch introduces a new field in the virtio_net_header called out_nh_offset, along with a new net device feature, VIRTIO_NET_F_OUT_NET_HEADER.

Quoted 2 lines · Kommula Shiva Shankar

This patch introduces a new field in the virtio_net_header called

out_nh_offset, along with a new net device feature, VIRTIO_NET_F_OUT_NET_HEADER.

Quoted 4 lines

Currently, drivers lack a dedicated field to signal the start of the network header to the device when performing checksum offload and

segmentation offload. This requires reading packet, which significantly affects performance in datapath.

Quoted 2 lines

Additionally, some hardware implementations requrie knowledege of the

outer L3 offset (aka L2 length) for inline IPSec hardware acceleration.

Quoted 3 lines

To address this limitation, we propose splitting the csum_offset field into

two 8-bit fields named csum_offset and out_nh_offset.

Quoted 20 lines

The csum_offset indicates the offset value from the csum_start and may not exceed 256B bits(2^8) for protocols that use a 16-bit one's complement checksum

Following table lists such protocols and their checksum offset fields within their headers

+-----+--------+
|Proto|csum_off|
+-----+--------+
| IPV4| 10B    |
| ICMP| 2B     |
| IGMP| 2B     |
|  TCP| 16B    |
|  UDP| 6B     |
+-----+--------+

The out_nh_offset represents the start byte offset of the outer network header from the beginning of the packet

Quoted 9 lines
Quoted 4 lines
Quoted 4 lines

This issue was briefly discussed on the mailing list in a different thread, which can be found here
https://urldefense.proofpoint.com/v2/url?u=https-3A__lore.kernel.o
rg_a

<https://urldefense.proofpoint.com/v2/url?u=https-3A__lore.kernel.org_
a >> > > ll_DM4PR18MB4269FAAC3CFC7E57E25DFBD2DF8B2-

Quoted 3 lines

40DM4PR18MB4269.namprd18.pro

Quoted 1 line

d.outlook.com_&d=DwIFaQ&c=nKjWec2b6R0mOyPaz7xtfQ&r=XP728rR_yI3b

Quoted 10 lines
Quoted 10 lines

CCyFq0M

Quoted 1 line

EWPS2176D_oSryc7cD-

0TRPs&m=jajIHiJb18aASNFHlkOtvANrA_06lDHp6zG9AEFU8u_

Quoted 1 line

nHyksxDbHI3UT0OFedi-b&s=MR9AX0Md9PFbZjQM-

bwJAmO13ZwETRZJCm9qFIYDy6E&e=

Quoted 5 lines

v2 -> v3:
- Rebase to virtio-1.4
- Addressed pending review commnets related to wording.

Quoted 7 lines
Quoted 3 lines
Quoted 3 lines

v2 -
https://urldefense.proofpoint.com/v2/url?u=https-3A__lore.kernel.o
rg_a

<https://urldefense.proofpoint.com/v2/url?u=https-3A__lore.kernel.org_
a >> > > ll_20250128142152.3662988-2D1-2Dkshankar-

Quoted 2 lines

40marvell.com_&d=DwIFaQ&c=nKj

Quoted 1 line

Wec2b6R0mOyPaz7xtfQ&r=XP728rR_yI3bCCyFq0MEWPS2176D_oSryc7cD-

Quoted 2 lines
Quoted 2 lines

0TRPs&m=ja

Quoted 1 line

jIHiJb18aASNFHlkOtvANrA_06lDHp6zG9AEFU8u_nHyksxDbHI3UT0OFedi-

Quoted 7 lines
Quoted 7 lines

b&s=jvYZB

Quoted 5 lines

QtOwWyydI5lFvCFz4hF94jDmU70nROTU44Ma0M&e=

v1 -> v2:
- explicitly state that the out_nh_offset can be set only when a
valid

network header is present.

Quoted 11 lines
Quoted 6 lines
Quoted 6 lines

- updated out_nh_offset usage in the RX direction.
- minor word cleanup.

v1 -
https://urldefense.proofpoint.com/v2/url?u=https-3A__lore.kernel.o
rg_v

<https://urldefense.proofpoint.com/v2/url?u=https-3A__lore.kernel.org_
v >> > > irtio-2Dcomment_20250114171636.3175670-2D1-2Dkshankar-

Quoted 3 lines

40marvell.com_&d

Quoted 1 line

=DwIFaQ&c=nKjWec2b6R0mOyPaz7xtfQ&r=XP728rR_yI3bCCyFq0MEWPS217

Quoted 3 lines
Quoted 3 lines

6D_oSryc7

Quoted 1 line

cD-

0TRPs&m=jajIHiJb18aASNFHlkOtvANrA_06lDHp6zG9AEFU8u_nHyksxDbHI3UT

Quoted 38 lines
Quoted 38 lines

0OF

Quoted 16 lines

edi-b&s=rzyoQrmZTznG5V9N03u7bpPbG1uQC0WPFmypJJKS9r8&e=

---
 device-types/net/description.tex | 37
++++++++++++++++++++++++++++++++
 1 file changed, 37 insertions(+)
device-types/net/description.tex +0 -0
index b546cd7..2a4aa66 100644
@@ -136,6 +136,10 @@ \subsection{Feature bits}\label{sec:Device

Types / Network Device / Feature bits

Quoted 1 line

\item[VIRTIO_NET_F_HOST_UDP_TUNNEL_GSO_CSUM (68)] Device

handles packets

Quoted 10 lines

carried by a UDP tunnel with partial csum for the outer header.

device-types/net/description.tex +4 -0
+
+\item[VIRTIO_NET_F_OUT_NET_HEADER(69)] Driver can provide the
+start of \field{out_nh_offset}  value. Device gains advantage by
+not reading packet to calculate outer network  header offset
 \end{description}
 
 \subsubsection{Feature bit requirements}\label{sec:Device Types /

Network Device / Feature bits / Feature bit requirements} @@ -592,6

+596,13 @@ \subsection{Device Operation}\label{sec:Device Types / +Network
Device / Device O

Quoted 6 lines
         le16 hdr_len;
         le16 gso_size;
         le16 csum_start;
device-types/net/description.tex +3 -0
+        union {
+                le16 csum_offset;
+                struct {        /* Only if VIRTIO_NET_F_OUT_NET_HEADER
Quoted 7 lines
Quoted 6 lines

negotiated */

device-types/net/description.tex \subsection{Device Operation}\label{sec:Device Types / Network Device / Device O · line 603

Quoted 2 lines
device-types/net/description.tex +2 -0
+                        le8 csum_offset;
+                        le8 out_nh_offset;

Need some explanation why reusing csum_offset is better than introducing new fields.

Our objective is to preserve the current size of the virtio_net_hdr structure,

to ensure compatibility with the existing header size.

Quoted 2 lines

(unlike other features such as hash reporting). Given that the csum_offset represents an offset within the header and may not exceed

2^8, we propose utilizing the existing space to accommodate this newly introduced field.

Quoted 2 lines

Would it be more prudent to add a new field for this feature ? Do you

Quoted 9 lines

foresee any issues with this approach ?

It might have some compatibility issue. For example in the current Linux implementation, csum_offset were 1:1 mapped to skb->csum_offset which is a u16.

I haven't checked all the protocols (especially the tunnels) but if one day there's a protocol that uses a csum_offset >= 256, it would be problematic.

I agree with the 1:1 mapping between csum_offset and skb->checksum_offset. Given that the virtio net header size is already a scarce resource for adding new fields, additionally increasing it for every new feature might impact performance in the data path. Would it be fine to add a note stating that when the VIRTIO_NET_F_OUT_NET_HEADER feature is negotiated, checksum_offset cannot exceed 2^8 offset ?

Quoted 17 lines

Thanks

Quoted 15 lines
Quoted 6 lines

device-types/net/description.tex \subsection{Device Operation}\label{sec:Device Types / Network Device / Device O · line 606

Quoted 4 lines
device-types/net/description.tex +2 -0
+                };
+        };
         le16 csum_offset;

It seems like it needs to be removed.

ACK .
Inadvertently included in version 3. Will fix up and send version4

Quoted 6 lines
Quoted 3 lines

le16 num_buffers;
le32 hash_value; (Only if VIRTIO_NET_F_HASH_REPORT

negotiated)

Thanks

#5 lore raw
Quoted 187 lines · Shiva Shankar Kommula
Quoted 187 lines

-----Original Message-----
From: Jason Wang <[email protected]>
Sent: Thursday, February 20, 2025 7:20 AM
To: Shiva Shankar Kommula <[email protected]>
Cc: [email protected]; [email protected]; [email protected]; virtio- [email protected]; Nithin Kumar Dabilpuram <[email protected]>; Jerin Jacob <[email protected]>; Rahul Bhansali <[email protected]>
Subject: Re: [EXTERNAL] Re: [PATCH v3] virtio-net: Introduce a new field to indicate outer network header offset

On Wed, Feb 19, 2025 at 2: 40 PM Shiva Shankar Kommula <kshankar@ marvell. com> wrote: > > > > > -----Original Message----- > >
From: Jason Wang <jasowang@ redhat. com> > > Sent: Wednesday, February 19, 2025

Quoted 57 lines · Shiva Shankar Kommula
Quoted 57 lines

-----Original Message-----
From: Jason Wang <[email protected]>
Sent: Wednesday, February 19, 2025 9:00 AM
To: Shiva Shankar Kommula <[email protected]>
Cc: [email protected]; [email protected]; [email protected]; virtio- [email protected]; Nithin Kumar Dabilpuram <[email protected]>; Jerin Jacob <[email protected]>; Rahul Bhansali <[email protected]>
Subject: [EXTERNAL] Re: [PATCH v3] virtio-net: Introduce a new field to indicate outer network header offset

On Tue, Feb 18, 2025 at 1: 25 AM Kommula Shiva Shankar <kshankar@ marvell. com> wrote: > > This patch introduces a new field in the virtio_net_header called out_nh_offset, along with a new net device feature, VIRTIO_NET_F_OUT_NET_HEADER.

Quoted 2 lines · Kommula Shiva Shankar

This patch introduces a new field in the virtio_net_header called

out_nh_offset, along with a new net device feature, VIRTIO_NET_F_OUT_NET_HEADER.

Quoted 4 lines

Currently, drivers lack a dedicated field to signal the start of the network header to the device when performing checksum offload and

segmentation offload. This requires reading packet, which significantly affects performance in datapath.

Quoted 2 lines

Additionally, some hardware implementations requrie knowledege of the

outer L3 offset (aka L2 length) for inline IPSec hardware acceleration.

Quoted 3 lines

To address this limitation, we propose splitting the csum_offset field into

two 8-bit fields named csum_offset and out_nh_offset.

Quoted 20 lines

The csum_offset indicates the offset value from the csum_start and may not exceed 256B bits(2^8) for protocols that use a 16-bit one's complement checksum

Following table lists such protocols and their checksum offset fields within their headers

+-----+--------+
|Proto|csum_off|
+-----+--------+
| IPV4| 10B    |
| ICMP| 2B     |
| IGMP| 2B     |
|  TCP| 16B    |
|  UDP| 6B     |
+-----+--------+

The out_nh_offset represents the start byte offset of the outer network header from the beginning of the packet

Quoted 9 lines
Quoted 4 lines
Quoted 4 lines

This issue was briefly discussed on the mailing list in a different thread, which can be found here
https://urldefense.proofpoint.com/v2/url?u=https-3A__lore.kernel.o
rg_a

<https://urldefense.proofpoint.com/v2/url?u=https-3A__lore.kernel.org_
a >> > > ll_DM4PR18MB4269FAAC3CFC7E57E25DFBD2DF8B2-

Quoted 3 lines

40DM4PR18MB4269.namprd18.pro

Quoted 1 line

d.outlook.com_&d=DwIFaQ&c=nKjWec2b6R0mOyPaz7xtfQ&r=XP728rR_yI3b

Quoted 10 lines
Quoted 10 lines

CCyFq0M

Quoted 1 line

EWPS2176D_oSryc7cD-

0TRPs&m=jajIHiJb18aASNFHlkOtvANrA_06lDHp6zG9AEFU8u_

Quoted 1 line

nHyksxDbHI3UT0OFedi-b&s=MR9AX0Md9PFbZjQM-

bwJAmO13ZwETRZJCm9qFIYDy6E&e=

Quoted 5 lines

v2 -> v3:
- Rebase to virtio-1.4
- Addressed pending review commnets related to wording.

Quoted 7 lines
Quoted 3 lines
Quoted 3 lines

v2 -
https://urldefense.proofpoint.com/v2/url?u=https-3A__lore.kernel.o
rg_a

<https://urldefense.proofpoint.com/v2/url?u=https-3A__lore.kernel.org_
a >> > > ll_20250128142152.3662988-2D1-2Dkshankar-

Quoted 2 lines

40marvell.com_&d=DwIFaQ&c=nKj

Quoted 1 line

Wec2b6R0mOyPaz7xtfQ&r=XP728rR_yI3bCCyFq0MEWPS2176D_oSryc7cD-

Quoted 2 lines
Quoted 2 lines

0TRPs&m=ja

Quoted 1 line

jIHiJb18aASNFHlkOtvANrA_06lDHp6zG9AEFU8u_nHyksxDbHI3UT0OFedi-

Quoted 7 lines
Quoted 7 lines

b&s=jvYZB

Quoted 5 lines

QtOwWyydI5lFvCFz4hF94jDmU70nROTU44Ma0M&e=

v1 -> v2:
- explicitly state that the out_nh_offset can be set only when a
valid

network header is present.

Quoted 11 lines
Quoted 6 lines
Quoted 6 lines

- updated out_nh_offset usage in the RX direction.
- minor word cleanup.

v1 -
https://urldefense.proofpoint.com/v2/url?u=https-3A__lore.kernel.o
rg_v

<https://urldefense.proofpoint.com/v2/url?u=https-3A__lore.kernel.org_
v >> > > irtio-2Dcomment_20250114171636.3175670-2D1-2Dkshankar-

Quoted 3 lines

40marvell.com_&d

Quoted 1 line

=DwIFaQ&c=nKjWec2b6R0mOyPaz7xtfQ&r=XP728rR_yI3bCCyFq0MEWPS217

Quoted 3 lines
Quoted 3 lines

6D_oSryc7

Quoted 1 line

cD-

0TRPs&m=jajIHiJb18aASNFHlkOtvANrA_06lDHp6zG9AEFU8u_nHyksxDbHI3UT

Quoted 38 lines
Quoted 38 lines

0OF

Quoted 16 lines

edi-b&s=rzyoQrmZTznG5V9N03u7bpPbG1uQC0WPFmypJJKS9r8&e=

---
 device-types/net/description.tex | 37
++++++++++++++++++++++++++++++++
 1 file changed, 37 insertions(+)
device-types/net/description.tex +0 -0
index b546cd7..2a4aa66 100644
@@ -136,6 +136,10 @@ \subsection{Feature bits}\label{sec:Device

Types / Network Device / Feature bits

Quoted 1 line

\item[VIRTIO_NET_F_HOST_UDP_TUNNEL_GSO_CSUM (68)] Device

handles packets

Quoted 10 lines

carried by a UDP tunnel with partial csum for the outer header.

device-types/net/description.tex +4 -0
+
+\item[VIRTIO_NET_F_OUT_NET_HEADER(69)] Driver can provide the
+start of \field{out_nh_offset}  value. Device gains advantage by
+not reading packet to calculate outer network  header offset
 \end{description}
 
 \subsubsection{Feature bit requirements}\label{sec:Device Types /

Network Device / Feature bits / Feature bit requirements} @@ -592,6

+596,13 @@ \subsection{Device Operation}\label{sec:Device Types / +Network
Device / Device O

Quoted 6 lines
         le16 hdr_len;
         le16 gso_size;
         le16 csum_start;
device-types/net/description.tex +3 -0
+        union {
+                le16 csum_offset;
+                struct {        /* Only if VIRTIO_NET_F_OUT_NET_HEADER
Quoted 7 lines
Quoted 6 lines

negotiated */

device-types/net/description.tex \subsection{Device Operation}\label{sec:Device Types / Network Device / Device O · line 603

Quoted 2 lines
device-types/net/description.tex +2 -0
+                        le8 csum_offset;
+                        le8 out_nh_offset;

Need some explanation why reusing csum_offset is better than introducing new fields.

Our objective is to preserve the current size of the virtio_net_hdr structure,

to ensure compatibility with the existing header size.

Quoted 2 lines

(unlike other features such as hash reporting). Given that the csum_offset represents an offset within the header and may not exceed

2^8, we propose utilizing the existing space to accommodate this newly introduced field.

Quoted 2 lines

Would it be more prudent to add a new field for this feature ? Do you

foresee any issues with this approach ?

It might have some compatibility issue. For example in the current Linux implementation, csum_offset were 1:1 mapped to skb->csum_offset which is a u16.

Quoted 6 lines
Quoted 3 lines

I haven't checked all the protocols (especially the tunnels) but if one day there's a protocol that uses a csum_offset >= 256, it would be problematic.

I agree with the 1:1 mapping between csum_offset and skb->checksum_offset. Given that the virtio net header size is already a scarce resource for adding new fields, additionally increasing it for every new feature might impact performance in the data path.

New fields were added when new features were added. It doesn't harm too much as long as it doesn't cost more than one cacheline.

And addressing this "issue" seems to be orthogonal to the proposal here. Once we get a better way to place vnet header, network header will work there as well.

Quoted 2 lines

Would it be fine to add a note stating that when the VIRTIO_NET_F_OUT_NET_HEADER feature is negotiated, checksum_offset cannot exceed 2^8 offset ?

So if one day there's a protocol that uses a cusm_offset >= 256 with the network header, we need to invent new fields which are kind of not elegant.

I think it would be cleaner if we just introduce a new field.

Thanks

Quoted 19 lines
Quoted 18 lines

Thanks

Quoted 15 lines
Quoted 6 lines

device-types/net/description.tex \subsection{Device Operation}\label{sec:Device Types / Network Device / Device O · line 606

Quoted 4 lines
device-types/net/description.tex +2 -0
+                };
+        };
         le16 csum_offset;

It seems like it needs to be removed.

ACK .
Inadvertently included in version 3. Will fix up and send version4

Quoted 6 lines
Quoted 3 lines

le16 num_buffers;
le32 hash_value; (Only if VIRTIO_NET_F_HASH_REPORT

negotiated)

Thanks

#6 lore raw
Quoted 247 lines

-----Original Message-----
From: Jason Wang <[email protected]>
Sent: Wednesday, February 26, 2025 11:26 AM
To: Shiva Shankar Kommula <[email protected]>
Cc: [email protected]; [email protected]; [email protected]; virtio- [email protected]; Nithin Kumar Dabilpuram <[email protected]>; Jerin Jacob <[email protected]>; Rahul Bhansali <[email protected]>
Subject: Re: [EXTERNAL] Re: [PATCH v3] virtio-net: Introduce a new field to indicate outer network header offset

On Tue, Feb 25, 2025 at 1: 48 AM Shiva Shankar Kommula <kshankar@ marvell. com> wrote: > > > -----Original Message----- > > From:
Jason Wang <jasowang@ redhat. com> > > Sent: Thursday, February 20, 2025 7: 20 AM >

Quoted 79 lines · Shiva Shankar Kommula
Quoted 12 lines

-----Original Message-----
From: Jason Wang <[email protected]>
Sent: Thursday, February 20, 2025 7:20 AM
To: Shiva Shankar Kommula <[email protected]>
Cc: [email protected]; [email protected]; [email protected]; virtio- [email protected]; Nithin Kumar Dabilpuram <[email protected]>; Jerin Jacob <[email protected]>; Rahul Bhansali <[email protected]>
Subject: Re: [EXTERNAL] Re: [PATCH v3] virtio-net: Introduce a new field to indicate outer network header offset

Quoted 67 lines

On Wed, Feb 19, 2025 at 2: 40 PM Shiva Shankar Kommula <kshankar@ marvell. com> wrote: > > > > > -----Original Message----- > >
From: Jason Wang <jasowang@ redhat. com> > > Sent: Wednesday, February 19, 2025 On Wed, Feb 19, 2025 at 2:40 PM Shiva Shankar

Quoted 62 lines · Kommula
Quoted 62 lines

-----Original Message-----
From: Jason Wang <[email protected]>
Sent: Wednesday, February 19, 2025 9:00 AM
To: Shiva Shankar Kommula <[email protected]>
Cc: [email protected]; [email protected]; [email protected]; virtio- [email protected]; Nithin Kumar Dabilpuram <[email protected]>; Jerin Jacob <[email protected]>; Rahul Bhansali <[email protected]>
Subject: [EXTERNAL] Re: [PATCH v3] virtio-net: Introduce a new field to indicate outer network header offset

On Tue, Feb 18, 2025 at 1: 25 AM Kommula Shiva Shankar <kshankar@ marvell. com> wrote: > > This patch introduces a new field in the virtio_net_header called out_nh_offset, along with a new net device feature, VIRTIO_NET_F_OUT_NET_HEADER.

Quoted 3 lines · Kommula Shiva Shankar

This patch introduces a new field in the virtio_net_header called

out_nh_offset, along with a new net device feature, VIRTIO_NET_F_OUT_NET_HEADER.

Quoted 4 lines

Currently, drivers lack a dedicated field to signal the start of the network header to the device when performing checksum offload and

segmentation offload. This requires reading packet, which significantly affects performance in datapath.

Quoted 2 lines

Additionally, some hardware implementations requrie knowledege of the

outer L3 offset (aka L2 length) for inline IPSec hardware acceleration.

Quoted 3 lines

To address this limitation, we propose splitting the csum_offset field into

two 8-bit fields named csum_offset and out_nh_offset.

Quoted 24 lines

The csum_offset indicates the offset value from the csum_start and may not exceed 256B bits(2^8) for protocols that use a 16-bit one's complement checksum

Following table lists such protocols and their checksum offset fields within their headers

+-----+--------+
|Proto|csum_off|
+-----+--------+
| IPV4| 10B    |
| ICMP| 2B     |
| IGMP| 2B     |
|  TCP| 16B    |
|  UDP| 6B     |
+-----+--------+

The out_nh_offset represents the start byte offset of the outer network header from the beginning of the packet

This issue was briefly discussed on the mailing list in a different thread, which can be found here
https://urldefense.proofpoint.com/v2/url?u=https-3A__lore.kern
el.o

Quoted 10 lines

<https://urldefense.proofpoint.com/v2/url?u=https-3A__lore.kernel.o

Quoted 3 lines
Quoted 3 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

rg_a

<https://urldefense.proofpoint.com/v2/url?u=https-3A__lore.kernel.
org_

<https://urldefense.proofpoint.com/v2/url?u=https-3A__lore.kernel.org_

Quoted 5 lines
Quoted 4 lines
Quoted 4 lines
Quoted 1 line

a >> > > ll_DM4PR18MB4269FAAC3CFC7E57E25DFBD2DF8B2-

40DM4PR18MB4269.namprd18.pro

Quoted 1 line

d.outlook.com_&d=DwIFaQ&c=nKjWec2b6R0mOyPaz7xtfQ&r=XP728rR_yI3b

Quoted 10 lines
Quoted 10 lines
Quoted 10 lines
Quoted 10 lines

CCyFq0M

Quoted 1 line

EWPS2176D_oSryc7cD-

0TRPs&m=jajIHiJb18aASNFHlkOtvANrA_06lDHp6zG9AEFU8u_

Quoted 1 line

nHyksxDbHI3UT0OFedi-b&s=MR9AX0Md9PFbZjQM-

bwJAmO13ZwETRZJCm9qFIYDy6E&e=

Quoted 5 lines

v2 -> v3:
- Rebase to virtio-1.4
- Addressed pending review commnets related to wording.

Quoted 12 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

v2 -
https://urldefense.proofpoint.com/v2/url?u=https-3A__lore.kern
el.o

<https://urldefense.proofpoint.com/v2/url?u=https-3A__lore.kernel.o

Quoted 3 lines
Quoted 3 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

rg_a

<https://urldefense.proofpoint.com/v2/url?u=https-3A__lore.kernel.
org_

<https://urldefense.proofpoint.com/v2/url?u=https-3A__lore.kernel.org_

Quoted 4 lines
Quoted 3 lines
Quoted 3 lines
Quoted 1 line

a >> > > ll_20250128142152.3662988-2D1-2Dkshankar-

40marvell.com_&d=DwIFaQ&c=nKj

Quoted 1 line

Wec2b6R0mOyPaz7xtfQ&r=XP728rR_yI3bCCyFq0MEWPS2176D_oSryc7cD-

Quoted 16 lines
Quoted 16 lines
Quoted 2 lines
Quoted 2 lines

0TRPs&m=ja

Quoted 1 line

jIHiJb18aASNFHlkOtvANrA_06lDHp6zG9AEFU8u_nHyksxDbHI3UT0OFedi-

Quoted 13 lines
Quoted 13 lines

b&s=jvYZB

Quoted 5 lines

QtOwWyydI5lFvCFz4hF94jDmU70nROTU44Ma0M&e=

v1 -> v2:
- explicitly state that the out_nh_offset can be set only
when a valid

network header is present.

Quoted 6 lines

- updated out_nh_offset usage in the RX direction.
- minor word cleanup.

v1 -
https://urldefense.proofpoint.com/v2/url?u=https-3A__lore.kern
el.o

Quoted 11 lines

<https://urldefense.proofpoint.com/v2/url?u=https-3A__lore.kernel.o

Quoted 3 lines
Quoted 3 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

rg_v

<https://urldefense.proofpoint.com/v2/url?u=https-3A__lore.kernel.
org_

<https://urldefense.proofpoint.com/v2/url?u=https-3A__lore.kernel.org_

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

v >> > >

irtio-2Dcomment_20250114171636.3175670-2D1-2Dkshankar-

Quoted 4 lines
Quoted 3 lines
Quoted 3 lines

40marvell.com_&d

Quoted 1 line

=DwIFaQ&c=nKjWec2b6R0mOyPaz7xtfQ&r=XP728rR_yI3bCCyFq0MEWPS21 7

Quoted 4 lines
Quoted 4 lines
Quoted 3 lines
Quoted 3 lines

6D_oSryc7

Quoted 1 line

cD-

0TRPs&m=jajIHiJb18aASNFHlkOtvANrA_06lDHp6zG9AEFU8u_nHyksxDbHI3U T

Quoted 57 lines
Quoted 57 lines
Quoted 40 lines
Quoted 40 lines

0OF

Quoted 16 lines

edi-b&s=rzyoQrmZTznG5V9N03u7bpPbG1uQC0WPFmypJJKS9r8&e=

---
 device-types/net/description.tex | 37
++++++++++++++++++++++++++++++++
 1 file changed, 37 insertions(+)
device-types/net/description.tex +0 -0
index b546cd7..2a4aa66 100644
@@ -136,6 +136,10 @@ \subsection{Feature

bits}\label{sec:Device Types / Network Device / Feature bits

Quoted 1 line

\item[VIRTIO_NET_F_HOST_UDP_TUNNEL_GSO_CSUM (68)] Device

handles packets

Quoted 9 lines

carried by a UDP tunnel with partial csum for the outer header.

device-types/net/description.tex +5 -0
+
+\item[VIRTIO_NET_F_OUT_NET_HEADER(69)] Driver can provide the
+start of \field{out_nh_offset}  value. Device gains advantage
+by not reading packet to calculate outer network  header
+offset
 \end{description}
 
 \subsubsection{Feature bit requirements}\label{sec:Device
Quoted 3 lines

Types / Network Device / Feature bits / Feature bit requirements} @@
-592,6

+596,13 @@ \subsection{Device Operation}\label{sec:Device Types +/ Network
Device / Device O

Quoted 6 lines
         le16 hdr_len;
         le16 gso_size;
         le16 csum_start;
device-types/net/description.tex +3 -0
+        union {
+                le16 csum_offset;
+                struct {        /* Only if VIRTIO_NET_F_OUT_NET_HEADER
Quoted 8 lines
Quoted 6 lines

negotiated */

device-types/net/description.tex \subsection{Device Operation}\label{sec:Device Types / Network Device / Device O · line 603

Quoted 2 lines
device-types/net/description.tex +2 -0
+                        le8 csum_offset;
+                        le8 out_nh_offset;

Need some explanation why reusing csum_offset is better than introducing new fields.

Our objective is to preserve the current size of the virtio_net_hdr structure,

to ensure compatibility with the existing header size.

Quoted 3 lines

(unlike other features such as hash reporting). Given that the csum_offset represents an offset within the header and may not exceed

2^8, we propose utilizing the existing space to accommodate this newly introduced field.

Quoted 3 lines

Would it be more prudent to add a new field for this feature ? Do you

Quoted 8 lines
Quoted 8 lines

foresee any issues with this approach ?

It might have some compatibility issue. For example in the current Linux implementation, csum_offset were 1:1 mapped to skb->csum_offset which is a u16.

I haven't checked all the protocols (especially the tunnels) but if one day there's a protocol that uses a csum_offset >= 256, it would be

problematic.

Quoted 5 lines
Quoted 1 line

I agree with the 1:1 mapping between csum_offset and skb- checksum_offset.
Given that the virtio net header size is already a scarce resource for adding new fields, additionally increasing it for every new feature might

impact performance in the data path.

New fields were added when new features were added. It doesn't harm too much as long as it doesn't cost more than one cacheline.

And addressing this "issue" seems to be orthogonal to the proposal here. Once we get a better way to place vnet header, network header will work there as well.

Quoted 8 lines
Quoted 2 lines

Would it be fine to add a note stating that when the VIRTIO_NET_F_OUT_NET_HEADER feature is negotiated, checksum_offset

cannot exceed 2^8 offset ?

So if one day there's a protocol that uses a cusm_offset >= 256 with the network header, we need to invent new fields which are kind of not elegant.

I think it would be cleaner if we just introduce a new field.

Thank you for your feedback. I will send version 4 with the updated changes.

Quoted 22 lines

Thanks

Quoted 19 lines
Quoted 18 lines

Thanks

Quoted 15 lines
Quoted 6 lines

device-types/net/description.tex \subsection{Device Operation}\label{sec:Device Types / Network Device / Device O · line 606

Quoted 4 lines
device-types/net/description.tex +2 -0
+                };
+        };
         le16 csum_offset;

It seems like it needs to be removed.

ACK .
Inadvertently included in version 3. Will fix up and send version4

Quoted 6 lines
Quoted 3 lines

le16 num_buffers;
le32 hash_value; (Only if VIRTIO_NET_F_HASH_REPORT

negotiated)

Thanks

#7 lore raw
Quoted 6 lines

From: Shiva Shankar Kommula <[email protected]>
Sent: Thursday, February 27, 2025 3:19 PM

Quoted 2 lines

New fields were added when new features were added. It doesn't harm too much as long as it doesn't cost more than one cacheline.

Have you considered to having fixed vnet header of 64B size for this + any new fields?
This eliminates the existing problem where a partial cache line can be written for the vnet header on rx path. Because the data is typically written by one DMA and vnet header as second DMA as they originate from different sources within the device. A device can always do single DMA too to avoid partial cache line write but it does not seem practical enough.

#8 lore raw
Quoted 8 lines · Parav Pandit
Quoted 7 lines

From: Shiva Shankar Kommula <[email protected]>
Sent: Thursday, February 27, 2025 3:19 PM

Quoted 2 lines

New fields were added when new features were added. It doesn't harm too much as long as it doesn't cost more than one cacheline.

Have you considered to having fixed vnet header of 64B size for this + any new fields?

Yes, this should work (or even the vnet header length could be negotiated to match the cache line size).

Quoted 3 lines

This eliminates the existing problem where a partial cache line can be written for the vnet header on rx path. Because the data is typically written by one DMA and vnet header as second DMA as they originate from different sources within the device. A device can always do single DMA too to avoid partial cache line write but it does not seem practical enough.

Right. But I meant it needs to be addressed in a separate patch.

Thanks

#9 lore raw
Quoted 11 lines

From: Jason Wang <[email protected]>
Sent: Friday, February 28, 2025 1:50 PM

Quoted 5 lines · Parav Pandit
Quoted 4 lines

From: Shiva Shankar Kommula <[email protected]>
Sent: Thursday, February 27, 2025 3:19 PM

Quoted 8 lines
Quoted 3 lines
Quoted 2 lines
Quoted 2 lines

New fields were added when new features were added. It doesn't harm too much as long as it doesn't cost more than one cacheline.

Have you considered to having fixed vnet header of 64B size for this + any

new fields?

Yes, this should work (or even the vnet header length could be negotiated to match the cache line size).

Ok.
Negotiation needs early admin queue interface patch that I has been pending for a while now. Let me revise that.

Quoted 9 lines
Quoted 1 line

This eliminates the existing problem where a partial cache line can be

written for the vnet header on rx path.

Quoted 1 line

Because the data is typically written by one DMA and vnet header as second

DMA as they originate from different sources within the device.

Quoted 1 line

A device can always do single DMA too to avoid partial cache line write but

it does not seem practical enough.

Right. But I meant it needs to be addressed in a separate patch.

Sounds good.
Thanks

Quoted 1 line

Thanks