virtio-net: Introduce a new field to indicate outer network header offset
PATCH v3 1/1 virtio-net: Introduce a new field to indicate outer network header offset
This patch introduces a new field in the virtio_net_header called out_nh_offset, along with a new net device feature, VIRTIO_NET_F_OUT_NET_HEADER.
Currently, drivers lack a dedicated field to signal the start of the network header to the device when performing checksum offload and segmentation offload. This requires reading packet, which significantly affects performance in datapath. Additionally, some hardware implementations requrie knowledege of the outer L3 offset (aka L2 length) for inline IPSec hardware acceleration.
To address this limitation, we propose splitting the csum_offset field into two 8-bit fields named csum_offset and out_nh_offset. The csum_offset indicates the offset value from the csum_start and may not exceed 256B bits(2^8) for protocols that use a 16-bit one's complement checksum
Following table lists such protocols and their checksum offset fields within their headers
+-----+--------+ |Proto|csum_off| +-----+--------+ | IPV4| 10B | | ICMP| 2B | | IGMP| 2B | | TCP| 16B | | UDP| 6B | +-----+--------+
The out_nh_offset represents the start byte offset of the outer network header from the beginning of the packet
This issue was briefly discussed on the mailing list in a different thread, which can be found here
https://lore.kernel.org/all/DM4PR18MB4269FAAC3CFC7E57E25DFBD2DF8B2@DM4PR18MB4269.namprd18.prod.outlook.com/
v2 -> v3:
- Rebase to virtio-1.4
- Addressed pending review commnets related to wording.
v2 - https://lore.kernel.org/all/[email protected]/
v1 -> v2:
- explicitly state that the out_nh_offset can be set only when a valid network header is present.
- updated out_nh_offset usage in the RX direction.
- minor word cleanup.
v1 - https://lore.kernel.org/virtio-comment/[email protected]/
--- device-types/net/description.tex | 37 ++++++++++++++++++++++++++++++++ 1 file changed, 37 insertions(+)
device-types/net/description.tex +37 -0
@@ -136,6 +136,10 @@ \subsection{Feature bits}\label{sec:Device Types / Network Device / Feature bits \item[VIRTIO_NET_F_HOST_UDP_TUNNEL_GSO_CSUM (68)] Device handles packets carried by a UDP tunnel with partial csum for the outer header. + +\item[VIRTIO_NET_F_OUT_NET_HEADER(69)] Driver can provide the start of \field{out_nh_offset} + value. Device gains advantage by not reading packet to calculate outer network + header offset \end{description} \subsubsection{Feature bit requirements}\label{sec:Device Types / Network Device / Feature bits / Feature bit requirements}
@@ -592,6 +596,13 @@ \subsection{Device Operation}\label{sec:Device Types / Network Device / Device O le16 hdr_len; le16 gso_size; le16 csum_start; + union { + le16 csum_offset; + struct { /* Only if VIRTIO_NET_F_OUT_NET_HEADER negotiated */ + le8 csum_offset; + le8 out_nh_offset; + }; + }; le16 csum_offset; le16 num_buffers; le32 hash_value; (Only if VIRTIO_NET_F_HASH_REPORT negotiated)
@@ -633,6 +644,11 @@ \subsubsection{Packet Transmission}\label{sec:Device Types / Network Device / De \item The driver can send a completely checksummed packet. In this case, \field{flags} will be zero, and \field{gso_type} will be VIRTIO_NET_HDR_GSO_NONE. +\item When VIRTIO_NET_F_OUT_NET_HEADER is negotiated, the driver MAY optionally provide + the \field{out_nh_offset} value. A nonzero value of \field{out_nh_offset} indicates + a valid outer network header with in the packet, and specifies the offset in bytes from + the beginning of the packet. Otherwise \field{out_nh_offset} MUST not be used. + \item If the driver negotiated VIRTIO_NET_F_CSUM, it can skip checksumming the packet: \begin{itemize}
@@ -767,6 +783,11 @@ \subsubsection{Packet Transmission}\label{sec:Device Types / Network Device / De \field{flags} to zero and SHOULD supply a fully checksummed packet to the device. +If the VIRTIO_NET_F_OUT_NET_HEADER feature has been negotiated, +the driver MAY set \field{out_nh_offset} to indicate the start of the +outer network header offset, if the packet contains a valid network header. +Otherwise, \field{out_nh_offset} is not used. + If VIRTIO_NET_F_HOST_TSO4 is negotiated, the driver MAY set \field{gso_type} to VIRTIO_NET_HDR_GSO_TCPV4 to request TCPv4 segmentation, otherwise the driver MUST NOT set
@@ -948,6 +969,11 @@ \subsubsection{Packet Transmission}\label{sec:Device Types / Network Device / De If VIRTIO_NET_HDR_F_NEEDS_CSUM bit in \field{flags} is not set, the device MUST NOT use the \field{csum_start} and \field{csum_offset}. +If the VIRTIO_NET_F_OUT_NET_HEADER feature has been negotiated, +and \field{out_nh_offset} is not zero, the device MAY use \field{out_nh_offset} +as the outer network header offset. Otherwise, device MUST NOT use +the \field{out_nh_offset}. + If one of the VIRTIO_NET_F_HOST_TSO4, TSO6, USO or UFO options have been negotiated: \begin{itemize}
@@ -1102,6 +1128,9 @@ \subsubsection{Processing of Incoming Packets}\label{sec:Device Types / Network both the outer UDP checksum and the inner transport checksum have been validated, otherwise only one level of checksums (the outer one in case of tunnels) has been validated. +\item If the VIRTIO_NET_F_OUT_NET_HEADER has been negotiated, and if the packet + contains a valid network header, \field{out_nh_offset} MAY be set to indicate the + outer network header offset in packet. \end{enumerate} Additionally, VIRTIO_NET_F_GUEST_CSUM, TSO4, TSO6, UDP, UDP_TUNNEL
@@ -1215,6 +1244,10 @@ \subsubsection{Processing of Incoming Packets}\label{sec:Device Types / Network device MUST set the VIRTIO_NET_HDR_GSO_ECN bit in \field{gso_type}. +If VIRTIO_NET_F_OUT_NET_HEADER has been negotiated, the device MAY +set the \field{out_nh_offset} to indicate outer network header offset, if packet contains +a valid network header. Otherwise, the device MUST not use \field{out_nh_offset}. + If the VIRTIO_NET_F_GUEST_CSUM feature has been negotiated, the device MAY set the VIRTIO_NET_HDR_F_NEEDS_CSUM bit in \field{flags}, if so:
@@ -1302,6 +1335,10 @@ \subsubsection{Processing of Incoming Packets}\label{sec:Device Types / Network The driver MUST ignore \field{flag} bits that it does not recognize. +If VIRTIO_NET_F_OUT_NET_HEADER has been negotiated, and if \field{out_nh_offset} +is nonzero, the driver MAY use \field{out_nh_offset} as outer network header +offset. Otherwise, the driver MUST not use the \field{out_nh_offset}. + If VIRTIO_NET_HDR_F_NEEDS_CSUM bit in \field{flags} is not set or if VIRTIO_NET_HDR_F_RSC_INFO bit \field{flags} is set, the driver MUST NOT use the \field{csum_start} and \field{csum_offset}.
Signature · 1 line
2.43.0
Quoted 61 lines · Kommula Shiva Shankar
This patch introduces a new field in the
virtio_net_headercalledout_nh_offset, along with a new net device feature,VIRTIO_NET_F_OUT_NET_HEADER.Currently, drivers lack a dedicated field to signal the start of the network header to the device when performing checksum offload and segmentation offload. This requires reading packet, which significantly affects performance in datapath. Additionally, some hardware implementations requrie knowledege of the outer L3 offset (aka L2 length) for inline IPSec hardware acceleration.
To address this limitation, we propose splitting the
csum_offsetfield into two 8-bit fields namedcsum_offsetandout_nh_offset. Thecsum_offsetindicates the offset value from thecsum_startand may not exceed 256B bits(2^8) for protocols that use a 16-bit one's complement checksumFollowing table lists such protocols and their checksum offset fields within their headers
+-----+--------+ |Proto|csum_off| +-----+--------+ | IPV4| 10B | | ICMP| 2B | | IGMP| 2B | | TCP| 16B | | UDP| 6B | +-----+--------+The
out_nh_offsetrepresents the start byte offset of the outer network header from the beginning of the packetThis issue was briefly discussed on the mailing list in a different thread, which can be found here
https://lore.kernel.org/all/DM4PR18MB4269FAAC3CFC7E57E25DFBD2DF8B2@DM4PR18MB4269.namprd18.prod.outlook.com/v2 -> v3:
- Rebase to virtio-1.4
- Addressed pending review commnets related to wording.v2 - https://lore.kernel.org/all/[email protected]/
v1 -> v2:
- explicitly state that theout_nh_offsetcan be set only when a valid network header is present.
- updatedout_nh_offsetusage in the RX direction.
- minor word cleanup.v1 - https://lore.kernel.org/virtio-comment/[email protected]/
Signed-off-by Kommula Shiva Shankar <[email protected]>--- device-types/net/description.tex | 37 ++++++++++++++++++++++++++++++++ 1 file changed, 37 insertions(+)device-types/net/description.tex +4
-0@@ -136,6 +136,10 @@ \subsection{Feature bits}\label{sec:Device Types / Network Device / Feature bits \item[VIRTIO_NET_F_HOST_UDP_TUNNEL_GSO_CSUM (68)] Device handles packets carried by a UDP tunnel with partial csum for the outer header. + +\item[VIRTIO_NET_F_OUT_NET_HEADER(69)] Driver can provide the start of \field{out_nh_offset} + value. Device gains advantage by not reading packet to calculate outer network + header offset \end{description} \subsubsection{Feature bit requirements}\label{sec:Device Types / Network Device / Feature bits / Feature bit requirements}
device-types/net/description.tex \subsection{Device Operation}\label{sec:Device Types / Network Device / Device O · line 603
Quoted 8 lines
device-types/net/description.tex +5
-0@@ -592,6 +596,13 @@ \subsection{Device Operation}\label{sec:Device Types / Network Device / Device O le16 hdr_len; le16 gso_size; le16 csum_start; + union { + le16 csum_offset; + struct { /* Only if VIRTIO_NET_F_OUT_NET_HEADER negotiated */ + le8 csum_offset; + le8 out_nh_offset;
Need some explanation why reusing csum_offset is better than introducing new fields.
device-types/net/description.tex \subsection{Device Operation}\label{sec:Device Types / Network Device / Device O · line 606
Quoted 3 lines
device-types/net/description.tex +2
-0+ }; + }; le16 csum_offset;
It seems like it needs to be removed.
device-types/net/description.tex \subsection{Device Operation}\label{sec:Device Types / Network Device / Device O · line 608
Quoted 2 lines
device-types/net/description.tex +0
-0le16 num_buffers; le32 hash_value; (Only if VIRTIO_NET_F_HASH_REPORT negotiated)
Thanks
Quoted 131 lines
-----Original Message-----
From: Jason Wang <[email protected]>
Sent: Wednesday, February 19, 2025 9:00 AM
To: Shiva Shankar Kommula <[email protected]>
Cc: [email protected]; [email protected]; [email protected]; virtio- [email protected]; Nithin Kumar Dabilpuram <[email protected]>; Jerin Jacob <[email protected]>; Rahul Bhansali <[email protected]>
Subject: [EXTERNAL] Re: [PATCH v3] virtio-net: Introduce a new field to indicate outer network header offsetOn Tue, Feb 18, 2025 at 1: 25 AM Kommula Shiva Shankar <kshankar@ marvell. com> wrote: > > This patch introduces a new field in the
virtio_net_headercalledout_nh_offset, along with a new net device feature,VIRTIO_NET_F_OUT_NET_HEADER.Quoted 2 lines · Kommula Shiva Shankar
This patch introduces a new field in the
virtio_net_headercalled
out_nh_offset, along with a new net device feature,VIRTIO_NET_F_OUT_NET_HEADER.Quoted 3 lines
Currently, drivers lack a dedicated field to signal the start of the network header to the device when performing checksum offload and
segmentation offload. This requires reading packet, which significantly affects performance in datapath.
Quoted 1 line
Additionally, some hardware implementations requrie knowledege of the
outer L3 offset (aka L2 length) for inline IPSec hardware acceleration.
Quoted 2 lines
To address this limitation, we propose splitting the
csum_offsetfield intotwo 8-bit fields named
csum_offsetandout_nh_offset.Quoted 17 lines
The
csum_offsetindicates the offset value from thecsum_startand may not exceed 256B bits(2^8) for protocols that use a 16-bit one's complement checksumFollowing table lists such protocols and their checksum offset fields within their headers
+-----+--------+ |Proto|csum_off| +-----+--------+ | IPV4| 10B | | ICMP| 2B | | IGMP| 2B | | TCP| 16B | | UDP| 6B | +-----+--------+Quoted 7 lines
The
out_nh_offsetrepresents the start byte offset of the outer network header from the beginning of the packetThis issue was briefly discussed on the mailing list in a different thread, which can be found here
https://urldefense.proofpoint.com/v2/url?u=https-3A__lore.kernel.org_a
ll_DM4PR18MB4269FAAC3CFC7E57E25DFBD2DF8B2-40DM4PR18MB4269.namprd18.pro
Quoted 1 line
d.outlook.com_&d=DwIFaQ&c=nKjWec2b6R0mOyPaz7xtfQ&r=XP728rR_yI3b CCyFq0M
Quoted 1 line
EWPS2176D_oSryc7cD-
0TRPs&m=jajIHiJb18aASNFHlkOtvANrA_06lDHp6zG9AEFU8u_
Quoted 1 line
nHyksxDbHI3UT0OFedi-b&s=MR9AX0Md9PFbZjQM-
bwJAmO13ZwETRZJCm9qFIYDy6E&e=
Quoted 8 lines
v2 -> v3:
- Rebase to virtio-1.4
- Addressed pending review commnets related to wording.v2 -
https://urldefense.proofpoint.com/v2/url?u=https-3A__lore.kernel.org_a
ll_20250128142152.3662988-2D1-2Dkshankar-40marvell.com_&d=DwIFaQ&c=nKj
Quoted 1 line
Wec2b6R0mOyPaz7xtfQ&r=XP728rR_yI3bCCyFq0MEWPS2176D_oSryc7cD-
0TRPs&m=ja
Quoted 1 line
jIHiJb18aASNFHlkOtvANrA_06lDHp6zG9AEFU8u_nHyksxDbHI3UT0OFedi-
b&s=jvYZB
Quoted 4 lines
QtOwWyydI5lFvCFz4hF94jDmU70nROTU44Ma0M&e=
v1 -> v2:
- explicitly state that theout_nh_offsetcan be set only when a validnetwork header is present.
Quoted 6 lines
- updated
out_nh_offsetusage in the RX direction.
- minor word cleanup.v1 -
https://urldefense.proofpoint.com/v2/url?u=https-3A__lore.kernel.org_v
irtio-2Dcomment_20250114171636.3175670-2D1-2Dkshankar-40marvell.com_&d
Quoted 1 line
=DwIFaQ&c=nKjWec2b6R0mOyPaz7xtfQ&r=XP728rR_yI3bCCyFq0MEWPS217 6D_oSryc7
Quoted 1 line
cD-
0TRPs&m=jajIHiJb18aASNFHlkOtvANrA_06lDHp6zG9AEFU8u_nHyksxDbHI3UT 0OF
Quoted 16 lines
edi-b&s=rzyoQrmZTznG5V9N03u7bpPbG1uQC0WPFmypJJKS9r8&e=
Signed-off-by Kommula Shiva Shankar <[email protected]>--- device-types/net/description.tex | 37 ++++++++++++++++++++++++++++++++ 1 file changed, 37 insertions(+)device-types/net/description.tex +0
-0@@ -136,6 +136,10 @@ \subsection{Feature bits}\label{sec:Device Types/ Network Device / Feature bits
Quoted 1 line
\item[VIRTIO_NET_F_HOST_UDP_TUNNEL_GSO_CSUM (68)] Device
handles packets
Quoted 9 lines
carried by a UDP tunnel with partial csum for the outer header.
device-types/net/description.tex +4
-0+ +\item[VIRTIO_NET_F_OUT_NET_HEADER(69)] Driver can provide the start +of \field{out_nh_offset} value. Device gains advantage by not +reading packet to calculate outer network header offset \end{description} \subsubsection{Feature bit requirements}\label{sec:Device Types /Network Device / Feature bits / Feature bit requirements} @@ -592,6
+596,13 @@ \subsection{Device Operation}\label{sec:Device Types / Network Device / Device O
device-types/net/description.tex \subsection{Device Operation}\label{sec:Device Types / Network Device / Device O
Quoted 6 lines
le16 hdr_len; le16 gso_size; le16 csum_start;device-types/net/description.tex +3
-0+ union { + le16 csum_offset; + struct { /* Only if VIRTIO_NET_F_OUT_NET_HEADER
Quoted 6 lines
negotiated */
device-types/net/description.tex \subsection{Device Operation}\label{sec:Device Types / Network Device / Device O · line 603
Quoted 2 lines
device-types/net/description.tex +2
-0+ le8 csum_offset; + le8 out_nh_offset;Need some explanation why reusing
csum_offsetis better than introducing new fields.
Our objective is to preserve the current size of the virtio_net_hdr structure, to ensure compatibility with the existing header size. (unlike other features such as hash reporting). Given that the csum_offset represents an offset within the header and may not exceed 2^8, we propose utilizing the existing space to accommodate this newly introduced field.
Would it be more prudent to add a new field for this feature ? Do you foresee any issues with this approach ?
Quoted 6 lines
device-types/net/description.tex \subsection{Device Operation}\label{sec:Device Types / Network Device / Device O · line 606
Quoted 4 lines
device-types/net/description.tex +2
-0+ }; + }; le16 csum_offset;It seems like it needs to be removed.
ACK .
Inadvertently included in version 3. Will fix up and send version4
Quoted 6 lines
Quoted 3 lines
le16
num_buffers;
le32hash_value; (Only ifVIRTIO_NET_F_HASH_REPORTnegotiated)
Thanks
Quoted 136 lines · Shiva Shankar Kommula
Quoted 136 lines
-----Original Message-----
From: Jason Wang <[email protected]>
Sent: Wednesday, February 19, 2025 9:00 AM
To: Shiva Shankar Kommula <[email protected]>
Cc: [email protected]; [email protected]; [email protected]; virtio- [email protected]; Nithin Kumar Dabilpuram <[email protected]>; Jerin Jacob <[email protected]>; Rahul Bhansali <[email protected]>
Subject: [EXTERNAL] Re: [PATCH v3] virtio-net: Introduce a new field to indicate outer network header offsetOn Tue, Feb 18, 2025 at 1: 25 AM Kommula Shiva Shankar <kshankar@ marvell. com> wrote: > > This patch introduces a new field in the
virtio_net_headercalledout_nh_offset, along with a new net device feature,VIRTIO_NET_F_OUT_NET_HEADER.Quoted 2 lines · Kommula Shiva Shankar
This patch introduces a new field in the
virtio_net_headercalled
out_nh_offset, along with a new net device feature,VIRTIO_NET_F_OUT_NET_HEADER.Quoted 3 lines
Currently, drivers lack a dedicated field to signal the start of the network header to the device when performing checksum offload and
segmentation offload. This requires reading packet, which significantly affects performance in datapath.
Quoted 1 line
Additionally, some hardware implementations requrie knowledege of the
outer L3 offset (aka L2 length) for inline IPSec hardware acceleration.
Quoted 2 lines
To address this limitation, we propose splitting the
csum_offsetfield intotwo 8-bit fields named
csum_offsetandout_nh_offset.Quoted 17 lines
The
csum_offsetindicates the offset value from thecsum_startand may not exceed 256B bits(2^8) for protocols that use a 16-bit one's complement checksumFollowing table lists such protocols and their checksum offset fields within their headers
+-----+--------+ |Proto|csum_off| +-----+--------+ | IPV4| 10B | | ICMP| 2B | | IGMP| 2B | | TCP| 16B | | UDP| 6B | +-----+--------+Quoted 7 lines
The
out_nh_offsetrepresents the start byte offset of the outer network header from the beginning of the packetThis issue was briefly discussed on the mailing list in a different thread, which can be found here
https://urldefense.proofpoint.com/v2/url?u=https-3A__lore.kernel.org_a
ll_DM4PR18MB4269FAAC3CFC7E57E25DFBD2DF8B2-40DM4PR18MB4269.namprd18.pro
Quoted 1 line
d.outlook.com_&d=DwIFaQ&c=nKjWec2b6R0mOyPaz7xtfQ&r=XP728rR_yI3b CCyFq0M
Quoted 1 line
EWPS2176D_oSryc7cD-
0TRPs&m=jajIHiJb18aASNFHlkOtvANrA_06lDHp6zG9AEFU8u_
Quoted 1 line
nHyksxDbHI3UT0OFedi-b&s=MR9AX0Md9PFbZjQM-
bwJAmO13ZwETRZJCm9qFIYDy6E&e=
Quoted 8 lines
v2 -> v3:
- Rebase to virtio-1.4
- Addressed pending review commnets related to wording.v2 -
https://urldefense.proofpoint.com/v2/url?u=https-3A__lore.kernel.org_a
ll_20250128142152.3662988-2D1-2Dkshankar-40marvell.com_&d=DwIFaQ&c=nKj
Quoted 1 line
Wec2b6R0mOyPaz7xtfQ&r=XP728rR_yI3bCCyFq0MEWPS2176D_oSryc7cD-
0TRPs&m=ja
Quoted 1 line
jIHiJb18aASNFHlkOtvANrA_06lDHp6zG9AEFU8u_nHyksxDbHI3UT0OFedi-
b&s=jvYZB
Quoted 4 lines
QtOwWyydI5lFvCFz4hF94jDmU70nROTU44Ma0M&e=
v1 -> v2:
- explicitly state that theout_nh_offsetcan be set only when a validnetwork header is present.
Quoted 6 lines
- updated
out_nh_offsetusage in the RX direction.
- minor word cleanup.v1 -
https://urldefense.proofpoint.com/v2/url?u=https-3A__lore.kernel.org_v
irtio-2Dcomment_20250114171636.3175670-2D1-2Dkshankar-40marvell.com_&d
Quoted 1 line
=DwIFaQ&c=nKjWec2b6R0mOyPaz7xtfQ&r=XP728rR_yI3bCCyFq0MEWPS217 6D_oSryc7
Quoted 1 line
cD-
0TRPs&m=jajIHiJb18aASNFHlkOtvANrA_06lDHp6zG9AEFU8u_nHyksxDbHI3UT 0OF
Quoted 16 lines
edi-b&s=rzyoQrmZTznG5V9N03u7bpPbG1uQC0WPFmypJJKS9r8&e=
Signed-off-by Kommula Shiva Shankar <[email protected]>--- device-types/net/description.tex | 37 ++++++++++++++++++++++++++++++++ 1 file changed, 37 insertions(+)device-types/net/description.tex +0
-0@@ -136,6 +136,10 @@ \subsection{Feature bits}\label{sec:Device Types/ Network Device / Feature bits
Quoted 1 line
\item[VIRTIO_NET_F_HOST_UDP_TUNNEL_GSO_CSUM (68)] Device
handles packets
Quoted 9 lines
carried by a UDP tunnel with partial csum for the outer header.
device-types/net/description.tex +4
-0+ +\item[VIRTIO_NET_F_OUT_NET_HEADER(69)] Driver can provide the start +of \field{out_nh_offset} value. Device gains advantage by not +reading packet to calculate outer network header offset \end{description} \subsubsection{Feature bit requirements}\label{sec:Device Types /Network Device / Feature bits / Feature bit requirements} @@ -592,6
+596,13 @@ \subsection{Device Operation}\label{sec:Device Types / Network Device / Device O
device-types/net/description.tex \subsection{Device Operation}\label{sec:Device Types / Network Device / Device O
Quoted 6 lines
le16 hdr_len; le16 gso_size; le16 csum_start;device-types/net/description.tex +3
-0+ union { + le16 csum_offset; + struct { /* Only if VIRTIO_NET_F_OUT_NET_HEADERnegotiated */
Quoted 2 lines
device-types/net/description.tex +2
-0+ le8 csum_offset; + le8 out_nh_offset;
Quoted 7 lines
Quoted 2 lines
Need some explanation why reusing
csum_offsetis better than introducing new fields.Our objective is to preserve the current size of the
virtio_net_hdrstructure, to ensure compatibility with the existing header size. (unlike other features such as hash reporting). Given that thecsum_offsetrepresents an offset within the header and may not exceed 2^8, we propose utilizing the existing space to accommodate this newly introduced field.Would it be more prudent to add a new field for this feature ? Do you foresee any issues with this approach ?
It might have some compatibility issue. For example in the current Linux implementation, csum_offset were 1:1 mapped to skb->csum_offset which is a u16.
I haven't checked all the protocols (especially the tunnels) but if one day there's a protocol that uses a csum_offset >= 256, it would be problematic.
Thanks
Quoted 15 lines
Quoted 6 lines
device-types/net/description.tex \subsection{Device Operation}\label{sec:Device Types / Network Device / Device O · line 606
Quoted 4 lines
device-types/net/description.tex +2
-0+ }; + }; le16 csum_offset;It seems like it needs to be removed.
ACK .
Inadvertently included in version 3. Will fix up and send version4Quoted 6 lines
Quoted 3 lines
le16
num_buffers;
le32hash_value; (Only ifVIRTIO_NET_F_HASH_REPORTnegotiated)
Thanks
Quoted 180 lines
-----Original Message-----
From: Jason Wang <[email protected]>
Sent: Thursday, February 20, 2025 7:20 AM
To: Shiva Shankar Kommula <[email protected]>
Cc: [email protected]; [email protected]; [email protected]; virtio- [email protected]; Nithin Kumar Dabilpuram <[email protected]>; Jerin Jacob <[email protected]>; Rahul Bhansali <[email protected]>
Subject: Re: [EXTERNAL] Re: [PATCH v3] virtio-net: Introduce a new field to indicate outer network header offsetOn Wed, Feb 19, 2025 at 2: 40 PM Shiva Shankar Kommula <kshankar@ marvell. com> wrote: > > > > > -----Original Message----- > >
From: Jason Wang <jasowang@ redhat. com> > > Sent: Wednesday, February 19, 2025Quoted 57 lines · Shiva Shankar Kommula
Quoted 57 lines
-----Original Message-----
From: Jason Wang <[email protected]>
Sent: Wednesday, February 19, 2025 9:00 AM
To: Shiva Shankar Kommula <[email protected]>
Cc: [email protected]; [email protected]; [email protected]; virtio- [email protected]; Nithin Kumar Dabilpuram <[email protected]>; Jerin Jacob <[email protected]>; Rahul Bhansali <[email protected]>
Subject: [EXTERNAL] Re: [PATCH v3] virtio-net: Introduce a new field to indicate outer network header offsetOn Tue, Feb 18, 2025 at 1: 25 AM Kommula Shiva Shankar <kshankar@ marvell. com> wrote: > > This patch introduces a new field in the
virtio_net_headercalledout_nh_offset, along with a new net device feature,VIRTIO_NET_F_OUT_NET_HEADER.Quoted 2 lines · Kommula Shiva Shankar
This patch introduces a new field in the
virtio_net_headercalled
out_nh_offset, along with a new net device feature,VIRTIO_NET_F_OUT_NET_HEADER.Quoted 4 lines
Currently, drivers lack a dedicated field to signal the start of the network header to the device when performing checksum offload and
segmentation offload. This requires reading packet, which significantly affects performance in datapath.
Quoted 2 lines
Additionally, some hardware implementations requrie knowledege of the
outer L3 offset (aka L2 length) for inline IPSec hardware acceleration.
Quoted 3 lines
To address this limitation, we propose splitting the
csum_offsetfield intotwo 8-bit fields named
csum_offsetandout_nh_offset.Quoted 20 lines
The
csum_offsetindicates the offset value from thecsum_startand may not exceed 256B bits(2^8) for protocols that use a 16-bit one's complement checksumFollowing table lists such protocols and their checksum offset fields within their headers
+-----+--------+ |Proto|csum_off| +-----+--------+ | IPV4| 10B | | ICMP| 2B | | IGMP| 2B | | TCP| 16B | | UDP| 6B | +-----+--------+The
out_nh_offsetrepresents the start byte offset of the outer network header from the beginning of the packetQuoted 9 lines
Quoted 4 lines
Quoted 4 lines
This issue was briefly discussed on the mailing list in a different thread, which can be found here
https://urldefense.proofpoint.com/v2/url?u=https-3A__lore.kernel.org_a<https://urldefense.proofpoint.com/v2/url?u=https-3A__lore.kernel.org_
a >> > > ll_DM4PR18MB4269FAAC3CFC7E57E25DFBD2DF8B2-Quoted 3 lines
40DM4PR18MB4269.namprd18.pro
Quoted 1 line
d.outlook.com_&d=DwIFaQ&c=nKjWec2b6R0mOyPaz7xtfQ&r=XP728rR_yI3b
Quoted 10 lines
Quoted 10 lines
CCyFq0M
Quoted 1 line
EWPS2176D_oSryc7cD-
0TRPs&m=jajIHiJb18aASNFHlkOtvANrA_06lDHp6zG9AEFU8u_
Quoted 1 line
nHyksxDbHI3UT0OFedi-b&s=MR9AX0Md9PFbZjQM-
bwJAmO13ZwETRZJCm9qFIYDy6E&e=
Quoted 5 lines
v2 -> v3:
- Rebase to virtio-1.4
- Addressed pending review commnets related to wording.Quoted 7 lines
Quoted 3 lines
Quoted 3 lines
v2 -
https://urldefense.proofpoint.com/v2/url?u=https-3A__lore.kernel.org_a<https://urldefense.proofpoint.com/v2/url?u=https-3A__lore.kernel.org_
a >> > > ll_20250128142152.3662988-2D1-2Dkshankar-Quoted 2 lines
40marvell.com_&d=DwIFaQ&c=nKj
Quoted 1 line
Wec2b6R0mOyPaz7xtfQ&r=XP728rR_yI3bCCyFq0MEWPS2176D_oSryc7cD-
Quoted 2 lines
Quoted 2 lines
0TRPs&m=ja
Quoted 1 line
jIHiJb18aASNFHlkOtvANrA_06lDHp6zG9AEFU8u_nHyksxDbHI3UT0OFedi-
Quoted 7 lines
Quoted 7 lines
b&s=jvYZB
Quoted 5 lines
QtOwWyydI5lFvCFz4hF94jDmU70nROTU44Ma0M&e=
v1 -> v2:
- explicitly state that theout_nh_offsetcan be set only when a
validnetwork header is present.
Quoted 11 lines
Quoted 6 lines
Quoted 6 lines
- updated
out_nh_offsetusage in the RX direction.
- minor word cleanup.v1 -
https://urldefense.proofpoint.com/v2/url?u=https-3A__lore.kernel.org_v<https://urldefense.proofpoint.com/v2/url?u=https-3A__lore.kernel.org_
v >> > > irtio-2Dcomment_20250114171636.3175670-2D1-2Dkshankar-Quoted 3 lines
40marvell.com_&d
Quoted 1 line
=DwIFaQ&c=nKjWec2b6R0mOyPaz7xtfQ&r=XP728rR_yI3bCCyFq0MEWPS217
Quoted 3 lines
Quoted 3 lines
6D_oSryc7
Quoted 1 line
cD-
0TRPs&m=jajIHiJb18aASNFHlkOtvANrA_06lDHp6zG9AEFU8u_nHyksxDbHI3UT
Quoted 38 lines
Quoted 38 lines
0OF
Quoted 16 lines
edi-b&s=rzyoQrmZTznG5V9N03u7bpPbG1uQC0WPFmypJJKS9r8&e=
Signed-off-by Kommula Shiva Shankar <[email protected]>--- device-types/net/description.tex | 37 ++++++++++++++++++++++++++++++++ 1 file changed, 37 insertions(+)device-types/net/description.tex +0
-0@@ -136,6 +136,10 @@ \subsection{Feature bits}\label{sec:DeviceTypes / Network Device / Feature bits
Quoted 1 line
\item[VIRTIO_NET_F_HOST_UDP_TUNNEL_GSO_CSUM (68)] Device
handles packets
Quoted 10 lines
carried by a UDP tunnel with partial csum for the outer header.
device-types/net/description.tex +4
-0+ +\item[VIRTIO_NET_F_OUT_NET_HEADER(69)] Driver can provide the +start of \field{out_nh_offset} value. Device gains advantage by +not reading packet to calculate outer network header offset \end{description} \subsubsection{Feature bit requirements}\label{sec:Device Types /Network Device / Feature bits / Feature bit requirements} @@ -592,6
+596,13 @@ \subsection{Device Operation}\label{sec:Device Types / +Network
Device / Device OQuoted 6 lines
le16 hdr_len; le16 gso_size; le16 csum_start;device-types/net/description.tex +3
-0+ union { + le16 csum_offset; + struct { /* Only if VIRTIO_NET_F_OUT_NET_HEADERQuoted 7 lines
Quoted 6 lines
negotiated */
device-types/net/description.tex \subsection{Device Operation}\label{sec:Device Types / Network Device / Device O · line 603
Quoted 2 lines
device-types/net/description.tex +2
-0+ le8 csum_offset; + le8 out_nh_offset;Need some explanation why reusing
csum_offsetis better than introducing new fields.Our objective is to preserve the current size of the
virtio_net_hdrstructure,to ensure compatibility with the existing header size.
Quoted 2 lines
(unlike other features such as hash reporting). Given that the
csum_offsetrepresents an offset within the header and may not exceed2^8, we propose utilizing the existing space to accommodate this newly introduced field.
Quoted 2 lines
Would it be more prudent to add a new field for this feature ? Do you
Quoted 9 lines
foresee any issues with this approach ?
It might have some compatibility issue. For example in the current Linux implementation,
csum_offsetwere 1:1 mapped toskb->csum_offsetwhich is a u16.I haven't checked all the protocols (especially the tunnels) but if one day there's a protocol that uses a
csum_offset>= 256, it would be problematic.
I agree with the 1:1 mapping between csum_offset and skb->checksum_offset. Given that the virtio net header size is already a scarce resource for adding new fields, additionally increasing it for every new feature might impact performance in the data path. Would it be fine to add a note stating that when the VIRTIO_NET_F_OUT_NET_HEADER feature is negotiated, checksum_offset cannot exceed 2^8 offset ?
Quoted 17 lines
Thanks
Quoted 15 lines
Quoted 6 lines
device-types/net/description.tex \subsection{Device Operation}\label{sec:Device Types / Network Device / Device O · line 606
Quoted 4 lines
device-types/net/description.tex +2
-0+ }; + }; le16 csum_offset;It seems like it needs to be removed.
ACK .
Inadvertently included in version 3. Will fix up and send version4Quoted 6 lines
Quoted 3 lines
le16
num_buffers;
le32hash_value; (Only ifVIRTIO_NET_F_HASH_REPORTnegotiated)
Thanks
Quoted 187 lines · Shiva Shankar Kommula
Quoted 187 lines
-----Original Message-----
From: Jason Wang <[email protected]>
Sent: Thursday, February 20, 2025 7:20 AM
To: Shiva Shankar Kommula <[email protected]>
Cc: [email protected]; [email protected]; [email protected]; virtio- [email protected]; Nithin Kumar Dabilpuram <[email protected]>; Jerin Jacob <[email protected]>; Rahul Bhansali <[email protected]>
Subject: Re: [EXTERNAL] Re: [PATCH v3] virtio-net: Introduce a new field to indicate outer network header offsetOn Wed, Feb 19, 2025 at 2: 40 PM Shiva Shankar Kommula <kshankar@ marvell. com> wrote: > > > > > -----Original Message----- > >
From: Jason Wang <jasowang@ redhat. com> > > Sent: Wednesday, February 19, 2025Quoted 57 lines · Shiva Shankar Kommula
Quoted 57 lines
-----Original Message-----
From: Jason Wang <[email protected]>
Sent: Wednesday, February 19, 2025 9:00 AM
To: Shiva Shankar Kommula <[email protected]>
Cc: [email protected]; [email protected]; [email protected]; virtio- [email protected]; Nithin Kumar Dabilpuram <[email protected]>; Jerin Jacob <[email protected]>; Rahul Bhansali <[email protected]>
Subject: [EXTERNAL] Re: [PATCH v3] virtio-net: Introduce a new field to indicate outer network header offsetOn Tue, Feb 18, 2025 at 1: 25 AM Kommula Shiva Shankar <kshankar@ marvell. com> wrote: > > This patch introduces a new field in the
virtio_net_headercalledout_nh_offset, along with a new net device feature,VIRTIO_NET_F_OUT_NET_HEADER.Quoted 2 lines · Kommula Shiva Shankar
This patch introduces a new field in the
virtio_net_headercalled
out_nh_offset, along with a new net device feature,VIRTIO_NET_F_OUT_NET_HEADER.Quoted 4 lines
Currently, drivers lack a dedicated field to signal the start of the network header to the device when performing checksum offload and
segmentation offload. This requires reading packet, which significantly affects performance in datapath.
Quoted 2 lines
Additionally, some hardware implementations requrie knowledege of the
outer L3 offset (aka L2 length) for inline IPSec hardware acceleration.
Quoted 3 lines
To address this limitation, we propose splitting the
csum_offsetfield intotwo 8-bit fields named
csum_offsetandout_nh_offset.Quoted 20 lines
The
csum_offsetindicates the offset value from thecsum_startand may not exceed 256B bits(2^8) for protocols that use a 16-bit one's complement checksumFollowing table lists such protocols and their checksum offset fields within their headers
+-----+--------+ |Proto|csum_off| +-----+--------+ | IPV4| 10B | | ICMP| 2B | | IGMP| 2B | | TCP| 16B | | UDP| 6B | +-----+--------+The
out_nh_offsetrepresents the start byte offset of the outer network header from the beginning of the packetQuoted 9 lines
Quoted 4 lines
Quoted 4 lines
This issue was briefly discussed on the mailing list in a different thread, which can be found here
https://urldefense.proofpoint.com/v2/url?u=https-3A__lore.kernel.org_a<https://urldefense.proofpoint.com/v2/url?u=https-3A__lore.kernel.org_
a >> > > ll_DM4PR18MB4269FAAC3CFC7E57E25DFBD2DF8B2-Quoted 3 lines
40DM4PR18MB4269.namprd18.pro
Quoted 1 line
d.outlook.com_&d=DwIFaQ&c=nKjWec2b6R0mOyPaz7xtfQ&r=XP728rR_yI3b
Quoted 10 lines
Quoted 10 lines
CCyFq0M
Quoted 1 line
EWPS2176D_oSryc7cD-
0TRPs&m=jajIHiJb18aASNFHlkOtvANrA_06lDHp6zG9AEFU8u_
Quoted 1 line
nHyksxDbHI3UT0OFedi-b&s=MR9AX0Md9PFbZjQM-
bwJAmO13ZwETRZJCm9qFIYDy6E&e=
Quoted 5 lines
v2 -> v3:
- Rebase to virtio-1.4
- Addressed pending review commnets related to wording.Quoted 7 lines
Quoted 3 lines
Quoted 3 lines
v2 -
https://urldefense.proofpoint.com/v2/url?u=https-3A__lore.kernel.org_a<https://urldefense.proofpoint.com/v2/url?u=https-3A__lore.kernel.org_
a >> > > ll_20250128142152.3662988-2D1-2Dkshankar-Quoted 2 lines
40marvell.com_&d=DwIFaQ&c=nKj
Quoted 1 line
Wec2b6R0mOyPaz7xtfQ&r=XP728rR_yI3bCCyFq0MEWPS2176D_oSryc7cD-
Quoted 2 lines
Quoted 2 lines
0TRPs&m=ja
Quoted 1 line
jIHiJb18aASNFHlkOtvANrA_06lDHp6zG9AEFU8u_nHyksxDbHI3UT0OFedi-
Quoted 7 lines
Quoted 7 lines
b&s=jvYZB
Quoted 5 lines
QtOwWyydI5lFvCFz4hF94jDmU70nROTU44Ma0M&e=
v1 -> v2:
- explicitly state that theout_nh_offsetcan be set only when a
validnetwork header is present.
Quoted 11 lines
Quoted 6 lines
Quoted 6 lines
- updated
out_nh_offsetusage in the RX direction.
- minor word cleanup.v1 -
https://urldefense.proofpoint.com/v2/url?u=https-3A__lore.kernel.org_v<https://urldefense.proofpoint.com/v2/url?u=https-3A__lore.kernel.org_
v >> > > irtio-2Dcomment_20250114171636.3175670-2D1-2Dkshankar-Quoted 3 lines
40marvell.com_&d
Quoted 1 line
=DwIFaQ&c=nKjWec2b6R0mOyPaz7xtfQ&r=XP728rR_yI3bCCyFq0MEWPS217
Quoted 3 lines
Quoted 3 lines
6D_oSryc7
Quoted 1 line
cD-
0TRPs&m=jajIHiJb18aASNFHlkOtvANrA_06lDHp6zG9AEFU8u_nHyksxDbHI3UT
Quoted 38 lines
Quoted 38 lines
0OF
Quoted 16 lines
edi-b&s=rzyoQrmZTznG5V9N03u7bpPbG1uQC0WPFmypJJKS9r8&e=
Signed-off-by Kommula Shiva Shankar <[email protected]>--- device-types/net/description.tex | 37 ++++++++++++++++++++++++++++++++ 1 file changed, 37 insertions(+)device-types/net/description.tex +0
-0@@ -136,6 +136,10 @@ \subsection{Feature bits}\label{sec:DeviceTypes / Network Device / Feature bits
Quoted 1 line
\item[VIRTIO_NET_F_HOST_UDP_TUNNEL_GSO_CSUM (68)] Device
handles packets
Quoted 10 lines
carried by a UDP tunnel with partial csum for the outer header.
device-types/net/description.tex +4
-0+ +\item[VIRTIO_NET_F_OUT_NET_HEADER(69)] Driver can provide the +start of \field{out_nh_offset} value. Device gains advantage by +not reading packet to calculate outer network header offset \end{description} \subsubsection{Feature bit requirements}\label{sec:Device Types /Network Device / Feature bits / Feature bit requirements} @@ -592,6
+596,13 @@ \subsection{Device Operation}\label{sec:Device Types / +Network
Device / Device OQuoted 6 lines
le16 hdr_len; le16 gso_size; le16 csum_start;device-types/net/description.tex +3
-0+ union { + le16 csum_offset; + struct { /* Only if VIRTIO_NET_F_OUT_NET_HEADERQuoted 7 lines
Quoted 6 lines
negotiated */
device-types/net/description.tex \subsection{Device Operation}\label{sec:Device Types / Network Device / Device O · line 603
Quoted 2 lines
device-types/net/description.tex +2
-0+ le8 csum_offset; + le8 out_nh_offset;Need some explanation why reusing
csum_offsetis better than introducing new fields.Our objective is to preserve the current size of the
virtio_net_hdrstructure,to ensure compatibility with the existing header size.
Quoted 2 lines
(unlike other features such as hash reporting). Given that the
csum_offsetrepresents an offset within the header and may not exceed2^8, we propose utilizing the existing space to accommodate this newly introduced field.
Quoted 2 lines
Would it be more prudent to add a new field for this feature ? Do you
foresee any issues with this approach ?
It might have some compatibility issue. For example in the current Linux implementation,
csum_offsetwere 1:1 mapped toskb->csum_offsetwhich is a u16.
Quoted 6 lines
Quoted 3 lines
I haven't checked all the protocols (especially the tunnels) but if one day there's a protocol that uses a
csum_offset>= 256, it would be problematic.I agree with the 1:1 mapping between
csum_offsetandskb->checksum_offset. Given that the virtio net header size is already a scarce resource for adding new fields, additionally increasing it for every new feature might impact performance in the data path.
New fields were added when new features were added. It doesn't harm too much as long as it doesn't cost more than one cacheline.
And addressing this "issue" seems to be orthogonal to the proposal here. Once we get a better way to place vnet header, network header will work there as well.
Quoted 2 lines
Would it be fine to add a note stating that when the
VIRTIO_NET_F_OUT_NET_HEADERfeature is negotiated,checksum_offsetcannot exceed 2^8 offset ?
So if one day there's a protocol that uses a cusm_offset >= 256 with the network header, we need to invent new fields which are kind of not elegant.
I think it would be cleaner if we just introduce a new field.
Thanks
Quoted 19 lines
Quoted 18 lines
Thanks
Quoted 15 lines
Quoted 6 lines
device-types/net/description.tex \subsection{Device Operation}\label{sec:Device Types / Network Device / Device O · line 606
Quoted 4 lines
device-types/net/description.tex +2
-0+ }; + }; le16 csum_offset;It seems like it needs to be removed.
ACK .
Inadvertently included in version 3. Will fix up and send version4Quoted 6 lines
Quoted 3 lines
le16
num_buffers;
le32hash_value; (Only ifVIRTIO_NET_F_HASH_REPORTnegotiated)
Thanks
Quoted 247 lines
-----Original Message-----
From: Jason Wang <[email protected]>
Sent: Wednesday, February 26, 2025 11:26 AM
To: Shiva Shankar Kommula <[email protected]>
Cc: [email protected]; [email protected]; [email protected]; virtio- [email protected]; Nithin Kumar Dabilpuram <[email protected]>; Jerin Jacob <[email protected]>; Rahul Bhansali <[email protected]>
Subject: Re: [EXTERNAL] Re: [PATCH v3] virtio-net: Introduce a new field to indicate outer network header offsetOn Tue, Feb 25, 2025 at 1: 48 AM Shiva Shankar Kommula <kshankar@ marvell. com> wrote: > > > -----Original Message----- > > From:
Jason Wang <jasowang@ redhat. com> > > Sent: Thursday, February 20, 2025 7: 20 AM >Quoted 79 lines · Shiva Shankar Kommula
Quoted 12 lines
-----Original Message-----
From: Jason Wang <[email protected]>
Sent: Thursday, February 20, 2025 7:20 AM
To: Shiva Shankar Kommula <[email protected]>
Cc: [email protected]; [email protected]; [email protected]; virtio- [email protected]; Nithin Kumar Dabilpuram <[email protected]>; Jerin Jacob <[email protected]>; Rahul Bhansali <[email protected]>
Subject: Re: [EXTERNAL] Re: [PATCH v3] virtio-net: Introduce a new field to indicate outer network header offsetQuoted 67 lines
On Wed, Feb 19, 2025 at 2: 40 PM Shiva Shankar Kommula <kshankar@ marvell. com> wrote: > > > > > -----Original Message----- > >
From: Jason Wang <jasowang@ redhat. com> > > Sent: Wednesday, February 19, 2025 On Wed, Feb 19, 2025 at 2:40 PM Shiva ShankarQuoted 62 lines · Kommula
Quoted 62 lines
-----Original Message-----
From: Jason Wang <[email protected]>
Sent: Wednesday, February 19, 2025 9:00 AM
To: Shiva Shankar Kommula <[email protected]>
Cc: [email protected]; [email protected]; [email protected]; virtio- [email protected]; Nithin Kumar Dabilpuram <[email protected]>; Jerin Jacob <[email protected]>; Rahul Bhansali <[email protected]>
Subject: [EXTERNAL] Re: [PATCH v3] virtio-net: Introduce a new field to indicate outer network header offsetOn Tue, Feb 18, 2025 at 1: 25 AM Kommula Shiva Shankar <kshankar@ marvell. com> wrote: > > This patch introduces a new field in the
virtio_net_headercalledout_nh_offset, along with a new net device feature,VIRTIO_NET_F_OUT_NET_HEADER.Quoted 3 lines · Kommula Shiva Shankar
This patch introduces a new field in the
virtio_net_headercalled
out_nh_offset, along with a new net device feature,VIRTIO_NET_F_OUT_NET_HEADER.Quoted 4 lines
Currently, drivers lack a dedicated field to signal the start of the network header to the device when performing checksum offload and
segmentation offload. This requires reading packet, which significantly affects performance in datapath.
Quoted 2 lines
Additionally, some hardware implementations requrie knowledege of the
outer L3 offset (aka L2 length) for inline IPSec hardware acceleration.
Quoted 3 lines
To address this limitation, we propose splitting the
csum_offsetfield intotwo 8-bit fields named
csum_offsetandout_nh_offset.Quoted 24 lines
The
csum_offsetindicates the offset value from thecsum_startand may not exceed 256B bits(2^8) for protocols that use a 16-bit one's complement checksumFollowing table lists such protocols and their checksum offset fields within their headers
+-----+--------+ |Proto|csum_off| +-----+--------+ | IPV4| 10B | | ICMP| 2B | | IGMP| 2B | | TCP| 16B | | UDP| 6B | +-----+--------+The
out_nh_offsetrepresents the start byte offset of the outer network header from the beginning of the packetThis issue was briefly discussed on the mailing list in a different thread, which can be found here
https://urldefense.proofpoint.com/v2/url?u=https-3A__lore.kern
el.oQuoted 10 lines
<https://urldefense.proofpoint.com/v2/url?u=https-3A__lore.kernel.o
Quoted 3 lines
Quoted 3 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
rg_a<https://urldefense.proofpoint.com/v2/url?u=https-3A__lore.kernel.
org_<https://urldefense.proofpoint.com/v2/url?u=https-3A__lore.kernel.org_
Quoted 5 lines
Quoted 4 lines
Quoted 4 lines
Quoted 1 line
a >> > > ll_DM4PR18MB4269FAAC3CFC7E57E25DFBD2DF8B2-
40DM4PR18MB4269.namprd18.pro
Quoted 1 line
d.outlook.com_&d=DwIFaQ&c=nKjWec2b6R0mOyPaz7xtfQ&r=XP728rR_yI3b
Quoted 10 lines
Quoted 10 lines
Quoted 10 lines
Quoted 10 lines
CCyFq0M
Quoted 1 line
EWPS2176D_oSryc7cD-
0TRPs&m=jajIHiJb18aASNFHlkOtvANrA_06lDHp6zG9AEFU8u_
Quoted 1 line
nHyksxDbHI3UT0OFedi-b&s=MR9AX0Md9PFbZjQM-
bwJAmO13ZwETRZJCm9qFIYDy6E&e=
Quoted 5 lines
v2 -> v3:
- Rebase to virtio-1.4
- Addressed pending review commnets related to wording.Quoted 12 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
v2 -
https://urldefense.proofpoint.com/v2/url?u=https-3A__lore.kern
el.o<https://urldefense.proofpoint.com/v2/url?u=https-3A__lore.kernel.o
Quoted 3 lines
Quoted 3 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
rg_a<https://urldefense.proofpoint.com/v2/url?u=https-3A__lore.kernel.
org_<https://urldefense.proofpoint.com/v2/url?u=https-3A__lore.kernel.org_
Quoted 4 lines
Quoted 3 lines
Quoted 3 lines
Quoted 1 line
a >> > > ll_20250128142152.3662988-2D1-2Dkshankar-
40marvell.com_&d=DwIFaQ&c=nKj
Quoted 1 line
Wec2b6R0mOyPaz7xtfQ&r=XP728rR_yI3bCCyFq0MEWPS2176D_oSryc7cD-
Quoted 16 lines
Quoted 16 lines
Quoted 2 lines
Quoted 2 lines
0TRPs&m=ja
Quoted 1 line
jIHiJb18aASNFHlkOtvANrA_06lDHp6zG9AEFU8u_nHyksxDbHI3UT0OFedi-
Quoted 13 lines
Quoted 13 lines
b&s=jvYZB
Quoted 5 lines
QtOwWyydI5lFvCFz4hF94jDmU70nROTU44Ma0M&e=
v1 -> v2:
- explicitly state that theout_nh_offsetcan be set only
when a validnetwork header is present.
Quoted 6 lines
- updated
out_nh_offsetusage in the RX direction.
- minor word cleanup.v1 -
https://urldefense.proofpoint.com/v2/url?u=https-3A__lore.kern
el.oQuoted 11 lines
<https://urldefense.proofpoint.com/v2/url?u=https-3A__lore.kernel.o
Quoted 3 lines
Quoted 3 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
rg_v<https://urldefense.proofpoint.com/v2/url?u=https-3A__lore.kernel.
org_<https://urldefense.proofpoint.com/v2/url?u=https-3A__lore.kernel.org_
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
v >> > >
irtio-2Dcomment_20250114171636.3175670-2D1-2Dkshankar-
Quoted 4 lines
Quoted 3 lines
Quoted 3 lines
40marvell.com_&d
Quoted 1 line
=DwIFaQ&c=nKjWec2b6R0mOyPaz7xtfQ&r=XP728rR_yI3bCCyFq0MEWPS21 7
Quoted 4 lines
Quoted 4 lines
Quoted 3 lines
Quoted 3 lines
6D_oSryc7
Quoted 1 line
cD-
0TRPs&m=jajIHiJb18aASNFHlkOtvANrA_06lDHp6zG9AEFU8u_nHyksxDbHI3U T
Quoted 57 lines
Quoted 57 lines
Quoted 40 lines
Quoted 40 lines
0OF
Quoted 16 lines
edi-b&s=rzyoQrmZTznG5V9N03u7bpPbG1uQC0WPFmypJJKS9r8&e=
Signed-off-by Kommula Shiva Shankar <[email protected]>--- device-types/net/description.tex | 37 ++++++++++++++++++++++++++++++++ 1 file changed, 37 insertions(+)device-types/net/description.tex +0
-0@@ -136,6 +136,10 @@ \subsection{Featurebits}\label{sec:Device Types / Network Device / Feature bits
Quoted 1 line
\item[VIRTIO_NET_F_HOST_UDP_TUNNEL_GSO_CSUM (68)] Device
handles packets
Quoted 9 lines
carried by a UDP tunnel with partial csum for the outer header.
device-types/net/description.tex +5
-0+ +\item[VIRTIO_NET_F_OUT_NET_HEADER(69)] Driver can provide the +start of \field{out_nh_offset} value. Device gains advantage +by not reading packet to calculate outer network header +offset \end{description} \subsubsection{Feature bit requirements}\label{sec:DeviceQuoted 3 lines
Types / Network Device / Feature bits / Feature bit requirements} @@
-592,6+596,13 @@ \subsection{Device Operation}\label{sec:Device Types +/ Network
Device / Device OQuoted 6 lines
le16 hdr_len; le16 gso_size; le16 csum_start;device-types/net/description.tex +3
-0+ union { + le16 csum_offset; + struct { /* Only if VIRTIO_NET_F_OUT_NET_HEADERQuoted 8 lines
Quoted 6 lines
negotiated */
device-types/net/description.tex \subsection{Device Operation}\label{sec:Device Types / Network Device / Device O · line 603
Quoted 2 lines
device-types/net/description.tex +2
-0+ le8 csum_offset; + le8 out_nh_offset;Need some explanation why reusing
csum_offsetis better than introducing new fields.Our objective is to preserve the current size of the
virtio_net_hdrstructure,to ensure compatibility with the existing header size.
Quoted 3 lines
(unlike other features such as hash reporting). Given that the
csum_offsetrepresents an offset within the header and may not exceed2^8, we propose utilizing the existing space to accommodate this newly introduced field.
Quoted 3 lines
Would it be more prudent to add a new field for this feature ? Do you
Quoted 8 lines
Quoted 8 lines
foresee any issues with this approach ?
It might have some compatibility issue. For example in the current Linux implementation,
csum_offsetwere 1:1 mapped toskb->csum_offsetwhich is a u16.I haven't checked all the protocols (especially the tunnels) but if one day there's a protocol that uses a
csum_offset>= 256, it would beproblematic.
Quoted 5 lines
Quoted 1 line
I agree with the 1:1 mapping between
csum_offsetand skb-checksum_offset.
Given that the virtio net header size is already a scarce resource for adding new fields, additionally increasing it for every new feature mightimpact performance in the data path.
New fields were added when new features were added. It doesn't harm too much as long as it doesn't cost more than one cacheline.
And addressing this "issue" seems to be orthogonal to the proposal here. Once we get a better way to place vnet header, network header will work there as well.
Quoted 8 lines
Quoted 2 lines
Would it be fine to add a note stating that when the
VIRTIO_NET_F_OUT_NET_HEADERfeature is negotiated,checksum_offsetcannot exceed 2^8 offset ?
So if one day there's a protocol that uses a
cusm_offset>= 256 with the network header, we need to invent new fields which are kind of not elegant.I think it would be cleaner if we just introduce a new field.
Thank you for your feedback. I will send version 4 with the updated changes.
Quoted 22 lines
Thanks
Quoted 19 lines
Quoted 18 lines
Thanks
Quoted 15 lines
Quoted 6 lines
device-types/net/description.tex \subsection{Device Operation}\label{sec:Device Types / Network Device / Device O · line 606
Quoted 4 lines
device-types/net/description.tex +2
-0+ }; + }; le16 csum_offset;It seems like it needs to be removed.
ACK .
Inadvertently included in version 3. Will fix up and send version4Quoted 6 lines
Quoted 3 lines
le16
num_buffers;
le32hash_value; (Only ifVIRTIO_NET_F_HASH_REPORTnegotiated)
Thanks
Quoted 6 lines
From: Shiva Shankar Kommula <[email protected]>
Sent: Thursday, February 27, 2025 3:19 PMQuoted 2 lines
New fields were added when new features were added. It doesn't harm too much as long as it doesn't cost more than one cacheline.
Have you considered to having fixed vnet header of 64B size for this + any new fields?
This eliminates the existing problem where a partial cache line can be written for the vnet header on rx path. Because the data is typically written by one DMA and vnet header as second DMA as they originate from different sources within the device. A device can always do single DMA too to avoid partial cache line write but it does not seem practical enough.
Quoted 8 lines · Parav Pandit
Quoted 7 lines
From: Shiva Shankar Kommula <[email protected]>
Sent: Thursday, February 27, 2025 3:19 PMQuoted 2 lines
New fields were added when new features were added. It doesn't harm too much as long as it doesn't cost more than one cacheline.
Have you considered to having fixed vnet header of 64B size for this + any new fields?
Yes, this should work (or even the vnet header length could be negotiated to match the cache line size).
Quoted 3 lines
This eliminates the existing problem where a partial cache line can be written for the vnet header on rx path. Because the data is typically written by one DMA and vnet header as second DMA as they originate from different sources within the device. A device can always do single DMA too to avoid partial cache line write but it does not seem practical enough.
Right. But I meant it needs to be addressed in a separate patch.
Thanks
Quoted 11 lines
From: Jason Wang <[email protected]>
Sent: Friday, February 28, 2025 1:50 PMQuoted 5 lines · Parav Pandit
Quoted 4 lines
From: Shiva Shankar Kommula <[email protected]>
Sent: Thursday, February 27, 2025 3:19 PM
Quoted 8 lines
Quoted 3 lines
Quoted 2 lines
Quoted 2 lines
New fields were added when new features were added. It doesn't harm too much as long as it doesn't cost more than one cacheline.
Have you considered to having fixed vnet header of 64B size for this + any
new fields?
Yes, this should work (or even the vnet header length could be negotiated to match the cache line size).
Ok.
Negotiation needs early admin queue interface patch that I has been pending for a while now. Let me revise that.
Quoted 9 lines
Quoted 1 line
This eliminates the existing problem where a partial cache line can be
written for the vnet header on rx path.
Quoted 1 line
Because the data is typically written by one DMA and vnet header as second
DMA as they originate from different sources within the device.
Quoted 1 line
A device can always do single DMA too to avoid partial cache line write but
it does not seem practical enough.
Right. But I meant it needs to be addressed in a separate patch.
Sounds good.
Thanks
Quoted 1 line
Thanks