virtio-net: support inner header hash
PATCH v7 1/1 virtio-net: support inner header hash
If the tunnel is used to encapsulate the packets, the hash calculated using the outer header of the receive packets is always fixed for the same flow packets, i.e. they will be steered to the same receive queue.
We add a feature bit VIRTIO_NET_F_HASH_TUNNEL and related bitmasks in \field{hash_types}, which instructs the device to calculate the hash using the inner headers of tunnel-encapsulated packets. Besides, values in \field{hash_report_tunnel} are added to report tunnel types.
---
v6:
1. Modify the wording of some sentences for clarity. @Michael S. Tsirkin
2. Fix some syntax issues. @Michael S. Tsirkin
v5:
1. Fix some syntax and capitalization issues. @Michael S. Tsirkin
2. Use encapsulated/encaptulation uniformly. @Michael S. Tsirkin
3. Move the links to introduction section. @Michael S. Tsirkin
4. Clarify some sentences. @Michael S. Tsirkin
v4:
1. Clarify some paragraphs. @Cornelia Huck
2. Fix the u8 type. @Cornelia Huck
v3:
1. Rename VIRTIO_NET_F_HASH_GRE_VXLAN_GENEVE_INNER to VIRTIO_NET_F_HASH_TUNNEL. @Jason Wang
2. Make things clearer. @Jason Wang @Michael S. Tsirkin
3. Keep the possibility to use inner hash for automatic receive steering. @Jason Wang
4. Add the "Tunnel packet" paragraph to avoid repeating the GRE etc. many times. @Michael S. Tsirkin
v2:
1. Add a feature bit for GRE/VXLAN/GENEVE inner hash. @Jason Wang
2. Chang \field{hash_tunnel} to \field{hash_report_tunnel}. @Jason Wang, @Michael S. Tsirkin
v1:
1. Remove the patch for the bitmask fix. @Michael S. Tsirkin
2. Clarify some paragraphs. @Jason Wang
3. Add \field{hash_tunnel} and VIRTIO_NET_HASH_REPORT_GRE. @Yuri Benditovich
content.tex | 191 +++++++++++++++++++++++++++++++++++++++++++++-- introduction.tex | 19 +++++ 2 files changed, 203 insertions(+), 7 deletions(-)
content.tex +184 -7
@@ -3084,6 +3084,9 @@ \subsection{Feature bits}\label{sec:Device Types / Network Device / Feature bits \item[VIRTIO_NET_F_CTRL_MAC_ADDR(23)] Set MAC address through control channel. +\item[VIRTIO_NET_F_HASH_TUNNEL(52)] Device supports inner + header hash for GRE, VXLAN and GENEVE tunnel-encapsulated packets. + \item[VIRTIO_NET_F_NOTF_COAL(53)] Device supports notifications coalescing. \item[VIRTIO_NET_F_GUEST_USO4 (54)] Driver can receive USOv4 packets.
@@ -3095,7 +3098,8 @@ \subsection{Feature bits}\label{sec:Device Types / Network Device / Feature bits to several segments when each of these smaller packets has UDP header. \item[VIRTIO_NET_F_HASH_REPORT(57)] Device can report per-packet hash - value and a type of calculated hash. + value, a type of calculated hash, and, if VIRTIO_NET_F_HASH_TUNNEL + is negotiated, an encapsulation packet type. \item[VIRTIO_NET_F_GUEST_HDRLEN(59)] Driver can provide the exact \field{hdr_len} value. Device benefits from knowing the exact header length.
@@ -3140,6 +3144,7 @@ \subsubsection{Feature bit requirements}\label{sec:Device Types / Network Device \item[VIRTIO_NET_F_NOTF_COAL] Requires VIRTIO_NET_F_CTRL_VQ. \item[VIRTIO_NET_F_RSC_EXT] Requires VIRTIO_NET_F_HOST_TSO4 or VIRTIO_NET_F_HOST_TSO6. \item[VIRTIO_NET_F_RSS] Requires VIRTIO_NET_F_CTRL_VQ. +\item[VIRTIO_NET_F_HASH_TUNNEL] Requires VIRTIO_NET_F_CTRL_VQ. \end{description} \subsubsection{Legacy Interface: Feature bits}\label{sec:Device Types / Network Device / Feature bits / Legacy Interface: Feature bits}
@@ -3386,7 +3391,8 @@ \subsection{Device Operation}\label{sec:Device Types / Network Device / Device O le16 num_buffers; le32 hash_value; (Only if VIRTIO_NET_F_HASH_REPORT negotiated) le16 hash_report; (Only if VIRTIO_NET_F_HASH_REPORT negotiated) - le16 padding_reserved; (Only if VIRTIO_NET_F_HASH_REPORT negotiated) + u8 hash_report_tunnel; (Only if VIRTIO_NET_F_HASH_REPORT negotiated, only valid of VIRTIO_NET_F_HASH_TUNNEL negotiated) + u8 padding_reserved; (Only if VIRTIO_NET_F_HASH_REPORT negotiated) }; \end{lstlisting}
@@ -3837,7 +3843,8 @@ \subsubsection{Processing of Incoming Packets}\label{sec:Device Types / Network A device attempts to calculate a per-packet hash in the following cases: \begin{itemize} \item The feature VIRTIO_NET_F_RSS was negotiated. The device uses the hash to determine the receive virtqueue to place incoming packets. -\item The feature VIRTIO_NET_F_HASH_REPORT was negotiated. The device reports the hash value and the hash type with the packet. +\item The feature VIRTIO_NET_F_HASH_REPORT was negotiated. The device reports the hash value and the hash type. If additionally +VIRTIO_NET_F_HASH_TUNNEL was negotiated, the device reports the encapsulation type as well. \end{itemize} If the feature VIRTIO_NET_F_RSS was negotiated:
@@ -3863,8 +3870,36 @@ \subsubsection{Processing of Incoming Packets}\label{sec:Device Types / Network \ref{sec:Device Types / Network Device / Device Operation / Control Virtqueue / Automatic receive steering in multiqueue mode / Hash calculation}. \end{itemize} +\subparagraph{Tunnel/Encapsulated packet} +\label{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / Tunnel/Encapsulated packet} +A tunnel packet is encapsulated from the original packet based on the tunneling +protocol (only a single level of encapsulation is currently supported). The +encapsulated packet contains an outer header and an inner header, and the device +calculates the hash over either the inner header or the outer header. + +When the feature VIRTIO_NET_F_HASH_TUNNEL is negotiated and the corresponding +encapsulation type is set in \field{hash_types}, the hash for a specific type of +encapsulated packet is calculated over the inner as opposed to outer header. +Supported encapsulation types are listed in \ref{sec:Device Types / Network Device / +Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / +Supported/enabled hash types}. + +If both VIRTIO_NET_F_HASH_REPORT and VIRTIO_NET_F_HASH_TUNNEL are negotiated, +the device can support inner hash calculation for \hyperref[intro:GRE]{[GRE]}, +\hyperref[intro:VXLAN]{[VXLAN]} and \hyperref[intro:GENEVE]{[GENEVE]} +encapsulated packets, and the corresponding encapsulation type in \field{hash_types} +is VIRTIO_NET_HASH_TYPE_{GRE, VXLAN, GENEVE}_INNER respectively. The value in +\field{hash_report_tunnel} is VIRTIO_NET_HASH_REPORT_{NONE, GRE, VXLAN, GENEVE} respectively. + +If VIRTIO_NET_F_HASH_REPORT is negotiated but VIRTIO_NET_F_HASH_TUNNEL is not +negotiated, the device calculates the hash over the outer header, and +\field{hash_report} reports the hash type. \field{hash_report_tunnel} +is no longer valid. + \subparagraph{Supported/enabled hash types} \label{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / Supported/enabled hash types} +This paragraph relies on definitions from \hyperref[intro:IP]{[IP]}, +\hyperref[intro:UDP]{[UDP]} and \hyperref[intro:TCP]{[TCP]}. Hash types applicable for IPv4 packets: \begin{lstlisting} #define VIRTIO_NET_HASH_TYPE_IPv4 (1 << 0)
@@ -3884,6 +3919,22 @@ \subsubsection{Processing of Incoming Packets}\label{sec:Device Types / Network #define VIRTIO_NET_HASH_TYPE_UDP_EX (1 << 8) \end{lstlisting} +If the feature VIRTIO_NET_F_HASH_TUNNEL is negotiated, the encapsulation +hash type below indicates that the hash is calculated over the inner +header of the encapsulated packet: +Hash type applicable for inner payload of the gre-encapsulated packet +\begin{lstlisting} +#define VIRTIO_NET_HASH_TYPE_GRE_INNER (1 << 9) +\end{lstlisting} +Hash type applicable for inner payload of the vxlan-encapsulated packet +\begin{lstlisting} +#define VIRTIO_NET_HASH_TYPE_VXLAN_INNER (1 << 10) +\end{lstlisting} +Hash type applicable for inner payload of the geneve-encapsulated packet +\begin{lstlisting} +#define VIRTIO_NET_HASH_TYPE_GENEVE_INNER (1 << 11) +\end{lstlisting} + \subparagraph{IPv4 packets} \label{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / IPv4 packets} The device calculates the hash on IPv4 packets according to 'Enabled hash types' bitmask as follows:
@@ -3975,15 +4026,125 @@ \subsubsection{Processing of Incoming Packets}\label{sec:Device Types / Network (see \ref{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / IPv6 packets without extension header}). \end{itemize} +\subparagraph{Inner payload of an encapsulated packet} +\label{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / Inner payload of the encapsulated packet} +If the feature VIRTIO_NET_F_HASH_TUNNEL is negotiated and the corresponding +encapsulation hash type is set in \field{hash_types}, the device calculates the +inner header hash on an encapsulated packet (See \ref{sec:Device Types +/ Network Device / Device Operation / Processing of Incoming Packets / +Hash calculation for incoming packets / Tunnel/Encapsulated packet}) as follows: + +The device calculates the hash on the inner IPv4 packet of an encapsulated packet +according to 'Enabled hash types' bitmask as follows: +\begin{itemize} + \item If VIRTIO_NET_HASH_TYPE_TCPv4 is set and the encapsulated packet has an inner + TCPv4 header, the hash is calculated over the following fields: + \begin{itemsize} + \item inner Source IP address + \item inner Destination IP address + \item inner Source TCP port + \item inner Destination TCP port + \end{itemsize} + \item Else if VIRTIO_NET_HASH_TYPE_UDPv4 is set and the encapsulated packet has an + inner UDPv4 header, the hash is calculated over the following fields: + \begin{itemsize} + \item inner Source IP address + \item inner Destination IP address + \item inner Source UDP port + \item inner Destination UDP port + \end{itemize} + \item Else if VIRTIO_NET_HASH_TYPE_IPv4 is set, the hash is calculated over the + following fields: + \begin{itemsize} + \item inner Source IP address + \item inner Destination IP address + \end{itemsize} + \item Else the device does not calculate the hash +\end{itemize} + +The device calculates the hash on the inner IPv6 packet without an extension header +of an encapsulated packet according to 'Enabled hash types' bitmask as follows: +\begin{itemize} + \item If VIRTIO_NET_HASH_TYPE_TCPv6 is set and the encapsulated packet has an inner + TCPv6 header, the hash is calculated over the following fields: + \begin{itemsize} + \item inner Source IPv6 address + \item inner Destination IPv6 address + \item inner Source TCP port + \item inner Destination TCP port + \end{itemsize} + \item Else if VIRTIO_NET_HASH_TYPE_UDPv6 is set and the encapsulated packet has an + inner UDPv6 header, the hash is calculated over the following fields: + \begin{itemsize} + \item inner Source IPv6 address + \item inner Destination IPv6 address + \item inner Source UDP port + \item inner Destination UDP port + \end{itemize} + \item Else if VIRTIO_NET_HASH_TYPE_IPv6 is set, the hash is calculated over the + following fields: + \begin{itemsize} + \item inner Source IPv6 address + \item inner Destination IPv6 address + \end{itemsize} + \item Else the device does not calculate the hash +\end{itemize} + +The device calculates the hash on the inner IPv6 packet with an extension header +of an encapsulated packet according to 'Enabled hash types' bitmask as follows: +\begin{itemsize} + \item If VIRTIO_NET_HASH_TYPE_TCP_EX is set and the encapsulated packet has an inner + TCPv6 header, the hash is calculated over the following fields: + \begin{itemize} + \item Home address from the home address option in the inner IPv6 destination + options header. If the inner extension header is not present, use the + inner Source IPv6 address. + \item IPv6 address that is contained in the Routing-Header-Type-2 from the + associated inner extension header. If the inner extension header is not + present, use the inner Destination IPv6 address. + \item inner Source TCP port + \item inner Destination TCP port + \end{itemize} + \item Else if VIRTIO_NET_HASH_TYPE_UDP_EX is set and the encapsulated packet has an inner + UDPv6 header, the hash is calculated over the following fields: + \begin{itemsize} + \item Home address from the home address option in the inner IPv6 destination + options header. If the inner extension header is not present, use the + inner Source IPv6 address. + \item IPv6 address that is contained in the Routing-Header-Type-2 from the + associated inner extension header. If the inner extension header is not + present, use the inner Destination IPv6 address. + \item inner Source UDP port + \item inner Destination UDP port + \end{itemize} + \item Else if VIRTIO_NET_HASH_TYPE_IP_EX is set, the hash is calculated over the + following fields: + \begin{itemsize} + \item Home address from the home address option in the inner IPv6 destination + options header. If the inner extension header is not present, use the + inner Source IPv6 address. + \item IPv6 address that is contained in the Routing-Header-Type-2 from the + associated inner extension header. If the inner extension header is not + present, use the inner Destination IPv6 address. + \end{itemize} + \item Else skip the inner IPv6 extension header and calculate the inner header hash as + defined for an encapsulated packet whose inner payload is an IPv6 packet without + an extension header. +\end{itemsize} + \paragraph{Hash reporting for incoming packets} \label{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash reporting for incoming packets} -If VIRTIO_NET_F_HASH_REPORT was negotiated and - the device has calculated the hash for the packet, the device fills \field{hash_report} with the report type of calculated hash -and \field{hash_value} with the value of calculated hash. +If VIRTIO_NET_F_HASH_REPORT was negotiated and the device has calculated the +hash for the packet, the device fills \field{hash_report} with the report type +of calculated hash, and \field{hash_value} with the value of calculated hash. +Also, if VIRTIO_NET_F_HASH_TUNNEL was negotiated, the device needs to fill +\field{hash_report_tunnel} with the report type of the encapsulated packet, and +it is set to VIRTIO_NET_HASH_REPORT_TUNNEL_NONE for the unencapsulated packet. If VIRTIO_NET_F_HASH_REPORT was negotiated but due to any reason the -hash was not calculated, the device sets \field{hash_report} to VIRTIO_NET_HASH_REPORT_NONE. +hash was not calculated, the device sets \field{hash_report} to VIRTIO_NET_HASH_REPORT_NONE, +and sets \field{hash_report_tunnel} to VIRTIO_NET_HASH_REPORT_TUNNEL_NONE. Possible values that the device can report in \field{hash_report} are defined below. They correspond to supported hash types defined in
@@ -4005,6 +4166,22 @@ \subsubsection{Processing of Incoming Packets}\label{sec:Device Types / Network #define VIRTIO_NET_HASH_REPORT_UDPv6_EX 9 \end{lstlisting} +If \field{hash_report} differs from VIRTIO_NET_HASH_REPORT_NONE, +\field{hash_report_tunnel} can report the type of the encapsulated +packet to the driver over the inner header hash calculation. +Possible values that the device can report in \field{hash_report_tunnel} +are defined below: + +\begin{lstlisting} +#define VIRTIO_NET_HASH_REPORT_TUNNEL_NONE 0 +#define VIRTIO_NET_HASH_REPORT_GRE 1 +#define VIRTIO_NET_HASH_REPORT_VXLAN 2 +#define VIRTIO_NET_HASH_REPORT_GENEVE 3 +\end{lstlisting} + +They correspond to supported hash types defined in +\ref{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / Supported/enabled hash types}. + \subsubsection{Control Virtqueue}\label{sec:Device Types / Network Device / Device Operation / Control Virtqueue} The driver uses the control virtqueue (if VIRTIO_NET_F_CTRL_VQ is
introduction.tex +19 -0
@@ -98,6 +98,25 @@ \section{Normative References}\label{sec:Normative References} \phantomsection\label{intro:SEC1}\textbf{[SEC1]} & Standards for Efficient Cryptography Group(SECG), ``SEC1: Elliptic Cureve Cryptography'', Version 1.0, September 2000. \newline\url{https://www.secg.org/sec1-v2.pdf}\\ + \phantomsection\label{intro:GRE}\textbf{[GRE]} & + Generic Routing Encapsulation + \newline\url{https://datatracker.ietf.org/doc/rfc2784/}\\ + \phantomsection\label{intro:VXLAN}\textbf{[VXLAN]} & + Virtual eXtensible Local Area Network + \newline\url{https://datatracker.ietf.org/doc/rfc7348/}\\ + \phantomsection\label{intro:GENEVE}\textbf{[GENEVE]} & + Generic Network Virtualization Encapsulation + \newline\url{https://datatracker.ietf.org/doc/rfc8926/}\\ + \phantomsection\label{intro:IP}\textbf{[IP]} & + INTERNET PROTOCOL + \newline\url{https://www.rfc-editor.org/rfc/rfc791}\\ + \phantomsection\label{intro:UDP}\textbf{[UDP]} & + User Datagram Protocol + \newline\url{https://www.rfc-editor.org/rfc/rfc768}\\ + \phantomsection\label{intro:TCP}\textbf{[TCP]} & + TRANSMISSION CONTROL PROTOCOL + \newline\url{https://www.rfc-editor.org/rfc/rfc793}\\ + \end{longtable}
Signature · 1 line
2.19.1.6.gb485710b
Hi Michael, do you have any other comments?
Thanks.
Quoted 343 lines · Heng Qi
If the tunnel is used to encapsulate the packets, the hash calculated using the outer header of the receive packets is always fixed for the same flow packets, i.e. they will be steered to the same receive queue.
We add a feature bit
VIRTIO_NET_F_HASH_TUNNELand related bitmasks in \field{hash_types}, which instructs the device to calculate the hash using the inner headers of tunnel-encapsulated packets. Besides, values in \field{hash_report_tunnel} are added to report tunnel types.Reviewed-by Jason Wang <[email protected]>Signed-off-by Heng Qi <[email protected]>Signed-off-by Xuan Zhuo <[email protected]>---
v6:
1. Modify the wording of some sentences for clarity. @Michael S. Tsirkin
2. Fix some syntax issues. @Michael S. Tsirkinv5:
1. Fix some syntax and capitalization issues. @Michael S. Tsirkin
2. Use encapsulated/encaptulation uniformly. @Michael S. Tsirkin
3. Move the links to introduction section. @Michael S. Tsirkin
4. Clarify some sentences. @Michael S. Tsirkinv4:
1. Clarify some paragraphs. @Cornelia Huck
2. Fix the u8 type. @Cornelia Huckv3:
1. RenameVIRTIO_NET_F_HASH_GRE_VXLAN_GENEVE_INNERtoVIRTIO_NET_F_HASH_TUNNEL. @Jason Wang
2. Make things clearer. @Jason Wang @Michael S. Tsirkin
3. Keep the possibility to use inner hash for automatic receive steering. @Jason Wang
4. Add the "Tunnel packet" paragraph to avoid repeating the GRE etc. many times. @Michael S. Tsirkinv2:
1. Add a feature bit for GRE/VXLAN/GENEVE inner hash. @Jason Wang
2. Chang \field{hash_tunnel} to \field{hash_report_tunnel}. @Jason Wang, @Michael S. Tsirkinv1:
1. Remove the patch for the bitmask fix. @Michael S. Tsirkin
2. Clarify some paragraphs. @Jason Wang
3. Add \field{hash_tunnel} andVIRTIO_NET_HASH_REPORT_GRE. @Yuri Benditovichcontent.tex | 191 +++++++++++++++++++++++++++++++++++++++++++++-- introduction.tex | 19 +++++ 2 files changed, 203 insertions(+), 7 deletions(-)content.tex +184
-7@@ -3084,6 +3084,9 @@ \subsection{Feature bits}\label{sec:Device Types / Network Device / Feature bits \item[VIRTIO_NET_F_CTRL_MAC_ADDR(23)] Set MAC address through control channel. +\item[VIRTIO_NET_F_HASH_TUNNEL(52)] Device supports inner + header hash for GRE, VXLAN and GENEVE tunnel-encapsulated packets. + \item[VIRTIO_NET_F_NOTF_COAL(53)] Device supports notifications coalescing. \item[VIRTIO_NET_F_GUEST_USO4 (54)] Driver can receive USOv4 packets.@@ -3095,7 +3098,8 @@ \subsection{Feature bits}\label{sec:Device Types / Network Device / Feature bits to several segments when each of these smaller packets has UDP header. \item[VIRTIO_NET_F_HASH_REPORT(57)] Device can report per-packet hash - value and a type of calculated hash. + value, a type of calculated hash, and, if VIRTIO_NET_F_HASH_TUNNEL + is negotiated, an encapsulation packet type. \item[VIRTIO_NET_F_GUEST_HDRLEN(59)] Driver can provide the exact \field{hdr_len} value. Device benefits from knowing the exact header length.@@ -3140,6 +3144,7 @@ \subsubsection{Feature bit requirements}\label{sec:Device Types / Network Device \item[VIRTIO_NET_F_NOTF_COAL] Requires VIRTIO_NET_F_CTRL_VQ. \item[VIRTIO_NET_F_RSC_EXT] Requires VIRTIO_NET_F_HOST_TSO4 or VIRTIO_NET_F_HOST_TSO6. \item[VIRTIO_NET_F_RSS] Requires VIRTIO_NET_F_CTRL_VQ. +\item[VIRTIO_NET_F_HASH_TUNNEL] Requires VIRTIO_NET_F_CTRL_VQ. \end{description} \subsubsection{Legacy Interface: Feature bits}\label{sec:Device Types / Network Device / Feature bits / Legacy Interface: Feature bits}@@ -3386,7 +3391,8 @@ \subsection{Device Operation}\label{sec:Device Types / Network Device / Device O le16 num_buffers; le32 hash_value; (Only if VIRTIO_NET_F_HASH_REPORT negotiated) le16 hash_report; (Only if VIRTIO_NET_F_HASH_REPORT negotiated) - le16 padding_reserved; (Only if VIRTIO_NET_F_HASH_REPORT negotiated) + u8 hash_report_tunnel; (Only if VIRTIO_NET_F_HASH_REPORT negotiated, only valid of VIRTIO_NET_F_HASH_TUNNEL negotiated) + u8 padding_reserved; (Only if VIRTIO_NET_F_HASH_REPORT negotiated) }; \end{lstlisting}@@ -3837,7 +3843,8 @@ \subsubsection{Processing of Incoming Packets}\label{sec:Device Types / Network A device attempts to calculate a per-packet hash in the following cases: \begin{itemize} \item The feature VIRTIO_NET_F_RSS was negotiated. The device uses the hash to determine the receive virtqueue to place incoming packets. -\item The feature VIRTIO_NET_F_HASH_REPORT was negotiated. The device reports the hash value and the hash type with the packet. +\item The feature VIRTIO_NET_F_HASH_REPORT was negotiated. The device reports the hash value and the hash type. If additionally +VIRTIO_NET_F_HASH_TUNNEL was negotiated, the device reports the encapsulation type as well. \end{itemize} If the feature VIRTIO_NET_F_RSS was negotiated:@@ -3863,8 +3870,36 @@ \subsubsection{Processing of Incoming Packets}\label{sec:Device Types / Network \ref{sec:Device Types / Network Device / Device Operation / Control Virtqueue / Automatic receive steering in multiqueue mode / Hash calculation}. \end{itemize} +\subparagraph{Tunnel/Encapsulated packet} +\label{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / Tunnel/Encapsulated packet} +A tunnel packet is encapsulated from the original packet based on the tunneling +protocol (only a single level of encapsulation is currently supported). The +encapsulated packet contains an outer header and an inner header, and the device +calculates the hash over either the inner header or the outer header. + +When the feature VIRTIO_NET_F_HASH_TUNNEL is negotiated and the corresponding +encapsulation type is set in \field{hash_types}, the hash for a specific type of +encapsulated packet is calculated over the inner as opposed to outer header. +Supported encapsulation types are listed in \ref{sec:Device Types / Network Device / +Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / +Supported/enabled hash types}. + +If both VIRTIO_NET_F_HASH_REPORT and VIRTIO_NET_F_HASH_TUNNEL are negotiated, +the device can support inner hash calculation for \hyperref[intro:GRE]{[GRE]}, +\hyperref[intro:VXLAN]{[VXLAN]} and \hyperref[intro:GENEVE]{[GENEVE]} +encapsulated packets, and the corresponding encapsulation type in \field{hash_types} +is VIRTIO_NET_HASH_TYPE_{GRE, VXLAN, GENEVE}_INNER respectively. The value in +\field{hash_report_tunnel} is VIRTIO_NET_HASH_REPORT_{NONE, GRE, VXLAN, GENEVE} respectively. + +If VIRTIO_NET_F_HASH_REPORT is negotiated but VIRTIO_NET_F_HASH_TUNNEL is not +negotiated, the device calculates the hash over the outer header, and +\field{hash_report} reports the hash type. \field{hash_report_tunnel} +is no longer valid. + \subparagraph{Supported/enabled hash types} \label{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / Supported/enabled hash types} +This paragraph relies on definitions from \hyperref[intro:IP]{[IP]}, +\hyperref[intro:UDP]{[UDP]} and \hyperref[intro:TCP]{[TCP]}. Hash types applicable for IPv4 packets: \begin{lstlisting} #define VIRTIO_NET_HASH_TYPE_IPv4 (1 << 0)@@ -3884,6 +3919,22 @@ \subsubsection{Processing of Incoming Packets}\label{sec:Device Types / Network #define VIRTIO_NET_HASH_TYPE_UDP_EX (1 << 8) \end{lstlisting} +If the feature VIRTIO_NET_F_HASH_TUNNEL is negotiated, the encapsulation +hash type below indicates that the hash is calculated over the inner +header of the encapsulated packet: +Hash type applicable for inner payload of the gre-encapsulated packet +\begin{lstlisting} +#define VIRTIO_NET_HASH_TYPE_GRE_INNER (1 << 9) +\end{lstlisting} +Hash type applicable for inner payload of the vxlan-encapsulated packet +\begin{lstlisting} +#define VIRTIO_NET_HASH_TYPE_VXLAN_INNER (1 << 10) +\end{lstlisting} +Hash type applicable for inner payload of the geneve-encapsulated packet +\begin{lstlisting} +#define VIRTIO_NET_HASH_TYPE_GENEVE_INNER (1 << 11) +\end{lstlisting} + \subparagraph{IPv4 packets} \label{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / IPv4 packets} The device calculates the hash on IPv4 packets according to 'Enabled hash types' bitmask as follows:@@ -3975,15 +4026,125 @@ \subsubsection{Processing of Incoming Packets}\label{sec:Device Types / Network (see \ref{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / IPv6 packets without extension header}). \end{itemize} +\subparagraph{Inner payload of an encapsulated packet} +\label{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / Inner payload of the encapsulated packet} +If the feature VIRTIO_NET_F_HASH_TUNNEL is negotiated and the corresponding +encapsulation hash type is set in \field{hash_types}, the device calculates the +inner header hash on an encapsulated packet (See \ref{sec:Device Types +/ Network Device / Device Operation / Processing of Incoming Packets / +Hash calculation for incoming packets / Tunnel/Encapsulated packet}) as follows: + +The device calculates the hash on the inner IPv4 packet of an encapsulated packet +according to 'Enabled hash types' bitmask as follows: +\begin{itemize} + \item If VIRTIO_NET_HASH_TYPE_TCPv4 is set and the encapsulated packet has an inner + TCPv4 header, the hash is calculated over the following fields: + \begin{itemsize} + \item inner Source IP address + \item inner Destination IP address + \item inner Source TCP port + \item inner Destination TCP port + \end{itemsize} + \item Else if VIRTIO_NET_HASH_TYPE_UDPv4 is set and the encapsulated packet has an + inner UDPv4 header, the hash is calculated over the following fields: + \begin{itemsize} + \item inner Source IP address + \item inner Destination IP address + \item inner Source UDP port + \item inner Destination UDP port + \end{itemize} + \item Else if VIRTIO_NET_HASH_TYPE_IPv4 is set, the hash is calculated over the + following fields: + \begin{itemsize} + \item inner Source IP address + \item inner Destination IP address + \end{itemsize} + \item Else the device does not calculate the hash +\end{itemize} + +The device calculates the hash on the inner IPv6 packet without an extension header +of an encapsulated packet according to 'Enabled hash types' bitmask as follows: +\begin{itemize} + \item If VIRTIO_NET_HASH_TYPE_TCPv6 is set and the encapsulated packet has an inner + TCPv6 header, the hash is calculated over the following fields: + \begin{itemsize} + \item inner Source IPv6 address + \item inner Destination IPv6 address + \item inner Source TCP port + \item inner Destination TCP port + \end{itemsize} + \item Else if VIRTIO_NET_HASH_TYPE_UDPv6 is set and the encapsulated packet has an + inner UDPv6 header, the hash is calculated over the following fields: + \begin{itemsize} + \item inner Source IPv6 address + \item inner Destination IPv6 address + \item inner Source UDP port + \item inner Destination UDP port + \end{itemize} + \item Else if VIRTIO_NET_HASH_TYPE_IPv6 is set, the hash is calculated over the + following fields: + \begin{itemsize} + \item inner Source IPv6 address + \item inner Destination IPv6 address + \end{itemsize} + \item Else the device does not calculate the hash +\end{itemize} + +The device calculates the hash on the inner IPv6 packet with an extension header +of an encapsulated packet according to 'Enabled hash types' bitmask as follows: +\begin{itemsize} + \item If VIRTIO_NET_HASH_TYPE_TCP_EX is set and the encapsulated packet has an inner + TCPv6 header, the hash is calculated over the following fields: + \begin{itemize} + \item Home address from the home address option in the inner IPv6 destination + options header. If the inner extension header is not present, use the + inner Source IPv6 address. + \item IPv6 address that is contained in the Routing-Header-Type-2 from the + associated inner extension header. If the inner extension header is not + present, use the inner Destination IPv6 address. + \item inner Source TCP port + \item inner Destination TCP port + \end{itemize} + \item Else if VIRTIO_NET_HASH_TYPE_UDP_EX is set and the encapsulated packet has an inner + UDPv6 header, the hash is calculated over the following fields: + \begin{itemsize} + \item Home address from the home address option in the inner IPv6 destination + options header. If the inner extension header is not present, use the + inner Source IPv6 address. + \item IPv6 address that is contained in the Routing-Header-Type-2 from the + associated inner extension header. If the inner extension header is not + present, use the inner Destination IPv6 address. + \item inner Source UDP port + \item inner Destination UDP port + \end{itemize} + \item Else if VIRTIO_NET_HASH_TYPE_IP_EX is set, the hash is calculated over the + following fields: + \begin{itemsize} + \item Home address from the home address option in the inner IPv6 destination + options header. If the inner extension header is not present, use the + inner Source IPv6 address. + \item IPv6 address that is contained in the Routing-Header-Type-2 from the + associated inner extension header. If the inner extension header is not + present, use the inner Destination IPv6 address. + \end{itemize} + \item Else skip the inner IPv6 extension header and calculate the inner header hash as + defined for an encapsulated packet whose inner payload is an IPv6 packet without + an extension header. +\end{itemsize} + \paragraph{Hash reporting for incoming packets} \label{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash reporting for incoming packets} -If VIRTIO_NET_F_HASH_REPORT was negotiated and - the device has calculated the hash for the packet, the device fills \field{hash_report} with the report type of calculated hash -and \field{hash_value} with the value of calculated hash. +If VIRTIO_NET_F_HASH_REPORT was negotiated and the device has calculated the +hash for the packet, the device fills \field{hash_report} with the report type +of calculated hash, and \field{hash_value} with the value of calculated hash. +Also, if VIRTIO_NET_F_HASH_TUNNEL was negotiated, the device needs to fill +\field{hash_report_tunnel} with the report type of the encapsulated packet, and +it is set to VIRTIO_NET_HASH_REPORT_TUNNEL_NONE for the unencapsulated packet. If VIRTIO_NET_F_HASH_REPORT was negotiated but due to any reason the -hash was not calculated, the device sets \field{hash_report} to VIRTIO_NET_HASH_REPORT_NONE. +hash was not calculated, the device sets \field{hash_report} to VIRTIO_NET_HASH_REPORT_NONE, +and sets \field{hash_report_tunnel} to VIRTIO_NET_HASH_REPORT_TUNNEL_NONE. Possible values that the device can report in \field{hash_report} are defined below. They correspond to supported hash types defined in@@ -4005,6 +4166,22 @@ \subsubsection{Processing of Incoming Packets}\label{sec:Device Types / Network #define VIRTIO_NET_HASH_REPORT_UDPv6_EX 9 \end{lstlisting} +If \field{hash_report} differs from VIRTIO_NET_HASH_REPORT_NONE, +\field{hash_report_tunnel} can report the type of the encapsulated +packet to the driver over the inner header hash calculation. +Possible values that the device can report in \field{hash_report_tunnel} +are defined below: + +\begin{lstlisting} +#define VIRTIO_NET_HASH_REPORT_TUNNEL_NONE 0 +#define VIRTIO_NET_HASH_REPORT_GRE 1 +#define VIRTIO_NET_HASH_REPORT_VXLAN 2 +#define VIRTIO_NET_HASH_REPORT_GENEVE 3 +\end{lstlisting} + +They correspond to supported hash types defined in +\ref{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / Supported/enabled hash types}. + \subsubsection{Control Virtqueue}\label{sec:Device Types / Network Device / Device Operation / Control Virtqueue} The driver uses the control virtqueue (if VIRTIO_NET_F_CTRL_VQ isintroduction.tex +19
-0@@ -98,6 +98,25 @@ \section{Normative References}\label{sec:Normative References} \phantomsection\label{intro:SEC1}\textbf{[SEC1]} & Standards for Efficient Cryptography Group(SECG), ``SEC1: Elliptic Cureve Cryptography'', Version 1.0, September 2000. \newline\url{https://www.secg.org/sec1-v2.pdf}\\ + \phantomsection\label{intro:GRE}\textbf{[GRE]} & + Generic Routing Encapsulation + \newline\url{https://datatracker.ietf.org/doc/rfc2784/}\\ + \phantomsection\label{intro:VXLAN}\textbf{[VXLAN]} & + Virtual eXtensible Local Area Network + \newline\url{https://datatracker.ietf.org/doc/rfc7348/}\\ + \phantomsection\label{intro:GENEVE}\textbf{[GENEVE]} & + Generic Network Virtualization Encapsulation + \newline\url{https://datatracker.ietf.org/doc/rfc8926/}\\ + \phantomsection\label{intro:IP}\textbf{[IP]} & + INTERNET PROTOCOL + \newline\url{https://www.rfc-editor.org/rfc/rfc791}\\ + \phantomsection\label{intro:UDP}\textbf{[UDP]} & + User Datagram Protocol + \newline\url{https://www.rfc-editor.org/rfc/rfc768}\\ + \phantomsection\label{intro:TCP}\textbf{[TCP]} & + TRANSMISSION CONTROL PROTOCOL + \newline\url{https://www.rfc-editor.org/rfc/rfc793}\\ + \end{longtable}
Quoted 4 lines · Heng Qi
Hi Michael, do you have any other comments?
Thanks.
Sit tight pls, you only posted this 5 hours ago.
Signature · 1 line
MST
Quoted 9 lines · Heng Qi
If the tunnel is used to encapsulate the packets, the hash calculated using the outer header of the receive packets is always fixed for the same flow packets, i.e. they will be steered to the same receive queue.
We add a feature bit
VIRTIO_NET_F_HASH_TUNNELand related bitmasks in \field{hash_types}, which instructs the device to calculate the hash using the inner headers of tunnel-encapsulated packets. Besides, values in \field{hash_report_tunnel} are added to report tunnel types.
Quoted 5 lines
Reviewed-by Jason Wang <[email protected]>Signed-off-by Heng Qi <[email protected]>Signed-off-by Xuan Zhuo <[email protected]>
ok close to being ready. a couple of minor comments.
Quoted 39 lines
---
v6:
1. Modify the wording of some sentences for clarity. @Michael S. Tsirkin
2. Fix some syntax issues. @Michael S. Tsirkinv5:
1. Fix some syntax and capitalization issues. @Michael S. Tsirkin
2. Use encapsulated/encaptulation uniformly. @Michael S. Tsirkin
3. Move the links to introduction section. @Michael S. Tsirkin
4. Clarify some sentences. @Michael S. Tsirkinv4:
1. Clarify some paragraphs. @Cornelia Huck
2. Fix the u8 type. @Cornelia Huckv3:
1. RenameVIRTIO_NET_F_HASH_GRE_VXLAN_GENEVE_INNERtoVIRTIO_NET_F_HASH_TUNNEL. @Jason Wang
2. Make things clearer. @Jason Wang @Michael S. Tsirkin
3. Keep the possibility to use inner hash for automatic receive steering. @Jason Wang
4. Add the "Tunnel packet" paragraph to avoid repeating the GRE etc. many times. @Michael S. Tsirkinv2:
1. Add a feature bit for GRE/VXLAN/GENEVE inner hash. @Jason Wang
2. Chang \field{hash_tunnel} to \field{hash_report_tunnel}. @Jason Wang, @Michael S. Tsirkinv1:
1. Remove the patch for the bitmask fix. @Michael S. Tsirkin
2. Clarify some paragraphs. @Jason Wang
3. Add \field{hash_tunnel} andVIRTIO_NET_HASH_REPORT_GRE. @Yuri Benditovichcontent.tex | 191 +++++++++++++++++++++++++++++++++++++++++++++-- introduction.tex | 19 +++++ 2 files changed, 203 insertions(+), 7 deletions(-)content.tex +0
-0
content.tex \subsection{Feature bits}\label{sec:Device Types / Network Device / Feature bits · line 3088
Quoted 5 lines
content.tex +2
-0@@ -3084,6 +3084,9 @@ \subsection{Feature bits}\label{sec:Device Types / Network Device / Feature bits \item[VIRTIO_NET_F_CTRL_MAC_ADDR(23)] Set MAC address through control channel. +\item[VIRTIO_NET_F_HASH_TUNNEL(52)] Device supports inner + header hash for GRE, VXLAN and GENEVE tunnel-encapsulated packets.
I would probably drop the list of tunnel types here.
Quoted 212 lines
content.tex +158
-3+ \item[VIRTIO_NET_F_NOTF_COAL(53)] Device supports notifications coalescing. \item[VIRTIO_NET_F_GUEST_USO4 (54)] Driver can receive USOv4 packets.@@ -3095,7 +3098,8 @@ \subsection{Feature bits}\label{sec:Device Types / Network Device / Feature bits to several segments when each of these smaller packets has UDP header. \item[VIRTIO_NET_F_HASH_REPORT(57)] Device can report per-packet hash - value and a type of calculated hash. + value, a type of calculated hash, and, if VIRTIO_NET_F_HASH_TUNNEL + is negotiated, an encapsulation packet type. \item[VIRTIO_NET_F_GUEST_HDRLEN(59)] Driver can provide the exact \field{hdr_len} value. Device benefits from knowing the exact header length.@@ -3140,6 +3144,7 @@ \subsubsection{Feature bit requirements}\label{sec:Device Types / Network Device \item[VIRTIO_NET_F_NOTF_COAL] Requires VIRTIO_NET_F_CTRL_VQ. \item[VIRTIO_NET_F_RSC_EXT] Requires VIRTIO_NET_F_HOST_TSO4 or VIRTIO_NET_F_HOST_TSO6. \item[VIRTIO_NET_F_RSS] Requires VIRTIO_NET_F_CTRL_VQ. +\item[VIRTIO_NET_F_HASH_TUNNEL] Requires VIRTIO_NET_F_CTRL_VQ. \end{description} \subsubsection{Legacy Interface: Feature bits}\label{sec:Device Types / Network Device / Feature bits / Legacy Interface: Feature bits}@@ -3386,7 +3391,8 @@ \subsection{Device Operation}\label{sec:Device Types / Network Device / Device O le16 num_buffers; le32 hash_value; (Only if VIRTIO_NET_F_HASH_REPORT negotiated) le16 hash_report; (Only if VIRTIO_NET_F_HASH_REPORT negotiated) - le16 padding_reserved; (Only if VIRTIO_NET_F_HASH_REPORT negotiated) + u8 hash_report_tunnel; (Only if VIRTIO_NET_F_HASH_REPORT negotiated, only valid of VIRTIO_NET_F_HASH_TUNNEL negotiated) + u8 padding_reserved; (Only if VIRTIO_NET_F_HASH_REPORT negotiated) }; \end{lstlisting}@@ -3837,7 +3843,8 @@ \subsubsection{Processing of Incoming Packets}\label{sec:Device Types / Network A device attempts to calculate a per-packet hash in the following cases: \begin{itemize} \item The feature VIRTIO_NET_F_RSS was negotiated. The device uses the hash to determine the receive virtqueue to place incoming packets. -\item The feature VIRTIO_NET_F_HASH_REPORT was negotiated. The device reports the hash value and the hash type with the packet. +\item The feature VIRTIO_NET_F_HASH_REPORT was negotiated. The device reports the hash value and the hash type. If additionally +VIRTIO_NET_F_HASH_TUNNEL was negotiated, the device reports the encapsulation type as well. \end{itemize} If the feature VIRTIO_NET_F_RSS was negotiated:@@ -3863,8 +3870,36 @@ \subsubsection{Processing of Incoming Packets}\label{sec:Device Types / Network \ref{sec:Device Types / Network Device / Device Operation / Control Virtqueue / Automatic receive steering in multiqueue mode / Hash calculation}. \end{itemize} +\subparagraph{Tunnel/Encapsulated packet} +\label{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / Tunnel/Encapsulated packet} +A tunnel packet is encapsulated from the original packet based on the tunneling +protocol (only a single level of encapsulation is currently supported). The +encapsulated packet contains an outer header and an inner header, and the device +calculates the hash over either the inner header or the outer header. + +When the feature VIRTIO_NET_F_HASH_TUNNEL is negotiated and the corresponding +encapsulation type is set in \field{hash_types}, the hash for a specific type of +encapsulated packet is calculated over the inner as opposed to outer header. +Supported encapsulation types are listed in \ref{sec:Device Types / Network Device / +Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / +Supported/enabled hash types}. + +If both VIRTIO_NET_F_HASH_REPORT and VIRTIO_NET_F_HASH_TUNNEL are negotiated, +the device can support inner hash calculation for \hyperref[intro:GRE]{[GRE]}, +\hyperref[intro:VXLAN]{[VXLAN]} and \hyperref[intro:GENEVE]{[GENEVE]} +encapsulated packets, and the corresponding encapsulation type in \field{hash_types} +is VIRTIO_NET_HASH_TYPE_{GRE, VXLAN, GENEVE}_INNER respectively. The value in +\field{hash_report_tunnel} is VIRTIO_NET_HASH_REPORT_{NONE, GRE, VXLAN, GENEVE} respectively. + +If VIRTIO_NET_F_HASH_REPORT is negotiated but VIRTIO_NET_F_HASH_TUNNEL is not +negotiated, the device calculates the hash over the outer header, and +\field{hash_report} reports the hash type. \field{hash_report_tunnel} +is no longer valid. + \subparagraph{Supported/enabled hash types} \label{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / Supported/enabled hash types} +This paragraph relies on definitions from \hyperref[intro:IP]{[IP]}, +\hyperref[intro:UDP]{[UDP]} and \hyperref[intro:TCP]{[TCP]}. Hash types applicable for IPv4 packets: \begin{lstlisting} #define VIRTIO_NET_HASH_TYPE_IPv4 (1 << 0)@@ -3884,6 +3919,22 @@ \subsubsection{Processing of Incoming Packets}\label{sec:Device Types / Network #define VIRTIO_NET_HASH_TYPE_UDP_EX (1 << 8) \end{lstlisting} +If the feature VIRTIO_NET_F_HASH_TUNNEL is negotiated, the encapsulation +hash type below indicates that the hash is calculated over the inner +header of the encapsulated packet: +Hash type applicable for inner payload of the gre-encapsulated packet +\begin{lstlisting} +#define VIRTIO_NET_HASH_TYPE_GRE_INNER (1 << 9) +\end{lstlisting} +Hash type applicable for inner payload of the vxlan-encapsulated packet +\begin{lstlisting} +#define VIRTIO_NET_HASH_TYPE_VXLAN_INNER (1 << 10) +\end{lstlisting} +Hash type applicable for inner payload of the geneve-encapsulated packet +\begin{lstlisting} +#define VIRTIO_NET_HASH_TYPE_GENEVE_INNER (1 << 11) +\end{lstlisting} + \subparagraph{IPv4 packets} \label{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / IPv4 packets} The device calculates the hash on IPv4 packets according to 'Enabled hash types' bitmask as follows:@@ -3975,15 +4026,125 @@ \subsubsection{Processing of Incoming Packets}\label{sec:Device Types / Network (see \ref{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / IPv6 packets without extension header}). \end{itemize} +\subparagraph{Inner payload of an encapsulated packet} +\label{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / Inner payload of the encapsulated packet} +If the feature VIRTIO_NET_F_HASH_TUNNEL is negotiated and the corresponding +encapsulation hash type is set in \field{hash_types}, the device calculates the +inner header hash on an encapsulated packet (See \ref{sec:Device Types +/ Network Device / Device Operation / Processing of Incoming Packets / +Hash calculation for incoming packets / Tunnel/Encapsulated packet}) as follows: + +The device calculates the hash on the inner IPv4 packet of an encapsulated packet +according to 'Enabled hash types' bitmask as follows: +\begin{itemize} + \item If VIRTIO_NET_HASH_TYPE_TCPv4 is set and the encapsulated packet has an inner + TCPv4 header, the hash is calculated over the following fields: + \begin{itemsize} + \item inner Source IP address + \item inner Destination IP address + \item inner Source TCP port + \item inner Destination TCP port + \end{itemsize} + \item Else if VIRTIO_NET_HASH_TYPE_UDPv4 is set and the encapsulated packet has an + inner UDPv4 header, the hash is calculated over the following fields: + \begin{itemsize} + \item inner Source IP address + \item inner Destination IP address + \item inner Source UDP port + \item inner Destination UDP port + \end{itemize} + \item Else if VIRTIO_NET_HASH_TYPE_IPv4 is set, the hash is calculated over the + following fields: + \begin{itemsize} + \item inner Source IP address + \item inner Destination IP address + \end{itemsize} + \item Else the device does not calculate the hash +\end{itemize} + +The device calculates the hash on the inner IPv6 packet without an extension header +of an encapsulated packet according to 'Enabled hash types' bitmask as follows: +\begin{itemize} + \item If VIRTIO_NET_HASH_TYPE_TCPv6 is set and the encapsulated packet has an inner + TCPv6 header, the hash is calculated over the following fields: + \begin{itemsize} + \item inner Source IPv6 address + \item inner Destination IPv6 address + \item inner Source TCP port + \item inner Destination TCP port + \end{itemsize} + \item Else if VIRTIO_NET_HASH_TYPE_UDPv6 is set and the encapsulated packet has an + inner UDPv6 header, the hash is calculated over the following fields: + \begin{itemsize} + \item inner Source IPv6 address + \item inner Destination IPv6 address + \item inner Source UDP port + \item inner Destination UDP port + \end{itemize} + \item Else if VIRTIO_NET_HASH_TYPE_IPv6 is set, the hash is calculated over the + following fields: + \begin{itemsize} + \item inner Source IPv6 address + \item inner Destination IPv6 address + \end{itemsize} + \item Else the device does not calculate the hash +\end{itemize} + +The device calculates the hash on the inner IPv6 packet with an extension header +of an encapsulated packet according to 'Enabled hash types' bitmask as follows: +\begin{itemsize} + \item If VIRTIO_NET_HASH_TYPE_TCP_EX is set and the encapsulated packet has an inner + TCPv6 header, the hash is calculated over the following fields: + \begin{itemize} + \item Home address from the home address option in the inner IPv6 destination + options header. If the inner extension header is not present, use the + inner Source IPv6 address. + \item IPv6 address that is contained in the Routing-Header-Type-2 from the + associated inner extension header. If the inner extension header is not + present, use the inner Destination IPv6 address. + \item inner Source TCP port + \item inner Destination TCP port + \end{itemize} + \item Else if VIRTIO_NET_HASH_TYPE_UDP_EX is set and the encapsulated packet has an inner + UDPv6 header, the hash is calculated over the following fields: + \begin{itemsize} + \item Home address from the home address option in the inner IPv6 destination + options header. If the inner extension header is not present, use the + inner Source IPv6 address. + \item IPv6 address that is contained in the Routing-Header-Type-2 from the + associated inner extension header. If the inner extension header is not + present, use the inner Destination IPv6 address. + \item inner Source UDP port + \item inner Destination UDP port + \end{itemize} + \item Else if VIRTIO_NET_HASH_TYPE_IP_EX is set, the hash is calculated over the + following fields: + \begin{itemsize} + \item Home address from the home address option in the inner IPv6 destination + options header. If the inner extension header is not present, use the + inner Source IPv6 address. + \item IPv6 address that is contained in the Routing-Header-Type-2 from the + associated inner extension header. If the inner extension header is not + present, use the inner Destination IPv6 address. + \end{itemize} + \item Else skip the inner IPv6 extension header and calculate the inner header hash as + defined for an encapsulated packet whose inner payload is an IPv6 packet without + an extension header. +\end{itemsize} +
content.tex \subsubsection{Processing of Incoming Packets}\label{sec:Device Types / Network · line 4141
Quoted 10 lines
content.tex +4
-3\paragraph{Hash reporting for incoming packets} \label{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash reporting for incoming packets} -If VIRTIO_NET_F_HASH_REPORT was negotiated and - the device has calculated the hash for the packet, the device fills \field{hash_report} with the report type of calculated hash -and \field{hash_value} with the value of calculated hash. +If VIRTIO_NET_F_HASH_REPORT was negotiated and the device has calculated the +hash for the packet, the device fills \field{hash_report} with the report type +of calculated hash, and \field{hash_value} with the value of calculated hash. +Also, if VIRTIO_NET_F_HASH_TUNNEL was negotiated, the device needs to fill
needs to fill -> fills
for consistency
Quoted 65 lines
content.tex +20
-1+\field{hash_report_tunnel} with the report type of the encapsulated packet, and +it is set to VIRTIO_NET_HASH_REPORT_TUNNEL_NONE for the unencapsulated packet. If VIRTIO_NET_F_HASH_REPORT was negotiated but due to any reason the -hash was not calculated, the device sets \field{hash_report} to VIRTIO_NET_HASH_REPORT_NONE. +hash was not calculated, the device sets \field{hash_report} to VIRTIO_NET_HASH_REPORT_NONE, +and sets \field{hash_report_tunnel} to VIRTIO_NET_HASH_REPORT_TUNNEL_NONE. Possible values that the device can report in \field{hash_report} are defined below. They correspond to supported hash types defined in@@ -4005,6 +4166,22 @@ \subsubsection{Processing of Incoming Packets}\label{sec:Device Types / Network #define VIRTIO_NET_HASH_REPORT_UDPv6_EX 9 \end{lstlisting} +If \field{hash_report} differs from VIRTIO_NET_HASH_REPORT_NONE, +\field{hash_report_tunnel} can report the type of the encapsulated +packet to the driver over the inner header hash calculation. +Possible values that the device can report in \field{hash_report_tunnel} +are defined below: + +\begin{lstlisting} +#define VIRTIO_NET_HASH_REPORT_TUNNEL_NONE 0 +#define VIRTIO_NET_HASH_REPORT_GRE 1 +#define VIRTIO_NET_HASH_REPORT_VXLAN 2 +#define VIRTIO_NET_HASH_REPORT_GENEVE 3 +\end{lstlisting} + +They correspond to supported hash types defined in +\ref{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / Supported/enabled hash types}. + \subsubsection{Control Virtqueue}\label{sec:Device Types / Network Device / Device Operation / Control Virtqueue} The driver uses the control virtqueue (if VIRTIO_NET_F_CTRL_VQ isintroduction.tex +19
-0@@ -98,6 +98,25 @@ \section{Normative References}\label{sec:Normative References} \phantomsection\label{intro:SEC1}\textbf{[SEC1]} & Standards for Efficient Cryptography Group(SECG), ``SEC1: Elliptic Cureve Cryptography'', Version 1.0, September 2000. \newline\url{https://www.secg.org/sec1-v2.pdf}\\ + \phantomsection\label{intro:GRE}\textbf{[GRE]} & + Generic Routing Encapsulation + \newline\url{https://datatracker.ietf.org/doc/rfc2784/}\\ + \phantomsection\label{intro:VXLAN}\textbf{[VXLAN]} & + Virtual eXtensible Local Area Network + \newline\url{https://datatracker.ietf.org/doc/rfc7348/}\\ + \phantomsection\label{intro:GENEVE}\textbf{[GENEVE]} & + Generic Network Virtualization Encapsulation + \newline\url{https://datatracker.ietf.org/doc/rfc8926/}\\ + \phantomsection\label{intro:IP}\textbf{[IP]} & + INTERNET PROTOCOL + \newline\url{https://www.rfc-editor.org/rfc/rfc791}\\ + \phantomsection\label{intro:UDP}\textbf{[UDP]} & + User Datagram Protocol + \newline\url{https://www.rfc-editor.org/rfc/rfc768}\\ + \phantomsection\label{intro:TCP}\textbf{[TCP]} & + TRANSMISSION CONTROL PROTOCOL + \newline\url{https://www.rfc-editor.org/rfc/rfc793}\\ + \end{longtable}Signature · 1 line
2.19.1.6.gb485710b
Quoted 61 lines · Michael S. Tsirkin
Quoted 9 lines · Heng Qi
If the tunnel is used to encapsulate the packets, the hash calculated using the outer header of the receive packets is always fixed for the same flow packets, i.e. they will be steered to the same receive queue.
We add a feature bit
VIRTIO_NET_F_HASH_TUNNELand related bitmasks in \field{hash_types}, which instructs the device to calculate the hash using the inner headers of tunnel-encapsulated packets. Besides, values in \field{hash_report_tunnel} are added to report tunnel types.Quoted 5 lines
Reviewed-by Jason Wang <[email protected]>Signed-off-by Heng Qi <[email protected]>Signed-off-by Xuan Zhuo <[email protected]>ok close to being ready. a couple of minor comments.
Quoted 42 lines
---
v6:
1. Modify the wording of some sentences for clarity. @Michael S. Tsirkin
2. Fix some syntax issues. @Michael S. Tsirkinv5:
1. Fix some syntax and capitalization issues. @Michael S. Tsirkin
2. Use encapsulated/encaptulation uniformly. @Michael S. Tsirkin
3. Move the links to introduction section. @Michael S. Tsirkin
4. Clarify some sentences. @Michael S. Tsirkinv4:
1. Clarify some paragraphs. @Cornelia Huck
2. Fix the u8 type. @Cornelia Huckv3:
1. RenameVIRTIO_NET_F_HASH_GRE_VXLAN_GENEVE_INNERtoVIRTIO_NET_F_HASH_TUNNEL. @Jason Wang
2. Make things clearer. @Jason Wang @Michael S. Tsirkin
3. Keep the possibility to use inner hash for automatic receive steering. @Jason Wang
4. Add the "Tunnel packet" paragraph to avoid repeating the GRE etc. many times. @Michael S. Tsirkinv2:
1. Add a feature bit for GRE/VXLAN/GENEVE inner hash. @Jason Wang
2. Chang \field{hash_tunnel} to \field{hash_report_tunnel}. @Jason Wang, @Michael S. Tsirkinv1:
1. Remove the patch for the bitmask fix. @Michael S. Tsirkin
2. Clarify some paragraphs. @Jason Wang
3. Add \field{hash_tunnel} andVIRTIO_NET_HASH_REPORT_GRE. @Yuri Benditovichcontent.tex | 191 +++++++++++++++++++++++++++++++++++++++++++++-- introduction.tex | 19 +++++ 2 files changed, 203 insertions(+), 7 deletions(-)content.tex +0
-0@@ -3084,6 +3084,9 @@ \subsection{Feature bits}\label{sec:Device Types / Network Device / Feature bits \item[VIRTIO_NET_F_CTRL_MAC_ADDR(23)] Set MAC address through control channel.
Quoted 4 lines
content.tex \subsection{Feature bits}\label{sec:Device Types / Network Device / Feature bits · line 3088
Quoted 2 lines
content.tex +2
-0+\item[VIRTIO_NET_F_HASH_TUNNEL(52)] Device supports inner + header hash for GRE, VXLAN and GENEVE tunnel-encapsulated packets.I would probably drop the list of tunnel types here.
Do you mean to use "Device supports inner header hash for tunnel-encapsulated packets." instead? Why? We do want to use this feature bit to indicate that the device supports inner hashing of GRE, VXLAN and GENEVE encapsulated packets. As in the v3 discussion
https://lists.oasis-open.org/archives/virtio-dev/202212/msg00024.html , we discussed using VIRTIO_NET_F_HASH_TUNNEL to replace VIRTIO_NET_F_HASH_GRE_VXLAN_GENEVE_INNER and plan to use VIRTIO_NET_F_HASH_TUNNEL_XYZ for future extensions.
Quoted 226 lines
Quoted 213 lines
content.tex +158
-3+ \item[VIRTIO_NET_F_NOTF_COAL(53)] Device supports notifications coalescing. \item[VIRTIO_NET_F_GUEST_USO4 (54)] Driver can receive USOv4 packets.@@ -3095,7 +3098,8 @@ \subsection{Feature bits}\label{sec:Device Types / Network Device / Feature bits to several segments when each of these smaller packets has UDP header. \item[VIRTIO_NET_F_HASH_REPORT(57)] Device can report per-packet hash - value and a type of calculated hash. + value, a type of calculated hash, and, if VIRTIO_NET_F_HASH_TUNNEL + is negotiated, an encapsulation packet type. \item[VIRTIO_NET_F_GUEST_HDRLEN(59)] Driver can provide the exact \field{hdr_len} value. Device benefits from knowing the exact header length.@@ -3140,6 +3144,7 @@ \subsubsection{Feature bit requirements}\label{sec:Device Types / Network Device \item[VIRTIO_NET_F_NOTF_COAL] Requires VIRTIO_NET_F_CTRL_VQ. \item[VIRTIO_NET_F_RSC_EXT] Requires VIRTIO_NET_F_HOST_TSO4 or VIRTIO_NET_F_HOST_TSO6. \item[VIRTIO_NET_F_RSS] Requires VIRTIO_NET_F_CTRL_VQ. +\item[VIRTIO_NET_F_HASH_TUNNEL] Requires VIRTIO_NET_F_CTRL_VQ. \end{description} \subsubsection{Legacy Interface: Feature bits}\label{sec:Device Types / Network Device / Feature bits / Legacy Interface: Feature bits}@@ -3386,7 +3391,8 @@ \subsection{Device Operation}\label{sec:Device Types / Network Device / Device O le16 num_buffers; le32 hash_value; (Only if VIRTIO_NET_F_HASH_REPORT negotiated) le16 hash_report; (Only if VIRTIO_NET_F_HASH_REPORT negotiated) - le16 padding_reserved; (Only if VIRTIO_NET_F_HASH_REPORT negotiated) + u8 hash_report_tunnel; (Only if VIRTIO_NET_F_HASH_REPORT negotiated, only valid of VIRTIO_NET_F_HASH_TUNNEL negotiated) + u8 padding_reserved; (Only if VIRTIO_NET_F_HASH_REPORT negotiated) }; \end{lstlisting}@@ -3837,7 +3843,8 @@ \subsubsection{Processing of Incoming Packets}\label{sec:Device Types / Network A device attempts to calculate a per-packet hash in the following cases: \begin{itemize} \item The feature VIRTIO_NET_F_RSS was negotiated. The device uses the hash to determine the receive virtqueue to place incoming packets. -\item The feature VIRTIO_NET_F_HASH_REPORT was negotiated. The device reports the hash value and the hash type with the packet. +\item The feature VIRTIO_NET_F_HASH_REPORT was negotiated. The device reports the hash value and the hash type. If additionally +VIRTIO_NET_F_HASH_TUNNEL was negotiated, the device reports the encapsulation type as well. \end{itemize} If the feature VIRTIO_NET_F_RSS was negotiated:@@ -3863,8 +3870,36 @@ \subsubsection{Processing of Incoming Packets}\label{sec:Device Types / Network \ref{sec:Device Types / Network Device / Device Operation / Control Virtqueue / Automatic receive steering in multiqueue mode / Hash calculation}. \end{itemize} +\subparagraph{Tunnel/Encapsulated packet} +\label{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / Tunnel/Encapsulated packet} +A tunnel packet is encapsulated from the original packet based on the tunneling +protocol (only a single level of encapsulation is currently supported). The +encapsulated packet contains an outer header and an inner header, and the device +calculates the hash over either the inner header or the outer header. + +When the feature VIRTIO_NET_F_HASH_TUNNEL is negotiated and the corresponding +encapsulation type is set in \field{hash_types}, the hash for a specific type of +encapsulated packet is calculated over the inner as opposed to outer header. +Supported encapsulation types are listed in \ref{sec:Device Types / Network Device / +Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / +Supported/enabled hash types}. + +If both VIRTIO_NET_F_HASH_REPORT and VIRTIO_NET_F_HASH_TUNNEL are negotiated, +the device can support inner hash calculation for \hyperref[intro:GRE]{[GRE]}, +\hyperref[intro:VXLAN]{[VXLAN]} and \hyperref[intro:GENEVE]{[GENEVE]} +encapsulated packets, and the corresponding encapsulation type in \field{hash_types} +is VIRTIO_NET_HASH_TYPE_{GRE, VXLAN, GENEVE}_INNER respectively. The value in +\field{hash_report_tunnel} is VIRTIO_NET_HASH_REPORT_{NONE, GRE, VXLAN, GENEVE} respectively. + +If VIRTIO_NET_F_HASH_REPORT is negotiated but VIRTIO_NET_F_HASH_TUNNEL is not +negotiated, the device calculates the hash over the outer header, and +\field{hash_report} reports the hash type. \field{hash_report_tunnel} +is no longer valid. + \subparagraph{Supported/enabled hash types} \label{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / Supported/enabled hash types} +This paragraph relies on definitions from \hyperref[intro:IP]{[IP]}, +\hyperref[intro:UDP]{[UDP]} and \hyperref[intro:TCP]{[TCP]}. Hash types applicable for IPv4 packets: \begin{lstlisting} #define VIRTIO_NET_HASH_TYPE_IPv4 (1 << 0)@@ -3884,6 +3919,22 @@ \subsubsection{Processing of Incoming Packets}\label{sec:Device Types / Network #define VIRTIO_NET_HASH_TYPE_UDP_EX (1 << 8) \end{lstlisting} +If the feature VIRTIO_NET_F_HASH_TUNNEL is negotiated, the encapsulation +hash type below indicates that the hash is calculated over the inner +header of the encapsulated packet: +Hash type applicable for inner payload of the gre-encapsulated packet +\begin{lstlisting} +#define VIRTIO_NET_HASH_TYPE_GRE_INNER (1 << 9) +\end{lstlisting} +Hash type applicable for inner payload of the vxlan-encapsulated packet +\begin{lstlisting} +#define VIRTIO_NET_HASH_TYPE_VXLAN_INNER (1 << 10) +\end{lstlisting} +Hash type applicable for inner payload of the geneve-encapsulated packet +\begin{lstlisting} +#define VIRTIO_NET_HASH_TYPE_GENEVE_INNER (1 << 11) +\end{lstlisting} + \subparagraph{IPv4 packets} \label{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / IPv4 packets} The device calculates the hash on IPv4 packets according to 'Enabled hash types' bitmask as follows:@@ -3975,15 +4026,125 @@ \subsubsection{Processing of Incoming Packets}\label{sec:Device Types / Network (see \ref{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / IPv6 packets without extension header}). \end{itemize} +\subparagraph{Inner payload of an encapsulated packet} +\label{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / Inner payload of the encapsulated packet} +If the feature VIRTIO_NET_F_HASH_TUNNEL is negotiated and the corresponding +encapsulation hash type is set in \field{hash_types}, the device calculates the +inner header hash on an encapsulated packet (See \ref{sec:Device Types +/ Network Device / Device Operation / Processing of Incoming Packets / +Hash calculation for incoming packets / Tunnel/Encapsulated packet}) as follows: + +The device calculates the hash on the inner IPv4 packet of an encapsulated packet +according to 'Enabled hash types' bitmask as follows: +\begin{itemize} + \item If VIRTIO_NET_HASH_TYPE_TCPv4 is set and the encapsulated packet has an inner + TCPv4 header, the hash is calculated over the following fields: + \begin{itemsize} + \item inner Source IP address + \item inner Destination IP address + \item inner Source TCP port + \item inner Destination TCP port + \end{itemsize} + \item Else if VIRTIO_NET_HASH_TYPE_UDPv4 is set and the encapsulated packet has an + inner UDPv4 header, the hash is calculated over the following fields: + \begin{itemsize} + \item inner Source IP address + \item inner Destination IP address + \item inner Source UDP port + \item inner Destination UDP port + \end{itemize} + \item Else if VIRTIO_NET_HASH_TYPE_IPv4 is set, the hash is calculated over the + following fields: + \begin{itemsize} + \item inner Source IP address + \item inner Destination IP address + \end{itemsize} + \item Else the device does not calculate the hash +\end{itemize} + +The device calculates the hash on the inner IPv6 packet without an extension header +of an encapsulated packet according to 'Enabled hash types' bitmask as follows: +\begin{itemize} + \item If VIRTIO_NET_HASH_TYPE_TCPv6 is set and the encapsulated packet has an inner + TCPv6 header, the hash is calculated over the following fields: + \begin{itemsize} + \item inner Source IPv6 address + \item inner Destination IPv6 address + \item inner Source TCP port + \item inner Destination TCP port + \end{itemsize} + \item Else if VIRTIO_NET_HASH_TYPE_UDPv6 is set and the encapsulated packet has an + inner UDPv6 header, the hash is calculated over the following fields: + \begin{itemsize} + \item inner Source IPv6 address + \item inner Destination IPv6 address + \item inner Source UDP port + \item inner Destination UDP port + \end{itemize} + \item Else if VIRTIO_NET_HASH_TYPE_IPv6 is set, the hash is calculated over the + following fields: + \begin{itemsize} + \item inner Source IPv6 address + \item inner Destination IPv6 address + \end{itemsize} + \item Else the device does not calculate the hash +\end{itemize} + +The device calculates the hash on the inner IPv6 packet with an extension header +of an encapsulated packet according to 'Enabled hash types' bitmask as follows: +\begin{itemsize} + \item If VIRTIO_NET_HASH_TYPE_TCP_EX is set and the encapsulated packet has an inner + TCPv6 header, the hash is calculated over the following fields: + \begin{itemize} + \item Home address from the home address option in the inner IPv6 destination + options header. If the inner extension header is not present, use the + inner Source IPv6 address. + \item IPv6 address that is contained in the Routing-Header-Type-2 from the + associated inner extension header. If the inner extension header is not + present, use the inner Destination IPv6 address. + \item inner Source TCP port + \item inner Destination TCP port + \end{itemize} + \item Else if VIRTIO_NET_HASH_TYPE_UDP_EX is set and the encapsulated packet has an inner + UDPv6 header, the hash is calculated over the following fields: + \begin{itemsize} + \item Home address from the home address option in the inner IPv6 destination + options header. If the inner extension header is not present, use the + inner Source IPv6 address. + \item IPv6 address that is contained in the Routing-Header-Type-2 from the + associated inner extension header. If the inner extension header is not + present, use the inner Destination IPv6 address. + \item inner Source UDP port + \item inner Destination UDP port + \end{itemize} + \item Else if VIRTIO_NET_HASH_TYPE_IP_EX is set, the hash is calculated over the + following fields: + \begin{itemsize} + \item Home address from the home address option in the inner IPv6 destination + options header. If the inner extension header is not present, use the + inner Source IPv6 address. + \item IPv6 address that is contained in the Routing-Header-Type-2 from the + associated inner extension header. If the inner extension header is not + present, use the inner Destination IPv6 address. + \end{itemize} + \item Else skip the inner IPv6 extension header and calculate the inner header hash as + defined for an encapsulated packet whose inner payload is an IPv6 packet without + an extension header. +\end{itemsize} +content.tex \subsubsection{Processing of Incoming Packets}\label{sec:Device Types / Network · line 4141
Quoted 10 lines
content.tex +4
-3\paragraph{Hash reporting for incoming packets} \label{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash reporting for incoming packets} -If VIRTIO_NET_F_HASH_REPORT was negotiated and - the device has calculated the hash for the packet, the device fills \field{hash_report} with the report type of calculated hash -and \field{hash_value} with the value of calculated hash. +If VIRTIO_NET_F_HASH_REPORT was negotiated and the device has calculated the +hash for the packet, the device fills \field{hash_report} with the report type +of calculated hash, and \field{hash_value} with the value of calculated hash. +Also, if VIRTIO_NET_F_HASH_TUNNEL was negotiated, the device needs to fillneeds to fill -> fills
Quoted 1 line
for consistency
I agree.
Thanks.
Quoted 83 lines
Quoted 66 lines
content.tex +20
-1+\field{hash_report_tunnel} with the report type of the encapsulated packet, and +it is set to VIRTIO_NET_HASH_REPORT_TUNNEL_NONE for the unencapsulated packet. If VIRTIO_NET_F_HASH_REPORT was negotiated but due to any reason the -hash was not calculated, the device sets \field{hash_report} to VIRTIO_NET_HASH_REPORT_NONE. +hash was not calculated, the device sets \field{hash_report} to VIRTIO_NET_HASH_REPORT_NONE, +and sets \field{hash_report_tunnel} to VIRTIO_NET_HASH_REPORT_TUNNEL_NONE. Possible values that the device can report in \field{hash_report} are defined below. They correspond to supported hash types defined in@@ -4005,6 +4166,22 @@ \subsubsection{Processing of Incoming Packets}\label{sec:Device Types / Network #define VIRTIO_NET_HASH_REPORT_UDPv6_EX 9 \end{lstlisting} +If \field{hash_report} differs from VIRTIO_NET_HASH_REPORT_NONE, +\field{hash_report_tunnel} can report the type of the encapsulated +packet to the driver over the inner header hash calculation. +Possible values that the device can report in \field{hash_report_tunnel} +are defined below: + +\begin{lstlisting} +#define VIRTIO_NET_HASH_REPORT_TUNNEL_NONE 0 +#define VIRTIO_NET_HASH_REPORT_GRE 1 +#define VIRTIO_NET_HASH_REPORT_VXLAN 2 +#define VIRTIO_NET_HASH_REPORT_GENEVE 3 +\end{lstlisting} + +They correspond to supported hash types defined in +\ref{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / Supported/enabled hash types}. + \subsubsection{Control Virtqueue}\label{sec:Device Types / Network Device / Device Operation / Control Virtqueue} The driver uses the control virtqueue (if VIRTIO_NET_F_CTRL_VQ isintroduction.tex +19
-0@@ -98,6 +98,25 @@ \section{Normative References}\label{sec:Normative References} \phantomsection\label{intro:SEC1}\textbf{[SEC1]} & Standards for Efficient Cryptography Group(SECG), ``SEC1: Elliptic Cureve Cryptography'', Version 1.0, September 2000. \newline\url{https://www.secg.org/sec1-v2.pdf}\\ + \phantomsection\label{intro:GRE}\textbf{[GRE]} & + Generic Routing Encapsulation + \newline\url{https://datatracker.ietf.org/doc/rfc2784/}\\ + \phantomsection\label{intro:VXLAN}\textbf{[VXLAN]} & + Virtual eXtensible Local Area Network + \newline\url{https://datatracker.ietf.org/doc/rfc7348/}\\ + \phantomsection\label{intro:GENEVE}\textbf{[GENEVE]} & + Generic Network Virtualization Encapsulation + \newline\url{https://datatracker.ietf.org/doc/rfc8926/}\\ + \phantomsection\label{intro:IP}\textbf{[IP]} & + INTERNET PROTOCOL + \newline\url{https://www.rfc-editor.org/rfc/rfc791}\\ + \phantomsection\label{intro:UDP}\textbf{[UDP]} & + User Datagram Protocol + \newline\url{https://www.rfc-editor.org/rfc/rfc768}\\ + \phantomsection\label{intro:TCP}\textbf{[TCP]} & + TRANSMISSION CONTROL PROTOCOL + \newline\url{https://www.rfc-editor.org/rfc/rfc793}\\ + \end{longtable}Signature · 1 line
2.19.1.6.gb485710bThis publicly archived list offers a means to provide input to the OASIS Virtual I/O Device (VIRTIO) TC.
In order to verify user consent to the Feedback License terms and to minimize spam in the list archive, subscription is required before posting.
Subscribe: [email protected]
Unsubscribe: [email protected]
List help: [email protected]
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/
Quoted 67 lines · Heng Qi
Quoted 61 lines · Michael S. Tsirkin
Quoted 9 lines · Heng Qi
If the tunnel is used to encapsulate the packets, the hash calculated using the outer header of the receive packets is always fixed for the same flow packets, i.e. they will be steered to the same receive queue.
We add a feature bit
VIRTIO_NET_F_HASH_TUNNELand related bitmasks in \field{hash_types}, which instructs the device to calculate the hash using the inner headers of tunnel-encapsulated packets. Besides, values in \field{hash_report_tunnel} are added to report tunnel types.Quoted 5 lines
Reviewed-by Jason Wang <[email protected]>Signed-off-by Heng Qi <[email protected]>Signed-off-by Xuan Zhuo <[email protected]>ok close to being ready. a couple of minor comments.
Quoted 42 lines
---
v6:
1. Modify the wording of some sentences for clarity. @Michael S. Tsirkin
2. Fix some syntax issues. @Michael S. Tsirkinv5:
1. Fix some syntax and capitalization issues. @Michael S. Tsirkin
2. Use encapsulated/encaptulation uniformly. @Michael S. Tsirkin
3. Move the links to introduction section. @Michael S. Tsirkin
4. Clarify some sentences. @Michael S. Tsirkinv4:
1. Clarify some paragraphs. @Cornelia Huck
2. Fix the u8 type. @Cornelia Huckv3:
1. RenameVIRTIO_NET_F_HASH_GRE_VXLAN_GENEVE_INNERtoVIRTIO_NET_F_HASH_TUNNEL. @Jason Wang
2. Make things clearer. @Jason Wang @Michael S. Tsirkin
3. Keep the possibility to use inner hash for automatic receive steering. @Jason Wang
4. Add the "Tunnel packet" paragraph to avoid repeating the GRE etc. many times. @Michael S. Tsirkinv2:
1. Add a feature bit for GRE/VXLAN/GENEVE inner hash. @Jason Wang
2. Chang \field{hash_tunnel} to \field{hash_report_tunnel}. @Jason Wang, @Michael S. Tsirkinv1:
1. Remove the patch for the bitmask fix. @Michael S. Tsirkin
2. Clarify some paragraphs. @Jason Wang
3. Add \field{hash_tunnel} andVIRTIO_NET_HASH_REPORT_GRE. @Yuri Benditovichcontent.tex | 191 +++++++++++++++++++++++++++++++++++++++++++++-- introduction.tex | 19 +++++ 2 files changed, 203 insertions(+), 7 deletions(-)content.tex +0
-0@@ -3084,6 +3084,9 @@ \subsection{Feature bits}\label{sec:Device Types / Network Device / Feature bits \item[VIRTIO_NET_F_CTRL_MAC_ADDR(23)] Set MAC address through control channel.Quoted 4 lines
content.tex \subsection{Feature bits}\label{sec:Device Types / Network Device / Feature bits · line 3088
Quoted 2 lines
content.tex +2
-0+\item[VIRTIO_NET_F_HASH_TUNNEL(52)] Device supports inner + header hash for GRE, VXLAN and GENEVE tunnel-encapsulated packets.I would probably drop the list of tunnel types here.
Quoted 8 lines
Do you mean to use "Device supports inner header hash for tunnel-encapsulated packets." instead? Why? We do want to use this feature bit to indicate that the device supports inner hashing of GRE, VXLAN and GENEVE encapsulated packets. As in the v3 discussion
https://lists.oasis-open.org/archives/virtio-dev/202212/msg00024.html , we discussed usingVIRTIO_NET_F_HASH_TUNNELto replaceVIRTIO_NET_F_HASH_GRE_VXLAN_GENEVE_INNERand plan to useVIRTIO_NET_F_HASH_TUNNEL_XYZfor future extensions.
So imagine we add a new tunnel type. Let's say there's VXLAN v2. why would we need a new feature bit? I think a new hash type will be sufficient. No?
Signature · 1 line
MST
Quoted 77 lines · Michael S. Tsirkin
Quoted 67 lines · Heng Qi
Quoted 61 lines · Michael S. Tsirkin
Quoted 9 lines · Heng Qi
If the tunnel is used to encapsulate the packets, the hash calculated using the outer header of the receive packets is always fixed for the same flow packets, i.e. they will be steered to the same receive queue.
We add a feature bit
VIRTIO_NET_F_HASH_TUNNELand related bitmasks in \field{hash_types}, which instructs the device to calculate the hash using the inner headers of tunnel-encapsulated packets. Besides, values in \field{hash_report_tunnel} are added to report tunnel types.Quoted 5 lines
Reviewed-by Jason Wang <[email protected]>Signed-off-by Heng Qi <[email protected]>Signed-off-by Xuan Zhuo <[email protected]>ok close to being ready. a couple of minor comments.
Quoted 42 lines
---
v6:
1. Modify the wording of some sentences for clarity. @Michael S. Tsirkin
2. Fix some syntax issues. @Michael S. Tsirkinv5:
1. Fix some syntax and capitalization issues. @Michael S. Tsirkin
2. Use encapsulated/encaptulation uniformly. @Michael S. Tsirkin
3. Move the links to introduction section. @Michael S. Tsirkin
4. Clarify some sentences. @Michael S. Tsirkinv4:
1. Clarify some paragraphs. @Cornelia Huck
2. Fix the u8 type. @Cornelia Huckv3:
1. RenameVIRTIO_NET_F_HASH_GRE_VXLAN_GENEVE_INNERtoVIRTIO_NET_F_HASH_TUNNEL. @Jason Wang
2. Make things clearer. @Jason Wang @Michael S. Tsirkin
3. Keep the possibility to use inner hash for automatic receive steering. @Jason Wang
4. Add the "Tunnel packet" paragraph to avoid repeating the GRE etc. many times. @Michael S. Tsirkinv2:
1. Add a feature bit for GRE/VXLAN/GENEVE inner hash. @Jason Wang
2. Chang \field{hash_tunnel} to \field{hash_report_tunnel}. @Jason Wang, @Michael S. Tsirkinv1:
1. Remove the patch for the bitmask fix. @Michael S. Tsirkin
2. Clarify some paragraphs. @Jason Wang
3. Add \field{hash_tunnel} andVIRTIO_NET_HASH_REPORT_GRE. @Yuri Benditovichcontent.tex | 191 +++++++++++++++++++++++++++++++++++++++++++++-- introduction.tex | 19 +++++ 2 files changed, 203 insertions(+), 7 deletions(-)content.tex +0
-0@@ -3084,6 +3084,9 @@ \subsection{Feature bits}\label{sec:Device Types / Network Device / Feature bits \item[VIRTIO_NET_F_CTRL_MAC_ADDR(23)] Set MAC address through control channel.Quoted 4 lines
content.tex \subsection{Feature bits}\label{sec:Device Types / Network Device / Feature bits · line 3088
Quoted 2 lines
content.tex +2
-0+\item[VIRTIO_NET_F_HASH_TUNNEL(52)] Device supports inner + header hash for GRE, VXLAN and GENEVE tunnel-encapsulated packets.I would probably drop the list of tunnel types here.
Quoted 8 lines
Do you mean to use "Device supports inner header hash for tunnel-encapsulated packets." instead? Why? We do want to use this feature bit to indicate that the device supports inner hashing of GRE, VXLAN and GENEVE encapsulated packets. As in the v3 discussion
https://lists.oasis-open.org/archives/virtio-dev/202212/msg00024.html , we discussed usingVIRTIO_NET_F_HASH_TUNNELto replaceVIRTIO_NET_F_HASH_GRE_VXLAN_GENEVE_INNERand plan to useVIRTIO_NET_F_HASH_TUNNEL_XYZfor future extensions.
Quoted 3 lines
So imagine we add a new tunnel type. Let's say there's VXLAN v2. why would we need a new feature bit? I think a new hash type will be sufficient. No?
If the description for VIRTIO_NET_F_HASH_TUNNEL is as follows:
"[VIRTIO_NET_F_HASH_TUNNEL(52)] Device supports inner header hash for tunnel-encapsulated packets.". Then the following may happen
1. For VXLANv2, if both device src and device dst have negotiated this feature, it is assumed that device src supports VXLAN and VXLANv2, but device dst may only support VXLAN, not VXLANv2.
2. For other encapsulation protocols such as ip in ip, after device src and device dst have negotiated this feature, it is assumed that device src supports GRE, VXLAN, GENEVE and ip in ip, but it is not clear that device dst also supports ip in ip. Especially when migrating, this can lead to inconsistencies in live migrations.
So, I think it's better to keep the original description:
"[VIRTIO_NET_F_HASH_TUNNEL(52)] Device supports inner header hash for GRE, VXLAN and GENEVE tunnel-encapsulated packets."
Thanks.
Quoted 8 lines
Signature · 7 lines
-- MST --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
This publicly archived list offers a means to provide input to the OASIS Virtual I/O Device (VIRTIO) TC.
In order to verify user consent to the Feedback License terms and to minimize spam in the list archive, subscription is required before posting.
Subscribe: [email protected]
Unsubscribe: [email protected]
List help: [email protected]
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/
Quoted 83 lines · Heng Qi
Quoted 77 lines · Michael S. Tsirkin
Quoted 67 lines · Heng Qi
Quoted 61 lines · Michael S. Tsirkin
Quoted 9 lines · Heng Qi
If the tunnel is used to encapsulate the packets, the hash calculated using the outer header of the receive packets is always fixed for the same flow packets, i.e. they will be steered to the same receive queue.
We add a feature bit
VIRTIO_NET_F_HASH_TUNNELand related bitmasks in \field{hash_types}, which instructs the device to calculate the hash using the inner headers of tunnel-encapsulated packets. Besides, values in \field{hash_report_tunnel} are added to report tunnel types.Quoted 5 lines
Reviewed-by Jason Wang <[email protected]>Signed-off-by Heng Qi <[email protected]>Signed-off-by Xuan Zhuo <[email protected]>ok close to being ready. a couple of minor comments.
Quoted 42 lines
---
v6:
1. Modify the wording of some sentences for clarity. @Michael S. Tsirkin
2. Fix some syntax issues. @Michael S. Tsirkinv5:
1. Fix some syntax and capitalization issues. @Michael S. Tsirkin
2. Use encapsulated/encaptulation uniformly. @Michael S. Tsirkin
3. Move the links to introduction section. @Michael S. Tsirkin
4. Clarify some sentences. @Michael S. Tsirkinv4:
1. Clarify some paragraphs. @Cornelia Huck
2. Fix the u8 type. @Cornelia Huckv3:
1. RenameVIRTIO_NET_F_HASH_GRE_VXLAN_GENEVE_INNERtoVIRTIO_NET_F_HASH_TUNNEL. @Jason Wang
2. Make things clearer. @Jason Wang @Michael S. Tsirkin
3. Keep the possibility to use inner hash for automatic receive steering. @Jason Wang
4. Add the "Tunnel packet" paragraph to avoid repeating the GRE etc. many times. @Michael S. Tsirkinv2:
1. Add a feature bit for GRE/VXLAN/GENEVE inner hash. @Jason Wang
2. Chang \field{hash_tunnel} to \field{hash_report_tunnel}. @Jason Wang, @Michael S. Tsirkinv1:
1. Remove the patch for the bitmask fix. @Michael S. Tsirkin
2. Clarify some paragraphs. @Jason Wang
3. Add \field{hash_tunnel} andVIRTIO_NET_HASH_REPORT_GRE. @Yuri Benditovichcontent.tex | 191 +++++++++++++++++++++++++++++++++++++++++++++-- introduction.tex | 19 +++++ 2 files changed, 203 insertions(+), 7 deletions(-)content.tex +0
-0@@ -3084,6 +3084,9 @@ \subsection{Feature bits}\label{sec:Device Types / Network Device / Feature bits \item[VIRTIO_NET_F_CTRL_MAC_ADDR(23)] Set MAC address through control channel.Quoted 4 lines
content.tex \subsection{Feature bits}\label{sec:Device Types / Network Device / Feature bits · line 3088
Quoted 2 lines
content.tex +2
-0+\item[VIRTIO_NET_F_HASH_TUNNEL(52)] Device supports inner + header hash for GRE, VXLAN and GENEVE tunnel-encapsulated packets.I would probably drop the list of tunnel types here.
Quoted 8 lines
Do you mean to use "Device supports inner header hash for tunnel-encapsulated packets." instead? Why? We do want to use this feature bit to indicate that the device supports inner hashing of GRE, VXLAN and GENEVE encapsulated packets. As in the v3 discussion
https://lists.oasis-open.org/archives/virtio-dev/202212/msg00024.html , we discussed usingVIRTIO_NET_F_HASH_TUNNELto replaceVIRTIO_NET_F_HASH_GRE_VXLAN_GENEVE_INNERand plan to useVIRTIO_NET_F_HASH_TUNNEL_XYZfor future extensions.Quoted 3 lines
So imagine we add a new tunnel type. Let's say there's VXLAN v2. why would we need a new feature bit? I think a new hash type will be sufficient. No?
Quoted 9 lines
If the description for
VIRTIO_NET_F_HASH_TUNNELis as follows:
"[VIRTIO_NET_F_HASH_TUNNEL(52)] Device supports inner header hash for tunnel-encapsulated packets.". Then the following may happen
1. For VXLANv2, if both device src and device dst have negotiated this feature, it is assumed that device src supports VXLAN and VXLANv2, but device dst may only support VXLAN, not VXLANv2.
2. For other encapsulation protocols such as ip in ip, after device src and device dst have negotiated this feature, it is assumed that device src supports GRE, VXLAN, GENEVE and ip in ip, but it is not clear that device dst also supports ip in ip. Especially when migrating, this can lead to inconsistencies in live migrations.
Yes, this looks like the only way that I can think of to keep migration compatibility in an easy way.
Thanks
Quoted 14 lines
So, I think it's better to keep the original description:
"[VIRTIO_NET_F_HASH_TUNNEL(52)] Device supports inner header hash for GRE, VXLAN and GENEVE tunnel-encapsulated packets."Thanks.
Quoted 8 lines
--
MST---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Quoted 84 lines · Jason Wang
Quoted 83 lines · Heng Qi
Quoted 80 lines · Michael S. Tsirkin
Quoted 67 lines · Heng Qi
Quoted 61 lines · Michael S. Tsirkin
Quoted 9 lines · Heng Qi
If the tunnel is used to encapsulate the packets, the hash calculated using the outer header of the receive packets is always fixed for the same flow packets, i.e. they will be steered to the same receive queue.
We add a feature bit
VIRTIO_NET_F_HASH_TUNNELand related bitmasks in \field{hash_types}, which instructs the device to calculate the hash using the inner headers of tunnel-encapsulated packets. Besides, values in \field{hash_report_tunnel} are added to report tunnel types.Quoted 5 lines
Reviewed-by Jason Wang <[email protected]>Signed-off-by Heng Qi <[email protected]>Signed-off-by Xuan Zhuo <[email protected]>ok close to being ready. a couple of minor comments.
Quoted 42 lines
---
v6:
1. Modify the wording of some sentences for clarity. @Michael S. Tsirkin
2. Fix some syntax issues. @Michael S. Tsirkinv5:
1. Fix some syntax and capitalization issues. @Michael S. Tsirkin
2. Use encapsulated/encaptulation uniformly. @Michael S. Tsirkin
3. Move the links to introduction section. @Michael S. Tsirkin
4. Clarify some sentences. @Michael S. Tsirkinv4:
1. Clarify some paragraphs. @Cornelia Huck
2. Fix the u8 type. @Cornelia Huckv3:
1. RenameVIRTIO_NET_F_HASH_GRE_VXLAN_GENEVE_INNERtoVIRTIO_NET_F_HASH_TUNNEL. @Jason Wang
2. Make things clearer. @Jason Wang @Michael S. Tsirkin
3. Keep the possibility to use inner hash for automatic receive steering. @Jason Wang
4. Add the "Tunnel packet" paragraph to avoid repeating the GRE etc. many times. @Michael S. Tsirkinv2:
1. Add a feature bit for GRE/VXLAN/GENEVE inner hash. @Jason Wang
2. Chang \field{hash_tunnel} to \field{hash_report_tunnel}. @Jason Wang, @Michael S. Tsirkinv1:
1. Remove the patch for the bitmask fix. @Michael S. Tsirkin
2. Clarify some paragraphs. @Jason Wang
3. Add \field{hash_tunnel} andVIRTIO_NET_HASH_REPORT_GRE. @Yuri Benditovichcontent.tex | 191 +++++++++++++++++++++++++++++++++++++++++++++-- introduction.tex | 19 +++++ 2 files changed, 203 insertions(+), 7 deletions(-)content.tex +0
-0@@ -3084,6 +3084,9 @@ \subsection{Feature bits}\label{sec:Device Types / Network Device / Feature bits \item[VIRTIO_NET_F_CTRL_MAC_ADDR(23)] Set MAC address through control channel.Quoted 4 lines
content.tex \subsection{Feature bits}\label{sec:Device Types / Network Device / Feature bits · line 3088
Quoted 2 lines
content.tex +2
-0+\item[VIRTIO_NET_F_HASH_TUNNEL(52)] Device supports inner + header hash for GRE, VXLAN and GENEVE tunnel-encapsulated packets.I would probably drop the list of tunnel types here.
Quoted 8 lines
Do you mean to use "Device supports inner header hash for tunnel-encapsulated packets." instead? Why? We do want to use this feature bit to indicate that the device supports inner hashing of GRE, VXLAN and GENEVE encapsulated packets. As in the v3 discussion
https://lists.oasis-open.org/archives/virtio-dev/202212/msg00024.html , we discussed usingVIRTIO_NET_F_HASH_TUNNELto replaceVIRTIO_NET_F_HASH_GRE_VXLAN_GENEVE_INNERand plan to useVIRTIO_NET_F_HASH_TUNNEL_XYZfor future extensions.So imagine we add a new tunnel type. Let's say there's VXLAN v2. why would we need a new feature bit? I think a new hash type will be sufficient. No?
Quoted 9 lines
Quoted 9 lines
If the description for
VIRTIO_NET_F_HASH_TUNNELis as follows:
"[VIRTIO_NET_F_HASH_TUNNEL(52)] Device supports inner header hash for tunnel-encapsulated packets.". Then the following may happen
1. For VXLANv2, if both device src and device dst have negotiated this feature, it is assumed that device src supports VXLAN and VXLANv2, but device dst may only support VXLAN, not VXLANv2.
2. For other encapsulation protocols such as ip in ip, after device src and device dst have negotiated this feature, it is assumed that device src supports GRE, VXLAN, GENEVE and ip in ip, but it is not clear that device dst also supports ip in ip. Especially when migrating, this can lead to inconsistencies in live migrations.
it does not matter. e.g. if device supports GRE but not VXLAN then feature bit it set but hash types are different.
and this is not different from any other offload we have.
Generally the only reason we even use a new feature bit for this tunneling is because the header structure is a bit different so this kind of makes sense.
Quoted 5 lines
Yes, this looks like the only way that I can think of to keep migration compatibility in an easy way.
Thanks
Jason migration compatibility must also check hash types. There's no chance with the current hash offload scheme to only rely on feature bits for migration compatibility.
Quoted 14 lines
Quoted 14 lines
So, I think it's better to keep the original description:
"[VIRTIO_NET_F_HASH_TUNNEL(52)] Device supports inner header hash for GRE, VXLAN and GENEVE tunnel-encapsulated packets."Thanks.
Quoted 8 lines
--
MST---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Quoted 96 lines · Michael S. Tsirkin
Quoted 84 lines · Jason Wang
Quoted 83 lines · Heng Qi
Quoted 80 lines · Michael S. Tsirkin
Quoted 67 lines · Heng Qi
Quoted 61 lines · Michael S. Tsirkin
Quoted 9 lines · Heng Qi
If the tunnel is used to encapsulate the packets, the hash calculated using the outer header of the receive packets is always fixed for the same flow packets, i.e. they will be steered to the same receive queue.
We add a feature bit
VIRTIO_NET_F_HASH_TUNNELand related bitmasks in \field{hash_types}, which instructs the device to calculate the hash using the inner headers of tunnel-encapsulated packets. Besides, values in \field{hash_report_tunnel} are added to report tunnel types.Quoted 5 lines
Reviewed-by Jason Wang <[email protected]>Signed-off-by Heng Qi <[email protected]>Signed-off-by Xuan Zhuo <[email protected]>ok close to being ready. a couple of minor comments.
Quoted 42 lines
---
v6:
1. Modify the wording of some sentences for clarity. @Michael S. Tsirkin
2. Fix some syntax issues. @Michael S. Tsirkinv5:
1. Fix some syntax and capitalization issues. @Michael S. Tsirkin
2. Use encapsulated/encaptulation uniformly. @Michael S. Tsirkin
3. Move the links to introduction section. @Michael S. Tsirkin
4. Clarify some sentences. @Michael S. Tsirkinv4:
1. Clarify some paragraphs. @Cornelia Huck
2. Fix the u8 type. @Cornelia Huckv3:
1. RenameVIRTIO_NET_F_HASH_GRE_VXLAN_GENEVE_INNERtoVIRTIO_NET_F_HASH_TUNNEL. @Jason Wang
2. Make things clearer. @Jason Wang @Michael S. Tsirkin
3. Keep the possibility to use inner hash for automatic receive steering. @Jason Wang
4. Add the "Tunnel packet" paragraph to avoid repeating the GRE etc. many times. @Michael S. Tsirkinv2:
1. Add a feature bit for GRE/VXLAN/GENEVE inner hash. @Jason Wang
2. Chang \field{hash_tunnel} to \field{hash_report_tunnel}. @Jason Wang, @Michael S. Tsirkinv1:
1. Remove the patch for the bitmask fix. @Michael S. Tsirkin
2. Clarify some paragraphs. @Jason Wang
3. Add \field{hash_tunnel} andVIRTIO_NET_HASH_REPORT_GRE. @Yuri Benditovichcontent.tex | 191 +++++++++++++++++++++++++++++++++++++++++++++-- introduction.tex | 19 +++++ 2 files changed, 203 insertions(+), 7 deletions(-)content.tex +0
-0@@ -3084,6 +3084,9 @@ \subsection{Feature bits}\label{sec:Device Types / Network Device / Feature bits \item[VIRTIO_NET_F_CTRL_MAC_ADDR(23)] Set MAC address through control channel.Quoted 4 lines
content.tex \subsection{Feature bits}\label{sec:Device Types / Network Device / Feature bits · line 3088
Quoted 2 lines
content.tex +2
-0+\item[VIRTIO_NET_F_HASH_TUNNEL(52)] Device supports inner + header hash for GRE, VXLAN and GENEVE tunnel-encapsulated packets.I would probably drop the list of tunnel types here.
Quoted 8 lines
Do you mean to use "Device supports inner header hash for tunnel-encapsulated packets." instead? Why? We do want to use this feature bit to indicate that the device supports inner hashing of GRE, VXLAN and GENEVE encapsulated packets. As in the v3 discussion
https://lists.oasis-open.org/archives/virtio-dev/202212/msg00024.html , we discussed usingVIRTIO_NET_F_HASH_TUNNELto replaceVIRTIO_NET_F_HASH_GRE_VXLAN_GENEVE_INNERand plan to useVIRTIO_NET_F_HASH_TUNNEL_XYZfor future extensions.So imagine we add a new tunnel type. Let's say there's VXLAN v2. why would we need a new feature bit? I think a new hash type will be sufficient. No?
Quoted 9 lines
Quoted 9 lines
If the description for
VIRTIO_NET_F_HASH_TUNNELis as follows:
"[VIRTIO_NET_F_HASH_TUNNEL(52)] Device supports inner header hash for tunnel-encapsulated packets.". Then the following may happen
1. For VXLANv2, if both device src and device dst have negotiated this feature, it is assumed that device src supports VXLAN and VXLANv2, but device dst may only support VXLAN, not VXLANv2.
2. For other encapsulation protocols such as ip in ip, after device src and device dst have negotiated this feature, it is assumed that device src supports GRE, VXLAN, GENEVE and ip in ip, but it is not clear that device dst also supports ip in ip. Especially when migrating, this can lead to inconsistencies in live migrations.
Quoted 2 lines
it does not matter. e.g. if device supports GRE but not VXLAN then feature bit it set but hash types are different.
But the hash will be used for RSS steering. Consider the steering on ipip work on src but not on dst it would break application logic.
Quoted 1 line
and this is not different from any other offload we have.
But we have different feature bits for TCPv4/v6.
Quoted 6 lines
Generally the only reason we even use a new feature bit for this tunneling is because the header structure is a bit different so this kind of makes sense.
Quoted 1 line
Quoted 8 lines
Quoted 4 lines
Yes, this looks like the only way that I can think of to keep migration compatibility in an easy way.
Thanks
Jason migration compatibility must also check hash types. There's no chance with the current hash offload scheme to only rely on feature bits for migration compatibility.
This will give extra pressure on the management stack, e.g it requires the device to have an out of spec way for introspection.
Thanks
Quoted 17 lines
Quoted 16 lines
Quoted 16 lines
So, I think it's better to keep the original description:
"[VIRTIO_NET_F_HASH_TUNNEL(52)] Device supports inner header hash for GRE, VXLAN and GENEVE tunnel-encapsulated packets."Thanks.
Quoted 8 lines
--
MST---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Quoted 97 lines · Jason Wang
Quoted 96 lines · Michael S. Tsirkin
Quoted 93 lines · Jason Wang
Quoted 92 lines · Heng Qi
Quoted 77 lines · Michael S. Tsirkin
Quoted 67 lines · Heng Qi
Quoted 61 lines · Michael S. Tsirkin
Quoted 9 lines · Heng Qi
If the tunnel is used to encapsulate the packets, the hash calculated using the outer header of the receive packets is always fixed for the same flow packets, i.e. they will be steered to the same receive queue.
We add a feature bit
VIRTIO_NET_F_HASH_TUNNELand related bitmasks in \field{hash_types}, which instructs the device to calculate the hash using the inner headers of tunnel-encapsulated packets. Besides, values in \field{hash_report_tunnel} are added to report tunnel types.Quoted 5 lines
Reviewed-by Jason Wang <[email protected]>Signed-off-by Heng Qi <[email protected]>Signed-off-by Xuan Zhuo <[email protected]>ok close to being ready. a couple of minor comments.
Quoted 42 lines
---
v6:
1. Modify the wording of some sentences for clarity. @Michael S. Tsirkin
2. Fix some syntax issues. @Michael S. Tsirkinv5:
1. Fix some syntax and capitalization issues. @Michael S. Tsirkin
2. Use encapsulated/encaptulation uniformly. @Michael S. Tsirkin
3. Move the links to introduction section. @Michael S. Tsirkin
4. Clarify some sentences. @Michael S. Tsirkinv4:
1. Clarify some paragraphs. @Cornelia Huck
2. Fix the u8 type. @Cornelia Huckv3:
1. RenameVIRTIO_NET_F_HASH_GRE_VXLAN_GENEVE_INNERtoVIRTIO_NET_F_HASH_TUNNEL. @Jason Wang
2. Make things clearer. @Jason Wang @Michael S. Tsirkin
3. Keep the possibility to use inner hash for automatic receive steering. @Jason Wang
4. Add the "Tunnel packet" paragraph to avoid repeating the GRE etc. many times. @Michael S. Tsirkinv2:
1. Add a feature bit for GRE/VXLAN/GENEVE inner hash. @Jason Wang
2. Chang \field{hash_tunnel} to \field{hash_report_tunnel}. @Jason Wang, @Michael S. Tsirkinv1:
1. Remove the patch for the bitmask fix. @Michael S. Tsirkin
2. Clarify some paragraphs. @Jason Wang
3. Add \field{hash_tunnel} andVIRTIO_NET_HASH_REPORT_GRE. @Yuri Benditovichcontent.tex | 191 +++++++++++++++++++++++++++++++++++++++++++++-- introduction.tex | 19 +++++ 2 files changed, 203 insertions(+), 7 deletions(-)content.tex +0
-0@@ -3084,6 +3084,9 @@ \subsection{Feature bits}\label{sec:Device Types / Network Device / Feature bits \item[VIRTIO_NET_F_CTRL_MAC_ADDR(23)] Set MAC address through control channel.Quoted 4 lines
content.tex \subsection{Feature bits}\label{sec:Device Types / Network Device / Feature bits · line 3088
Quoted 2 lines
content.tex +2
-0+\item[VIRTIO_NET_F_HASH_TUNNEL(52)] Device supports inner + header hash for GRE, VXLAN and GENEVE tunnel-encapsulated packets.I would probably drop the list of tunnel types here.
Quoted 8 lines
Do you mean to use "Device supports inner header hash for tunnel-encapsulated packets." instead? Why? We do want to use this feature bit to indicate that the device supports inner hashing of GRE, VXLAN and GENEVE encapsulated packets. As in the v3 discussion
https://lists.oasis-open.org/archives/virtio-dev/202212/msg00024.html , we discussed usingVIRTIO_NET_F_HASH_TUNNELto replaceVIRTIO_NET_F_HASH_GRE_VXLAN_GENEVE_INNERand plan to useVIRTIO_NET_F_HASH_TUNNEL_XYZfor future extensions.Quoted 3 lines
So imagine we add a new tunnel type. Let's say there's VXLAN v2. why would we need a new feature bit? I think a new hash type will be sufficient. No?
If the description for
VIRTIO_NET_F_HASH_TUNNELis as follows:
"[VIRTIO_NET_F_HASH_TUNNEL(52)] Device supports inner header hash for tunnel-encapsulated packets.". Then the following may happen
1. For VXLANv2, if both device src and device dst have negotiated this feature, it is assumed that device src supports VXLAN and VXLANv2, but device dst may only support VXLAN, not VXLANv2.
2. For other encapsulation protocols such as ip in ip, after device src and device dst have negotiated this feature, it is assumed that device src supports GRE, VXLAN, GENEVE and ip in ip, but it is not clear that device dst also supports ip in ip. Especially when migrating, this can lead to inconsistencies in live migrations.
Quoted 5 lines
Quoted 2 lines
it does not matter. e.g. if device supports GRE but not VXLAN then feature bit it set but hash types are different.
But the hash will be used for RSS steering. Consider the steering on ipip work on src but not on dst it would break application logic.
Exactly. And for this reason just making sure feature bits match is not sufficient for migration. It follows that hash types
must also match, and therefore there is no need to invent
new feature bits for migration if all we are doing is adding new hash types.
Quoted 3 lines
Quoted 1 line
and this is not different from any other offload we have.
But we have different feature bits for TCPv4/v6.
For segmentation but not for hash. Basically because we don't have segmentation types analogous to hash types.
Quoted 11 lines
Quoted 11 lines
Generally the only reason we even use a new feature bit for this tunneling is because the header structure is a bit different so this kind of makes sense.
Quoted 5 lines
Yes, this looks like the only way that I can think of to keep migration compatibility in an easy way.
Thanks
Quoted 8 lines
Quoted 3 lines
Jason migration compatibility must also check hash types. There's no chance with the current hash offload scheme to only rely on feature bits for migration compatibility.
This will give extra pressure on the management stack, e.g it requires the device to have an out of spec way for introspection.
Thanks
As I tried to explain this is already the case. Feature bits do not describe device capabilities fully, some of them are in config space.
Quoted 17 lines
Quoted 17 lines
Quoted 16 lines
Quoted 16 lines
So, I think it's better to keep the original description:
"[VIRTIO_NET_F_HASH_TUNNEL(52)] Device supports inner header hash for GRE, VXLAN and GENEVE tunnel-encapsulated packets."Thanks.
Quoted 8 lines
--
MST---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Quoted 7 lines · Michael S. Tsirkin
Quoted 4 lines
This will give extra pressure on the management stack, e.g it requires the device to have an out of spec way for introspection.
Thanks
As I tried to explain this is already the case. Feature bits do not describe device capabilities fully, some of them are in config space.
To be precise, this does not necessarily require introspection, but it does require management control over config space
such as supported hash types just like it has control over feature bits. E.g. QEMU currently seems to hard-code these to
#define VIRTIO_NET_RSS_SUPPORTED_HASHES (VIRTIO_NET_RSS_HASH_TYPE_IPv4 | \
VIRTIO_NET_RSS_HASH_TYPE_TCPv4 | \
VIRTIO_NET_RSS_HASH_TYPE_UDPv4 | \
VIRTIO_NET_RSS_HASH_TYPE_IPv6 | \
VIRTIO_NET_RSS_HASH_TYPE_TCPv6 | \
VIRTIO_NET_RSS_HASH_TYPE_UDPv6 | \
VIRTIO_NET_RSS_HASH_TYPE_IP_EX | \
VIRTIO_NET_RSS_HASH_TYPE_TCP_EX | \
VIRTIO_NET_RSS_HASH_TYPE_UDP_EX)but there's no reason not to give management control over these.
Signature · 1 line
MST
Quoted 23 lines · Michael S. Tsirkin
Quoted 7 lines · Michael S. Tsirkin
Quoted 4 lines
This will give extra pressure on the management stack, e.g it requires the device to have an out of spec way for introspection.
Thanks
As I tried to explain this is already the case. Feature bits do not describe device capabilities fully, some of them are in config space.
To be precise, this does not necessarily require introspection, but it does require management control over config space
such as supported hash types just like it has control over feature bits. E.g. QEMU currently seems to hard-code these to#define VIRTIO_NET_RSS_SUPPORTED_HASHES (VIRTIO_NET_RSS_HASH_TYPE_IPv4 | \ VIRTIO_NET_RSS_HASH_TYPE_TCPv4 | \ VIRTIO_NET_RSS_HASH_TYPE_UDPv4 | \ VIRTIO_NET_RSS_HASH_TYPE_IPv6 | \ VIRTIO_NET_RSS_HASH_TYPE_TCPv6 | \ VIRTIO_NET_RSS_HASH_TYPE_UDPv6 | \ VIRTIO_NET_RSS_HASH_TYPE_IP_EX | \ VIRTIO_NET_RSS_HASH_TYPE_TCP_EX | \ VIRTIO_NET_RSS_HASH_TYPE_UDP_EX)
Quoted 1 line
but there's no reason not to give management control over these.
Yes, QEMU has requirements for live migration: the PCI config space will be checked in get_pci_config_device(), and if src and dst are inconsistent, it will prompt that the live migration failed.
In fact, this is also done within our group. Live migration requires that the two VMs have the same rss configuration, otherwise the migration will fail.
Therefore, it seems that we can regularize the description of VIRTIO_NET_F_HASH_TUNNEL into "[VIRTIO_NET_F_HASH_TUNNEL(52)] Device supports inner header hash for tunnel-encapsulated packets.", and use different hash_types to help the migration determine whether it can succeed.
Thanks.
Quoted 3 lines
Signature · 2 lines
-- MST
Quoted 22 lines · Heng Qi
Quoted 21 lines · Michael S. Tsirkin
Quoted 6 lines · Michael S. Tsirkin
Quoted 4 lines
This will give extra pressure on the management stack, e.g it requires the device to have an out of spec way for introspection.
Thanks
As I tried to explain this is already the case. Feature bits do not describe device capabilities fully, some of them are in config space.
To be precise, this does not necessarily require introspection, but it does require management control over config space
such as supported hash types just like it has control over feature bits. E.g. QEMU currently seems to hard-code these to#define VIRTIO_NET_RSS_SUPPORTED_HASHES (VIRTIO_NET_RSS_HASH_TYPE_IPv4 | \ VIRTIO_NET_RSS_HASH_TYPE_TCPv4 | \ VIRTIO_NET_RSS_HASH_TYPE_UDPv4 | \ VIRTIO_NET_RSS_HASH_TYPE_IPv6 | \ VIRTIO_NET_RSS_HASH_TYPE_TCPv6 | \ VIRTIO_NET_RSS_HASH_TYPE_UDPv6 | \ VIRTIO_NET_RSS_HASH_TYPE_IP_EX | \ VIRTIO_NET_RSS_HASH_TYPE_TCP_EX | \ VIRTIO_NET_RSS_HASH_TYPE_UDP_EX)
Quoted 4 lines
Quoted 1 line
but there's no reason not to give management control over these.
Yes, QEMU has requirements for live migration: the PCI config space will be checked in
get_pci_config_device(), and if src and dst are inconsistent, it will prompt that the live migration failed.
To be clearer, I mean \filed{supported_hash_types} in structure virtio_net_config.
Thanks.
Quoted 26 lines
In fact, this is also done within our group. Live migration requires that the two VMs have the same rss configuration, otherwise the migration will fail.
Therefore, it seems that we can regularize the description of
VIRTIO_NET_F_HASH_TUNNELinto "[VIRTIO_NET_F_HASH_TUNNEL(52)] Device supports inner header hash for tunnel-encapsulated packets.", and use differenthash_typesto help the migration determine whether it can succeed.Thanks.
Quoted 2 lines
Signature · 1 line
MSTThis publicly archived list offers a means to provide input to the OASIS Virtual I/O Device (VIRTIO) TC.
In order to verify user consent to the Feedback License terms and to minimize spam in the list archive, subscription is required before posting.
Subscribe: [email protected]
Unsubscribe: [email protected]
List help: [email protected]
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/
Quoted 4 lines · Heng Qi
在 2023/1/10 下午3:26, Heng Qi 写道:
Quoted 8 lines
Quoted 8 lines · Michael S. Tsirkin
Quoted 7 lines · Michael S. Tsirkin
Quoted 6 lines
Quoted 4 lines
This will give extra pressure on the management stack, e.g it requires the device to have an out of spec way for introspection.
Thanks
As I tried to explain this is already the case. Feature bits do not describe device capabilities fully, some of them are in config space.
Yes.
Quoted 14 lines
Quoted 14 lines
Quoted 14 lines
To be precise, this does not necessarily require introspection, but it does require management control over config space
such as supported hash types just like it has control over feature bits. E.g. QEMU currently seems to hard-code these to#define VIRTIO_NET_RSS_SUPPORTED_HASHES (VIRTIO_NET_RSS_HASH_TYPE_IPv4 | \ VIRTIO_NET_RSS_HASH_TYPE_TCPv4 | \ VIRTIO_NET_RSS_HASH_TYPE_UDPv4 | \ VIRTIO_NET_RSS_HASH_TYPE_IPv6 | \ VIRTIO_NET_RSS_HASH_TYPE_TCPv6 | \ VIRTIO_NET_RSS_HASH_TYPE_UDPv6 | \ VIRTIO_NET_RSS_HASH_TYPE_IP_EX | \ VIRTIO_NET_RSS_HASH_TYPE_TCP_EX | \ VIRTIO_NET_RSS_HASH_TYPE_UDP_EX)
Quoted 1 line
Quoted 1 line
Quoted 1 line
but there's no reason not to give management control over these.
Note that the management expects the migration compatibility to work with machine types. So it needs a way to disable some tunnel hash types to make it work for old machine types.
Quoted 3 lines
Quoted 3 lines
Yes, QEMU has requirements for live migration: the PCI config space will be checked in
get_pci_config_device(), and if src and dst are inconsistent, it will prompt that the live migration failed.
It might be too late since it can't work for the second run (unlike subsection).
Quoted 3 lines
To be clearer, I mean \filed{supported_hash_types} in structure
virtio_net_config.
Yes.
Thanks
Quoted 30 lines
Thanks.
Quoted 26 lines
In fact, this is also done within our group. Live migration requires that the two VMs have the same rss configuration, otherwise the migration will fail.
Therefore, it seems that we can regularize the description of
VIRTIO_NET_F_HASH_TUNNELinto "[VIRTIO_NET_F_HASH_TUNNEL(52)] Device supports inner header hash for tunnel-encapsulated packets.", and use differenthash_typesto help the migration determine whether it can succeed.Thanks.
Quoted 2 lines
--
MSTThis publicly archived list offers a means to provide input to the OASIS Virtual I/O Device (VIRTIO) TC.
In order to verify user consent to the Feedback License terms and to minimize spam in the list archive, subscription is required before posting.
Subscribe: [email protected]
Unsubscribe: [email protected]
List help: [email protected]
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/
Quoted 30 lines · Jason Wang
Quoted 4 lines · Heng Qi
在 2023/1/10 下午3:26, Heng Qi 写道:
Quoted 8 lines
Quoted 8 lines · Michael S. Tsirkin
Quoted 7 lines · Michael S. Tsirkin
Quoted 6 lines
Quoted 4 lines
This will give extra pressure on the management stack, e.g it requires the device to have an out of spec way for introspection.
Thanks
As I tried to explain this is already the case. Feature bits do not describe device capabilities fully, some of them are in config space.
Yes.
Quoted 14 lines
Quoted 14 lines
Quoted 14 lines
To be precise, this does not necessarily require introspection, but it does require management control over config space
such as supported hash types just like it has control over feature bits. E.g. QEMU currently seems to hard-code these to#define VIRTIO_NET_RSS_SUPPORTED_HASHES (VIRTIO_NET_RSS_HASH_TYPE_IPv4 | \ VIRTIO_NET_RSS_HASH_TYPE_TCPv4 | \ VIRTIO_NET_RSS_HASH_TYPE_UDPv4 | \ VIRTIO_NET_RSS_HASH_TYPE_IPv6 | \ VIRTIO_NET_RSS_HASH_TYPE_TCPv6 | \ VIRTIO_NET_RSS_HASH_TYPE_UDPv6 | \ VIRTIO_NET_RSS_HASH_TYPE_IP_EX | \ VIRTIO_NET_RSS_HASH_TYPE_TCP_EX | \ VIRTIO_NET_RSS_HASH_TYPE_UDP_EX)
Quoted 5 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line
but there's no reason not to give management control over these.
Note that the management expects the migration compatibility to work with machine types. So it needs a way to disable some tunnel hash types to make it work for old machine types.
yes.
This means qemu will need to create properties for these things and control through machine type compatibility machinery. For those not hacking qemu - "machine type" is
a string roughly describing a version of guest/host interface used.
Quoted 5 lines
Quoted 3 lines
Quoted 3 lines
Yes, QEMU has requirements for live migration: the PCI config space will be checked in
get_pci_config_device(), and if src and dst are inconsistent, it will prompt that the live migration failed.It might be too late since it can't work for the second run (unlike subsection).
This is really a low level detail of qemu. I'm not sure how important this is for the spec.
Quoted 38 lines
Quoted 3 lines
To be clearer, I mean \filed{supported_hash_types} in structure
virtio_net_config.Yes.
Thanks
Quoted 30 lines
Thanks.
Quoted 26 lines
In fact, this is also done within our group. Live migration requires that the two VMs have the same rss configuration, otherwise the migration will fail.
Therefore, it seems that we can regularize the description of
VIRTIO_NET_F_HASH_TUNNELinto "[VIRTIO_NET_F_HASH_TUNNEL(52)] Device supports inner header hash for tunnel-encapsulated packets.", and use differenthash_typesto help the migration determine whether it can succeed.Thanks.
Quoted 2 lines
--
MSTThis publicly archived list offers a means to provide input to the OASIS Virtual I/O Device (VIRTIO) TC.
In order to verify user consent to the Feedback License terms and to minimize spam in the list archive, subscription is required before posting.
Subscribe: [email protected]
Unsubscribe: [email protected]
List help: [email protected]
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/
Quoted 291 lines · Heng Qi
If the tunnel is used to encapsulate the packets, the hash calculated using the outer header of the receive packets is always fixed for the same flow packets, i.e. they will be steered to the same receive queue.
We add a feature bit
VIRTIO_NET_F_HASH_TUNNELand related bitmasks in \field{hash_types}, which instructs the device to calculate the hash using the inner headers of tunnel-encapsulated packets. Besides, values in \field{hash_report_tunnel} are added to report tunnel types.Reviewed-by Jason Wang <[email protected]>Signed-off-by Heng Qi <[email protected]>Signed-off-by Xuan Zhuo <[email protected]>---
v6:
1. Modify the wording of some sentences for clarity. @Michael S. Tsirkin
2. Fix some syntax issues. @Michael S. Tsirkinv5:
1. Fix some syntax and capitalization issues. @Michael S. Tsirkin
2. Use encapsulated/encaptulation uniformly. @Michael S. Tsirkin
3. Move the links to introduction section. @Michael S. Tsirkin
4. Clarify some sentences. @Michael S. Tsirkinv4:
1. Clarify some paragraphs. @Cornelia Huck
2. Fix the u8 type. @Cornelia Huckv3:
1. RenameVIRTIO_NET_F_HASH_GRE_VXLAN_GENEVE_INNERtoVIRTIO_NET_F_HASH_TUNNEL. @Jason Wang
2. Make things clearer. @Jason Wang @Michael S. Tsirkin
3. Keep the possibility to use inner hash for automatic receive steering. @Jason Wang
4. Add the "Tunnel packet" paragraph to avoid repeating the GRE etc. many times. @Michael S. Tsirkinv2:
1. Add a feature bit for GRE/VXLAN/GENEVE inner hash. @Jason Wang
2. Chang \field{hash_tunnel} to \field{hash_report_tunnel}. @Jason Wang, @Michael S. Tsirkinv1:
1. Remove the patch for the bitmask fix. @Michael S. Tsirkin
2. Clarify some paragraphs. @Jason Wang
3. Add \field{hash_tunnel} andVIRTIO_NET_HASH_REPORT_GRE. @Yuri Benditovichcontent.tex | 191 +++++++++++++++++++++++++++++++++++++++++++++-- introduction.tex | 19 +++++ 2 files changed, 203 insertions(+), 7 deletions(-)content.tex +168
-7@@ -3084,6 +3084,9 @@ \subsection{Feature bits}\label{sec:Device Types / Network Device / Feature bits \item[VIRTIO_NET_F_CTRL_MAC_ADDR(23)] Set MAC address through control channel. +\item[VIRTIO_NET_F_HASH_TUNNEL(52)] Device supports inner + header hash for GRE, VXLAN and GENEVE tunnel-encapsulated packets. + \item[VIRTIO_NET_F_NOTF_COAL(53)] Device supports notifications coalescing. \item[VIRTIO_NET_F_GUEST_USO4 (54)] Driver can receive USOv4 packets.@@ -3095,7 +3098,8 @@ \subsection{Feature bits}\label{sec:Device Types / Network Device / Feature bits to several segments when each of these smaller packets has UDP header. \item[VIRTIO_NET_F_HASH_REPORT(57)] Device can report per-packet hash - value and a type of calculated hash. + value, a type of calculated hash, and, if VIRTIO_NET_F_HASH_TUNNEL + is negotiated, an encapsulation packet type. \item[VIRTIO_NET_F_GUEST_HDRLEN(59)] Driver can provide the exact \field{hdr_len} value. Device benefits from knowing the exact header length.@@ -3140,6 +3144,7 @@ \subsubsection{Feature bit requirements}\label{sec:Device Types / Network Device \item[VIRTIO_NET_F_NOTF_COAL] Requires VIRTIO_NET_F_CTRL_VQ. \item[VIRTIO_NET_F_RSC_EXT] Requires VIRTIO_NET_F_HOST_TSO4 or VIRTIO_NET_F_HOST_TSO6. \item[VIRTIO_NET_F_RSS] Requires VIRTIO_NET_F_CTRL_VQ. +\item[VIRTIO_NET_F_HASH_TUNNEL] Requires VIRTIO_NET_F_CTRL_VQ. \end{description} \subsubsection{Legacy Interface: Feature bits}\label{sec:Device Types / Network Device / Feature bits / Legacy Interface: Feature bits}@@ -3386,7 +3391,8 @@ \subsection{Device Operation}\label{sec:Device Types / Network Device / Device O le16 num_buffers; le32 hash_value; (Only if VIRTIO_NET_F_HASH_REPORT negotiated) le16 hash_report; (Only if VIRTIO_NET_F_HASH_REPORT negotiated) - le16 padding_reserved; (Only if VIRTIO_NET_F_HASH_REPORT negotiated) + u8 hash_report_tunnel; (Only if VIRTIO_NET_F_HASH_REPORT negotiated, only valid of VIRTIO_NET_F_HASH_TUNNEL negotiated) + u8 padding_reserved; (Only if VIRTIO_NET_F_HASH_REPORT negotiated) }; \end{lstlisting}@@ -3837,7 +3843,8 @@ \subsubsection{Processing of Incoming Packets}\label{sec:Device Types / Network A device attempts to calculate a per-packet hash in the following cases: \begin{itemize} \item The feature VIRTIO_NET_F_RSS was negotiated. The device uses the hash to determine the receive virtqueue to place incoming packets. -\item The feature VIRTIO_NET_F_HASH_REPORT was negotiated. The device reports the hash value and the hash type with the packet. +\item The feature VIRTIO_NET_F_HASH_REPORT was negotiated. The device reports the hash value and the hash type. If additionally +VIRTIO_NET_F_HASH_TUNNEL was negotiated, the device reports the encapsulation type as well. \end{itemize} If the feature VIRTIO_NET_F_RSS was negotiated:@@ -3863,8 +3870,36 @@ \subsubsection{Processing of Incoming Packets}\label{sec:Device Types / Network \ref{sec:Device Types / Network Device / Device Operation / Control Virtqueue / Automatic receive steering in multiqueue mode / Hash calculation}. \end{itemize} +\subparagraph{Tunnel/Encapsulated packet} +\label{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / Tunnel/Encapsulated packet} +A tunnel packet is encapsulated from the original packet based on the tunneling +protocol (only a single level of encapsulation is currently supported). The +encapsulated packet contains an outer header and an inner header, and the device +calculates the hash over either the inner header or the outer header. + +When the feature VIRTIO_NET_F_HASH_TUNNEL is negotiated and the corresponding +encapsulation type is set in \field{hash_types}, the hash for a specific type of +encapsulated packet is calculated over the inner as opposed to outer header. +Supported encapsulation types are listed in \ref{sec:Device Types / Network Device / +Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / +Supported/enabled hash types}. + +If both VIRTIO_NET_F_HASH_REPORT and VIRTIO_NET_F_HASH_TUNNEL are negotiated, +the device can support inner hash calculation for \hyperref[intro:GRE]{[GRE]}, +\hyperref[intro:VXLAN]{[VXLAN]} and \hyperref[intro:GENEVE]{[GENEVE]} +encapsulated packets, and the corresponding encapsulation type in \field{hash_types} +is VIRTIO_NET_HASH_TYPE_{GRE, VXLAN, GENEVE}_INNER respectively. The value in +\field{hash_report_tunnel} is VIRTIO_NET_HASH_REPORT_{NONE, GRE, VXLAN, GENEVE} respectively. + +If VIRTIO_NET_F_HASH_REPORT is negotiated but VIRTIO_NET_F_HASH_TUNNEL is not +negotiated, the device calculates the hash over the outer header, and +\field{hash_report} reports the hash type. \field{hash_report_tunnel} +is no longer valid. + \subparagraph{Supported/enabled hash types} \label{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / Supported/enabled hash types} +This paragraph relies on definitions from \hyperref[intro:IP]{[IP]}, +\hyperref[intro:UDP]{[UDP]} and \hyperref[intro:TCP]{[TCP]}. Hash types applicable for IPv4 packets: \begin{lstlisting} #define VIRTIO_NET_HASH_TYPE_IPv4 (1 << 0)@@ -3884,6 +3919,22 @@ \subsubsection{Processing of Incoming Packets}\label{sec:Device Types / Network #define VIRTIO_NET_HASH_TYPE_UDP_EX (1 << 8) \end{lstlisting} +If the feature VIRTIO_NET_F_HASH_TUNNEL is negotiated, the encapsulation +hash type below indicates that the hash is calculated over the inner +header of the encapsulated packet: +Hash type applicable for inner payload of the gre-encapsulated packet +\begin{lstlisting} +#define VIRTIO_NET_HASH_TYPE_GRE_INNER (1 << 9) +\end{lstlisting} +Hash type applicable for inner payload of the vxlan-encapsulated packet +\begin{lstlisting} +#define VIRTIO_NET_HASH_TYPE_VXLAN_INNER (1 << 10) +\end{lstlisting} +Hash type applicable for inner payload of the geneve-encapsulated packet +\begin{lstlisting} +#define VIRTIO_NET_HASH_TYPE_GENEVE_INNER (1 << 11) +\end{lstlisting} + \subparagraph{IPv4 packets} \label{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / IPv4 packets} The device calculates the hash on IPv4 packets according to 'Enabled hash types' bitmask as follows:@@ -3975,15 +4026,125 @@ \subsubsection{Processing of Incoming Packets}\label{sec:Device Types / Network (see \ref{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / IPv6 packets without extension header}). \end{itemize} +\subparagraph{Inner payload of an encapsulated packet} +\label{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / Inner payload of the encapsulated packet} +If the feature VIRTIO_NET_F_HASH_TUNNEL is negotiated and the corresponding +encapsulation hash type is set in \field{hash_types}, the device calculates the +inner header hash on an encapsulated packet (See \ref{sec:Device Types +/ Network Device / Device Operation / Processing of Incoming Packets / +Hash calculation for incoming packets / Tunnel/Encapsulated packet}) as follows: + +The device calculates the hash on the inner IPv4 packet of an encapsulated packet +according to 'Enabled hash types' bitmask as follows: +\begin{itemize} + \item If VIRTIO_NET_HASH_TYPE_TCPv4 is set and the encapsulated packet has an inner + TCPv4 header, the hash is calculated over the following fields: + \begin{itemsize} + \item inner Source IP address + \item inner Destination IP address + \item inner Source TCP port + \item inner Destination TCP port + \end{itemsize} + \item Else if VIRTIO_NET_HASH_TYPE_UDPv4 is set and the encapsulated packet has an + inner UDPv4 header, the hash is calculated over the following fields: + \begin{itemsize} + \item inner Source IP address + \item inner Destination IP address + \item inner Source UDP port + \item inner Destination UDP port + \end{itemize} + \item Else if VIRTIO_NET_HASH_TYPE_IPv4 is set, the hash is calculated over the + following fields: + \begin{itemsize} + \item inner Source IP address + \item inner Destination IP address + \end{itemsize} + \item Else the device does not calculate the hash +\end{itemize} + +The device calculates the hash on the inner IPv6 packet without an extension header +of an encapsulated packet according to 'Enabled hash types' bitmask as follows: +\begin{itemize} + \item If VIRTIO_NET_HASH_TYPE_TCPv6 is set and the encapsulated packet has an inner + TCPv6 header, the hash is calculated over the following fields: + \begin{itemsize} + \item inner Source IPv6 address + \item inner Destination IPv6 address + \item inner Source TCP port + \item inner Destination TCP port + \end{itemsize} + \item Else if VIRTIO_NET_HASH_TYPE_UDPv6 is set and the encapsulated packet has an + inner UDPv6 header, the hash is calculated over the following fields: + \begin{itemsize} + \item inner Source IPv6 address + \item inner Destination IPv6 address + \item inner Source UDP port + \item inner Destination UDP port + \end{itemize} + \item Else if VIRTIO_NET_HASH_TYPE_IPv6 is set, the hash is calculated over the + following fields: + \begin{itemsize} + \item inner Source IPv6 address + \item inner Destination IPv6 address + \end{itemsize} + \item Else the device does not calculate the hash +\end{itemize} + +The device calculates the hash on the inner IPv6 packet with an extension header +of an encapsulated packet according to 'Enabled hash types' bitmask as follows: +\begin{itemsize} + \item If VIRTIO_NET_HASH_TYPE_TCP_EX is set and the encapsulated packet has an inner + TCPv6 header, the hash is calculated over the following fields: + \begin{itemize} + \item Home address from the home address option in the inner IPv6 destination + options header. If the inner extension header is not present, use the + inner Source IPv6 address. + \item IPv6 address that is contained in the Routing-Header-Type-2 from the + associated inner extension header. If the inner extension header is not + present, use the inner Destination IPv6 address. + \item inner Source TCP port + \item inner Destination TCP port + \end{itemize} + \item Else if VIRTIO_NET_HASH_TYPE_UDP_EX is set and the encapsulated packet has an inner + UDPv6 header, the hash is calculated over the following fields: + \begin{itemsize} + \item Home address from the home address option in the inner IPv6 destination + options header. If the inner extension header is not present, use the + inner Source IPv6 address. + \item IPv6 address that is contained in the Routing-Header-Type-2 from the + associated inner extension header. If the inner extension header is not + present, use the inner Destination IPv6 address. + \item inner Source UDP port + \item inner Destination UDP port + \end{itemize} + \item Else if VIRTIO_NET_HASH_TYPE_IP_EX is set, the hash is calculated over the + following fields: + \begin{itemsize} + \item Home address from the home address option in the inner IPv6 destination + options header. If the inner extension header is not present, use the + inner Source IPv6 address. + \item IPv6 address that is contained in the Routing-Header-Type-2 from the + associated inner extension header. If the inner extension header is not + present, use the inner Destination IPv6 address. + \end{itemize} + \item Else skip the inner IPv6 extension header and calculate the inner header hash as + defined for an encapsulated packet whose inner payload is an IPv6 packet without + an extension header. +\end{itemsize} + \paragraph{Hash reporting for incoming packets} \label{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash reporting for incoming packets} -If VIRTIO_NET_F_HASH_REPORT was negotiated and - the device has calculated the hash for the packet, the device fills \field{hash_report} with the report type of calculated hash -and \field{hash_value} with the value of calculated hash. +If VIRTIO_NET_F_HASH_REPORT was negotiated and the device has calculated the +hash for the packet, the device fills \field{hash_report} with the report type +of calculated hash, and \field{hash_value} with the value of calculated hash. +Also, if VIRTIO_NET_F_HASH_TUNNEL was negotiated, the device needs to fill +\field{hash_report_tunnel} with the report type of the encapsulated packet, and +it is set to VIRTIO_NET_HASH_REPORT_TUNNEL_NONE for the unencapsulated packet. If VIRTIO_NET_F_HASH_REPORT was negotiated but due to any reason the -hash was not calculated, the device sets \field{hash_report} to VIRTIO_NET_HASH_REPORT_NONE. +hash was not calculated, the device sets \field{hash_report} to VIRTIO_NET_HASH_REPORT_NONE, +and sets \field{hash_report_tunnel} to VIRTIO_NET_HASH_REPORT_TUNNEL_NONE. Possible values that the device can report in \field{hash_report} are defined below. They correspond to supported hash types defined in
content.tex \subsubsection{Processing of Incoming Packets}\label{sec:Device Types / Network · line 4171
Quoted 6 lines
content.tex +3
-0@@ -4005,6 +4166,22 @@ \subsubsection{Processing of Incoming Packets}\label{sec:Device Types / Network #define VIRTIO_NET_HASH_REPORT_UDPv6_EX 9 \end{lstlisting} +If \field{hash_report} differs from VIRTIO_NET_HASH_REPORT_NONE, +\field{hash_report_tunnel} can report the type of the encapsulated +packet to the driver over the inner header hash calculation.
this sentence is a bit confused btw. so this reports the type of the encapsulated packet. but what does "over the inner header hash calculation." mean? what happens over what?
Quoted 48 lines
content.tex +13
-0+Possible values that the device can report in \field{hash_report_tunnel} +are defined below: + +\begin{lstlisting} +#define VIRTIO_NET_HASH_REPORT_TUNNEL_NONE 0 +#define VIRTIO_NET_HASH_REPORT_GRE 1 +#define VIRTIO_NET_HASH_REPORT_VXLAN 2 +#define VIRTIO_NET_HASH_REPORT_GENEVE 3 +\end{lstlisting} + +They correspond to supported hash types defined in +\ref{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / Supported/enabled hash types}. + \subsubsection{Control Virtqueue}\label{sec:Device Types / Network Device / Device Operation / Control Virtqueue} The driver uses the control virtqueue (if VIRTIO_NET_F_CTRL_VQ isintroduction.tex +19
-0@@ -98,6 +98,25 @@ \section{Normative References}\label{sec:Normative References} \phantomsection\label{intro:SEC1}\textbf{[SEC1]} & Standards for Efficient Cryptography Group(SECG), ``SEC1: Elliptic Cureve Cryptography'', Version 1.0, September 2000. \newline\url{https://www.secg.org/sec1-v2.pdf}\\ + \phantomsection\label{intro:GRE}\textbf{[GRE]} & + Generic Routing Encapsulation + \newline\url{https://datatracker.ietf.org/doc/rfc2784/}\\ + \phantomsection\label{intro:VXLAN}\textbf{[VXLAN]} & + Virtual eXtensible Local Area Network + \newline\url{https://datatracker.ietf.org/doc/rfc7348/}\\ + \phantomsection\label{intro:GENEVE}\textbf{[GENEVE]} & + Generic Network Virtualization Encapsulation + \newline\url{https://datatracker.ietf.org/doc/rfc8926/}\\ + \phantomsection\label{intro:IP}\textbf{[IP]} & + INTERNET PROTOCOL + \newline\url{https://www.rfc-editor.org/rfc/rfc791}\\ + \phantomsection\label{intro:UDP}\textbf{[UDP]} & + User Datagram Protocol + \newline\url{https://www.rfc-editor.org/rfc/rfc768}\\ + \phantomsection\label{intro:TCP}\textbf{[TCP]} & + TRANSMISSION CONTROL PROTOCOL + \newline\url{https://www.rfc-editor.org/rfc/rfc793}\\ + \end{longtable}Signature · 1 line
2.19.1.6.gb485710b
Quoted 295 lines · Michael S. Tsirkin
Quoted 294 lines · Heng Qi
If the tunnel is used to encapsulate the packets, the hash calculated using the outer header of the receive packets is always fixed for the same flow packets, i.e. they will be steered to the same receive queue.
We add a feature bit
VIRTIO_NET_F_HASH_TUNNELand related bitmasks in \field{hash_types}, which instructs the device to calculate the hash using the inner headers of tunnel-encapsulated packets. Besides, values in \field{hash_report_tunnel} are added to report tunnel types.Reviewed-by Jason Wang <[email protected]>Signed-off-by Heng Qi <[email protected]>Signed-off-by Xuan Zhuo <[email protected]>---
v6:
1. Modify the wording of some sentences for clarity. @Michael S. Tsirkin
2. Fix some syntax issues. @Michael S. Tsirkinv5:
1. Fix some syntax and capitalization issues. @Michael S. Tsirkin
2. Use encapsulated/encaptulation uniformly. @Michael S. Tsirkin
3. Move the links to introduction section. @Michael S. Tsirkin
4. Clarify some sentences. @Michael S. Tsirkinv4:
1. Clarify some paragraphs. @Cornelia Huck
2. Fix the u8 type. @Cornelia Huckv3:
1. RenameVIRTIO_NET_F_HASH_GRE_VXLAN_GENEVE_INNERtoVIRTIO_NET_F_HASH_TUNNEL. @Jason Wang
2. Make things clearer. @Jason Wang @Michael S. Tsirkin
3. Keep the possibility to use inner hash for automatic receive steering. @Jason Wang
4. Add the "Tunnel packet" paragraph to avoid repeating the GRE etc. many times. @Michael S. Tsirkinv2:
1. Add a feature bit for GRE/VXLAN/GENEVE inner hash. @Jason Wang
2. Chang \field{hash_tunnel} to \field{hash_report_tunnel}. @Jason Wang, @Michael S. Tsirkinv1:
1. Remove the patch for the bitmask fix. @Michael S. Tsirkin
2. Clarify some paragraphs. @Jason Wang
3. Add \field{hash_tunnel} andVIRTIO_NET_HASH_REPORT_GRE. @Yuri Benditovichcontent.tex | 191 +++++++++++++++++++++++++++++++++++++++++++++-- introduction.tex | 19 +++++ 2 files changed, 203 insertions(+), 7 deletions(-)content.tex +168
-7@@ -3084,6 +3084,9 @@ \subsection{Feature bits}\label{sec:Device Types / Network Device / Feature bits \item[VIRTIO_NET_F_CTRL_MAC_ADDR(23)] Set MAC address through control channel. +\item[VIRTIO_NET_F_HASH_TUNNEL(52)] Device supports inner + header hash for GRE, VXLAN and GENEVE tunnel-encapsulated packets. + \item[VIRTIO_NET_F_NOTF_COAL(53)] Device supports notifications coalescing. \item[VIRTIO_NET_F_GUEST_USO4 (54)] Driver can receive USOv4 packets.@@ -3095,7 +3098,8 @@ \subsection{Feature bits}\label{sec:Device Types / Network Device / Feature bits to several segments when each of these smaller packets has UDP header. \item[VIRTIO_NET_F_HASH_REPORT(57)] Device can report per-packet hash - value and a type of calculated hash. + value, a type of calculated hash, and, if VIRTIO_NET_F_HASH_TUNNEL + is negotiated, an encapsulation packet type. \item[VIRTIO_NET_F_GUEST_HDRLEN(59)] Driver can provide the exact \field{hdr_len} value. Device benefits from knowing the exact header length.@@ -3140,6 +3144,7 @@ \subsubsection{Feature bit requirements}\label{sec:Device Types / Network Device \item[VIRTIO_NET_F_NOTF_COAL] Requires VIRTIO_NET_F_CTRL_VQ. \item[VIRTIO_NET_F_RSC_EXT] Requires VIRTIO_NET_F_HOST_TSO4 or VIRTIO_NET_F_HOST_TSO6. \item[VIRTIO_NET_F_RSS] Requires VIRTIO_NET_F_CTRL_VQ. +\item[VIRTIO_NET_F_HASH_TUNNEL] Requires VIRTIO_NET_F_CTRL_VQ. \end{description} \subsubsection{Legacy Interface: Feature bits}\label{sec:Device Types / Network Device / Feature bits / Legacy Interface: Feature bits}@@ -3386,7 +3391,8 @@ \subsection{Device Operation}\label{sec:Device Types / Network Device / Device O le16 num_buffers; le32 hash_value; (Only if VIRTIO_NET_F_HASH_REPORT negotiated) le16 hash_report; (Only if VIRTIO_NET_F_HASH_REPORT negotiated) - le16 padding_reserved; (Only if VIRTIO_NET_F_HASH_REPORT negotiated) + u8 hash_report_tunnel; (Only if VIRTIO_NET_F_HASH_REPORT negotiated, only valid of VIRTIO_NET_F_HASH_TUNNEL negotiated) + u8 padding_reserved; (Only if VIRTIO_NET_F_HASH_REPORT negotiated) }; \end{lstlisting}@@ -3837,7 +3843,8 @@ \subsubsection{Processing of Incoming Packets}\label{sec:Device Types / Network A device attempts to calculate a per-packet hash in the following cases: \begin{itemize} \item The feature VIRTIO_NET_F_RSS was negotiated. The device uses the hash to determine the receive virtqueue to place incoming packets. -\item The feature VIRTIO_NET_F_HASH_REPORT was negotiated. The device reports the hash value and the hash type with the packet. +\item The feature VIRTIO_NET_F_HASH_REPORT was negotiated. The device reports the hash value and the hash type. If additionally +VIRTIO_NET_F_HASH_TUNNEL was negotiated, the device reports the encapsulation type as well. \end{itemize} If the feature VIRTIO_NET_F_RSS was negotiated:@@ -3863,8 +3870,36 @@ \subsubsection{Processing of Incoming Packets}\label{sec:Device Types / Network \ref{sec:Device Types / Network Device / Device Operation / Control Virtqueue / Automatic receive steering in multiqueue mode / Hash calculation}. \end{itemize} +\subparagraph{Tunnel/Encapsulated packet} +\label{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / Tunnel/Encapsulated packet} +A tunnel packet is encapsulated from the original packet based on the tunneling +protocol (only a single level of encapsulation is currently supported). The +encapsulated packet contains an outer header and an inner header, and the device +calculates the hash over either the inner header or the outer header. + +When the feature VIRTIO_NET_F_HASH_TUNNEL is negotiated and the corresponding +encapsulation type is set in \field{hash_types}, the hash for a specific type of +encapsulated packet is calculated over the inner as opposed to outer header. +Supported encapsulation types are listed in \ref{sec:Device Types / Network Device / +Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / +Supported/enabled hash types}. + +If both VIRTIO_NET_F_HASH_REPORT and VIRTIO_NET_F_HASH_TUNNEL are negotiated, +the device can support inner hash calculation for \hyperref[intro:GRE]{[GRE]}, +\hyperref[intro:VXLAN]{[VXLAN]} and \hyperref[intro:GENEVE]{[GENEVE]} +encapsulated packets, and the corresponding encapsulation type in \field{hash_types} +is VIRTIO_NET_HASH_TYPE_{GRE, VXLAN, GENEVE}_INNER respectively. The value in +\field{hash_report_tunnel} is VIRTIO_NET_HASH_REPORT_{NONE, GRE, VXLAN, GENEVE} respectively. + +If VIRTIO_NET_F_HASH_REPORT is negotiated but VIRTIO_NET_F_HASH_TUNNEL is not +negotiated, the device calculates the hash over the outer header, and +\field{hash_report} reports the hash type. \field{hash_report_tunnel} +is no longer valid. + \subparagraph{Supported/enabled hash types} \label{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / Supported/enabled hash types} +This paragraph relies on definitions from \hyperref[intro:IP]{[IP]}, +\hyperref[intro:UDP]{[UDP]} and \hyperref[intro:TCP]{[TCP]}. Hash types applicable for IPv4 packets: \begin{lstlisting} #define VIRTIO_NET_HASH_TYPE_IPv4 (1 << 0)@@ -3884,6 +3919,22 @@ \subsubsection{Processing of Incoming Packets}\label{sec:Device Types / Network #define VIRTIO_NET_HASH_TYPE_UDP_EX (1 << 8) \end{lstlisting} +If the feature VIRTIO_NET_F_HASH_TUNNEL is negotiated, the encapsulation +hash type below indicates that the hash is calculated over the inner +header of the encapsulated packet: +Hash type applicable for inner payload of the gre-encapsulated packet +\begin{lstlisting} +#define VIRTIO_NET_HASH_TYPE_GRE_INNER (1 << 9) +\end{lstlisting} +Hash type applicable for inner payload of the vxlan-encapsulated packet +\begin{lstlisting} +#define VIRTIO_NET_HASH_TYPE_VXLAN_INNER (1 << 10) +\end{lstlisting} +Hash type applicable for inner payload of the geneve-encapsulated packet +\begin{lstlisting} +#define VIRTIO_NET_HASH_TYPE_GENEVE_INNER (1 << 11) +\end{lstlisting} + \subparagraph{IPv4 packets} \label{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / IPv4 packets} The device calculates the hash on IPv4 packets according to 'Enabled hash types' bitmask as follows:@@ -3975,15 +4026,125 @@ \subsubsection{Processing of Incoming Packets}\label{sec:Device Types / Network (see \ref{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / IPv6 packets without extension header}). \end{itemize} +\subparagraph{Inner payload of an encapsulated packet} +\label{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / Inner payload of the encapsulated packet} +If the feature VIRTIO_NET_F_HASH_TUNNEL is negotiated and the corresponding +encapsulation hash type is set in \field{hash_types}, the device calculates the +inner header hash on an encapsulated packet (See \ref{sec:Device Types +/ Network Device / Device Operation / Processing of Incoming Packets / +Hash calculation for incoming packets / Tunnel/Encapsulated packet}) as follows: + +The device calculates the hash on the inner IPv4 packet of an encapsulated packet +according to 'Enabled hash types' bitmask as follows: +\begin{itemize} + \item If VIRTIO_NET_HASH_TYPE_TCPv4 is set and the encapsulated packet has an inner + TCPv4 header, the hash is calculated over the following fields: + \begin{itemsize} + \item inner Source IP address + \item inner Destination IP address + \item inner Source TCP port + \item inner Destination TCP port + \end{itemsize} + \item Else if VIRTIO_NET_HASH_TYPE_UDPv4 is set and the encapsulated packet has an + inner UDPv4 header, the hash is calculated over the following fields: + \begin{itemsize} + \item inner Source IP address + \item inner Destination IP address + \item inner Source UDP port + \item inner Destination UDP port + \end{itemize} + \item Else if VIRTIO_NET_HASH_TYPE_IPv4 is set, the hash is calculated over the + following fields: + \begin{itemsize} + \item inner Source IP address + \item inner Destination IP address + \end{itemsize} + \item Else the device does not calculate the hash +\end{itemize} + +The device calculates the hash on the inner IPv6 packet without an extension header +of an encapsulated packet according to 'Enabled hash types' bitmask as follows: +\begin{itemize} + \item If VIRTIO_NET_HASH_TYPE_TCPv6 is set and the encapsulated packet has an inner + TCPv6 header, the hash is calculated over the following fields: + \begin{itemsize} + \item inner Source IPv6 address + \item inner Destination IPv6 address + \item inner Source TCP port + \item inner Destination TCP port + \end{itemsize} + \item Else if VIRTIO_NET_HASH_TYPE_UDPv6 is set and the encapsulated packet has an + inner UDPv6 header, the hash is calculated over the following fields: + \begin{itemsize} + \item inner Source IPv6 address + \item inner Destination IPv6 address + \item inner Source UDP port + \item inner Destination UDP port + \end{itemize} + \item Else if VIRTIO_NET_HASH_TYPE_IPv6 is set, the hash is calculated over the + following fields: + \begin{itemsize} + \item inner Source IPv6 address + \item inner Destination IPv6 address + \end{itemsize} + \item Else the device does not calculate the hash +\end{itemize} + +The device calculates the hash on the inner IPv6 packet with an extension header +of an encapsulated packet according to 'Enabled hash types' bitmask as follows: +\begin{itemsize} + \item If VIRTIO_NET_HASH_TYPE_TCP_EX is set and the encapsulated packet has an inner + TCPv6 header, the hash is calculated over the following fields: + \begin{itemize} + \item Home address from the home address option in the inner IPv6 destination + options header. If the inner extension header is not present, use the + inner Source IPv6 address. + \item IPv6 address that is contained in the Routing-Header-Type-2 from the + associated inner extension header. If the inner extension header is not + present, use the inner Destination IPv6 address. + \item inner Source TCP port + \item inner Destination TCP port + \end{itemize} + \item Else if VIRTIO_NET_HASH_TYPE_UDP_EX is set and the encapsulated packet has an inner + UDPv6 header, the hash is calculated over the following fields: + \begin{itemsize} + \item Home address from the home address option in the inner IPv6 destination + options header. If the inner extension header is not present, use the + inner Source IPv6 address. + \item IPv6 address that is contained in the Routing-Header-Type-2 from the + associated inner extension header. If the inner extension header is not + present, use the inner Destination IPv6 address. + \item inner Source UDP port + \item inner Destination UDP port + \end{itemize} + \item Else if VIRTIO_NET_HASH_TYPE_IP_EX is set, the hash is calculated over the + following fields: + \begin{itemsize} + \item Home address from the home address option in the inner IPv6 destination + options header. If the inner extension header is not present, use the + inner Source IPv6 address. + \item IPv6 address that is contained in the Routing-Header-Type-2 from the + associated inner extension header. If the inner extension header is not + present, use the inner Destination IPv6 address. + \end{itemize} + \item Else skip the inner IPv6 extension header and calculate the inner header hash as + defined for an encapsulated packet whose inner payload is an IPv6 packet without + an extension header. +\end{itemsize} + \paragraph{Hash reporting for incoming packets} \label{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash reporting for incoming packets} -If VIRTIO_NET_F_HASH_REPORT was negotiated and - the device has calculated the hash for the packet, the device fills \field{hash_report} with the report type of calculated hash -and \field{hash_value} with the value of calculated hash. +If VIRTIO_NET_F_HASH_REPORT was negotiated and the device has calculated the +hash for the packet, the device fills \field{hash_report} with the report type +of calculated hash, and \field{hash_value} with the value of calculated hash. +Also, if VIRTIO_NET_F_HASH_TUNNEL was negotiated, the device needs to fill +\field{hash_report_tunnel} with the report type of the encapsulated packet, and +it is set to VIRTIO_NET_HASH_REPORT_TUNNEL_NONE for the unencapsulated packet. If VIRTIO_NET_F_HASH_REPORT was negotiated but due to any reason the -hash was not calculated, the device sets \field{hash_report} to VIRTIO_NET_HASH_REPORT_NONE. +hash was not calculated, the device sets \field{hash_report} to VIRTIO_NET_HASH_REPORT_NONE, +and sets \field{hash_report_tunnel} to VIRTIO_NET_HASH_REPORT_TUNNEL_NONE. Possible values that the device can report in \field{hash_report} are defined below. They correspond to supported hash types defined in@@ -4005,6 +4166,22 @@ \subsubsection{Processing of Incoming Packets}\label{sec:Device Types / Network #define VIRTIO_NET_HASH_REPORT_UDPv6_EX 9 \end{lstlisting}
Quoted 7 lines
content.tex \subsubsection{Processing of Incoming Packets}\label{sec:Device Types / Network · line 4171
Quoted 3 lines
content.tex +3
-0+If \field{hash_report} differs from VIRTIO_NET_HASH_REPORT_NONE, +\field{hash_report_tunnel} can report the type of the encapsulated +packet to the driver over the inner header hash calculation.this sentence is a bit confused btw. so this reports the type of the encapsulated packet. but what does "over the inner header hash calculation." mean? what happens over what?
Just means "After the hash is computed, the encapsulation type can be reported to the driver". It means redundant here, I will remove "over the inner header hash calculation".
Thanks.
Quoted 50 lines
Quoted 50 lines
content.tex +13
-0+Possible values that the device can report in \field{hash_report_tunnel} +are defined below: + +\begin{lstlisting} +#define VIRTIO_NET_HASH_REPORT_TUNNEL_NONE 0 +#define VIRTIO_NET_HASH_REPORT_GRE 1 +#define VIRTIO_NET_HASH_REPORT_VXLAN 2 +#define VIRTIO_NET_HASH_REPORT_GENEVE 3 +\end{lstlisting} + +They correspond to supported hash types defined in +\ref{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / Supported/enabled hash types}. + \subsubsection{Control Virtqueue}\label{sec:Device Types / Network Device / Device Operation / Control Virtqueue} The driver uses the control virtqueue (if VIRTIO_NET_F_CTRL_VQ isintroduction.tex +19
-0@@ -98,6 +98,25 @@ \section{Normative References}\label{sec:Normative References} \phantomsection\label{intro:SEC1}\textbf{[SEC1]} & Standards for Efficient Cryptography Group(SECG), ``SEC1: Elliptic Cureve Cryptography'', Version 1.0, September 2000. \newline\url{https://www.secg.org/sec1-v2.pdf}\\ + \phantomsection\label{intro:GRE}\textbf{[GRE]} & + Generic Routing Encapsulation + \newline\url{https://datatracker.ietf.org/doc/rfc2784/}\\ + \phantomsection\label{intro:VXLAN}\textbf{[VXLAN]} & + Virtual eXtensible Local Area Network + \newline\url{https://datatracker.ietf.org/doc/rfc7348/}\\ + \phantomsection\label{intro:GENEVE}\textbf{[GENEVE]} & + Generic Network Virtualization Encapsulation + \newline\url{https://datatracker.ietf.org/doc/rfc8926/}\\ + \phantomsection\label{intro:IP}\textbf{[IP]} & + INTERNET PROTOCOL + \newline\url{https://www.rfc-editor.org/rfc/rfc791}\\ + \phantomsection\label{intro:UDP}\textbf{[UDP]} & + User Datagram Protocol + \newline\url{https://www.rfc-editor.org/rfc/rfc768}\\ + \phantomsection\label{intro:TCP}\textbf{[TCP]} & + TRANSMISSION CONTROL PROTOCOL + \newline\url{https://www.rfc-editor.org/rfc/rfc793}\\ + \end{longtable}Signature · 1 line
2.19.1.6.gb485710b
content.tex \subsubsection{Processing of Incoming Packets}\label{sec:Device Types / Network · line 4180
Quoted 8 lines · Heng Qi
content.tex +8
-0+are defined below: + +\begin{lstlisting} +#define VIRTIO_NET_HASH_REPORT_TUNNEL_NONE 0 +#define VIRTIO_NET_HASH_REPORT_GRE 1 +#define VIRTIO_NET_HASH_REPORT_VXLAN 2 +#define VIRTIO_NET_HASH_REPORT_GENEVE 3 +\end{lstlisting}
Btw this "are defined below" all over the place is just contributing to making the spec unnecesarily verbose. Simple "are:" will do.
Signature · 1 line
MST
Quoted 2 lines · Michael S. Tsirkin
Btw this "are defined below" all over the place is just contributing to making the spec unnecesarily verbose. Simple "are:" will do.
Sure. I'll fix it in the next version.
Thanks.
Quoted 81 lines · Heng Qi
If the tunnel is used to encapsulate the packets, the hash calculated using the outer header of the receive packets is always fixed for the same flow packets, i.e. they will be steered to the same receive queue.
We add a feature bit
VIRTIO_NET_F_HASH_TUNNELand related bitmasks in \field{hash_types}, which instructs the device to calculate the hash using the inner headers of tunnel-encapsulated packets. Besides, values in \field{hash_report_tunnel} are added to report tunnel types.Reviewed-by Jason Wang <[email protected]>Signed-off-by Heng Qi <[email protected]>Signed-off-by Xuan Zhuo <[email protected]>---
v6:
1. Modify the wording of some sentences for clarity. @Michael S. Tsirkin
2. Fix some syntax issues. @Michael S. Tsirkinv5:
1. Fix some syntax and capitalization issues. @Michael S. Tsirkin
2. Use encapsulated/encaptulation uniformly. @Michael S. Tsirkin
3. Move the links to introduction section. @Michael S. Tsirkin
4. Clarify some sentences. @Michael S. Tsirkinv4:
1. Clarify some paragraphs. @Cornelia Huck
2. Fix the u8 type. @Cornelia Huckv3:
1. RenameVIRTIO_NET_F_HASH_GRE_VXLAN_GENEVE_INNERtoVIRTIO_NET_F_HASH_TUNNEL. @Jason Wang
2. Make things clearer. @Jason Wang @Michael S. Tsirkin
3. Keep the possibility to use inner hash for automatic receive steering. @Jason Wang
4. Add the "Tunnel packet" paragraph to avoid repeating the GRE etc. many times. @Michael S. Tsirkinv2:
1. Add a feature bit for GRE/VXLAN/GENEVE inner hash. @Jason Wang
2. Chang \field{hash_tunnel} to \field{hash_report_tunnel}. @Jason Wang, @Michael S. Tsirkinv1:
1. Remove the patch for the bitmask fix. @Michael S. Tsirkin
2. Clarify some paragraphs. @Jason Wang
3. Add \field{hash_tunnel} andVIRTIO_NET_HASH_REPORT_GRE. @Yuri Benditovichcontent.tex | 191 +++++++++++++++++++++++++++++++++++++++++++++-- introduction.tex | 19 +++++ 2 files changed, 203 insertions(+), 7 deletions(-)content.tex +6
-1@@ -3084,6 +3084,9 @@ \subsection{Feature bits}\label{sec:Device Types / Network Device / Feature bits \item[VIRTIO_NET_F_CTRL_MAC_ADDR(23)] Set MAC address through control channel. +\item[VIRTIO_NET_F_HASH_TUNNEL(52)] Device supports inner + header hash for GRE, VXLAN and GENEVE tunnel-encapsulated packets. + \item[VIRTIO_NET_F_NOTF_COAL(53)] Device supports notifications coalescing. \item[VIRTIO_NET_F_GUEST_USO4 (54)] Driver can receive USOv4 packets.@@ -3095,7 +3098,8 @@ \subsection{Feature bits}\label{sec:Device Types / Network Device / Feature bits to several segments when each of these smaller packets has UDP header. \item[VIRTIO_NET_F_HASH_REPORT(57)] Device can report per-packet hash - value and a type of calculated hash. + value, a type of calculated hash, and, if VIRTIO_NET_F_HASH_TUNNEL + is negotiated, an encapsulation packet type. \item[VIRTIO_NET_F_GUEST_HDRLEN(59)] Driver can provide the exact \field{hdr_len} value. Device benefits from knowing the exact header length.@@ -3140,6 +3144,7 @@ \subsubsection{Feature bit requirements}\label{sec:Device Types / Network Device \item[VIRTIO_NET_F_NOTF_COAL] Requires VIRTIO_NET_F_CTRL_VQ. \item[VIRTIO_NET_F_RSC_EXT] Requires VIRTIO_NET_F_HOST_TSO4 or VIRTIO_NET_F_HOST_TSO6. \item[VIRTIO_NET_F_RSS] Requires VIRTIO_NET_F_CTRL_VQ. +\item[VIRTIO_NET_F_HASH_TUNNEL] Requires VIRTIO_NET_F_CTRL_VQ. \end{description} \subsubsection{Legacy Interface: Feature bits}\label{sec:Device Types / Network Device / Feature bits / Legacy Interface: Feature bits}
content.tex \subsection{Device Operation}\label{sec:Device Types / Network Device / Device O · line 3394
Quoted 5 lines
content.tex +1
-1@@ -3386,7 +3391,8 @@ \subsection{Device Operation}\label{sec:Device Types / Network Device / Device O le16 num_buffers; le32 hash_value; (Only if VIRTIO_NET_F_HASH_REPORT negotiated) le16 hash_report; (Only if VIRTIO_NET_F_HASH_REPORT negotiated) - le16 padding_reserved; (Only if VIRTIO_NET_F_HASH_REPORT negotiated) + u8 hash_report_tunnel; (Only if VIRTIO_NET_F_HASH_REPORT negotiated, only valid of VIRTIO_NET_F_HASH_TUNNEL negotiated)
of -> if?
also let's add: otherwise reserved?
Quoted 35 lines
content.tex +18
-1+ u8 padding_reserved; (Only if VIRTIO_NET_F_HASH_REPORT negotiated) }; \end{lstlisting}@@ -3837,7 +3843,8 @@ \subsubsection{Processing of Incoming Packets}\label{sec:Device Types / Network A device attempts to calculate a per-packet hash in the following cases: \begin{itemize} \item The feature VIRTIO_NET_F_RSS was negotiated. The device uses the hash to determine the receive virtqueue to place incoming packets. -\item The feature VIRTIO_NET_F_HASH_REPORT was negotiated. The device reports the hash value and the hash type with the packet. +\item The feature VIRTIO_NET_F_HASH_REPORT was negotiated. The device reports the hash value and the hash type. If additionally +VIRTIO_NET_F_HASH_TUNNEL was negotiated, the device reports the encapsulation type as well. \end{itemize} If the feature VIRTIO_NET_F_RSS was negotiated:@@ -3863,8 +3870,36 @@ \subsubsection{Processing of Incoming Packets}\label{sec:Device Types / Network \ref{sec:Device Types / Network Device / Device Operation / Control Virtqueue / Automatic receive steering in multiqueue mode / Hash calculation}. \end{itemize} +\subparagraph{Tunnel/Encapsulated packet} +\label{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / Tunnel/Encapsulated packet} +A tunnel packet is encapsulated from the original packet based on the tunneling +protocol (only a single level of encapsulation is currently supported). The +encapsulated packet contains an outer header and an inner header, and the device +calculates the hash over either the inner header or the outer header. + +When the feature VIRTIO_NET_F_HASH_TUNNEL is negotiated and the corresponding +encapsulation type is set in \field{hash_types}, the hash for a specific type of +encapsulated packet is calculated over the inner as opposed to outer header. +Supported encapsulation types are listed in \ref{sec:Device Types / Network Device / +Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / +Supported/enabled hash types}. + +If both VIRTIO_NET_F_HASH_REPORT and VIRTIO_NET_F_HASH_TUNNEL are negotiated,
content.tex \subsubsection{Processing of Incoming Packets}\label{sec:Device Types / Network · line 3899
Quoted 10 lines
content.tex +10
-0+the device can support inner hash calculation for \hyperref[intro:GRE]{[GRE]}, +\hyperref[intro:VXLAN]{[VXLAN]} and \hyperref[intro:GENEVE]{[GENEVE]} +encapsulated packets, and the corresponding encapsulation type in \field{hash_types} +is VIRTIO_NET_HASH_TYPE_{GRE, VXLAN, GENEVE}_INNER respectively. The value in +\field{hash_report_tunnel} is VIRTIO_NET_HASH_REPORT_{NONE, GRE, VXLAN, GENEVE} respectively. + +If VIRTIO_NET_F_HASH_REPORT is negotiated but VIRTIO_NET_F_HASH_TUNNEL is not +negotiated, the device calculates the hash over the outer header, and +\field{hash_report} reports the hash type. \field{hash_report_tunnel} +is no longer valid.
Does this mean that VIRTIO_NET_HASH_TYPE_{GRE, VXLAN, GENEVE}_INNER must not be set without VIRTIO_NET_F_HASH_TUNNEL?
If yes then we can add conformance statements to this end and then drop the
"If the feature VIRTIO_NET_F_HASH_TUNNEL is negotiated" all over the place just relying on hash type being set instead.
Also, note that it has to be ok for device to expose hash types without negotiating VIRTIO_NET_F_HASH_TUNNEL since they are in config space.
Idea: do we want to decouple these completely? have VIRTIO_NET_F_HASH_TUNNEL just add hash_report_tunnel and have hash types speak for themselves? It makes things nicely orthogonal in that if one does not care about knowing encapsulation type (e.g. for RSS) one can disable VIRTIO_NET_F_HASH_TUNNEL.
Objections? Jason?
Quoted 216 lines
content.tex +3
-0+ \subparagraph{Supported/enabled hash types} \label{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / Supported/enabled hash types} +This paragraph relies on definitions from \hyperref[intro:IP]{[IP]}, +\hyperref[intro:UDP]{[UDP]} and \hyperref[intro:TCP]{[TCP]}. Hash types applicable for IPv4 packets:\begin{lstlisting}
#defineVIRTIO_NET_HASH_TYPE_IPv4(1 << 0)content.tex +146
-4@@ -3884,6 +3919,22 @@ \subsubsection{Processing of Incoming Packets}\label{sec:Device Types / Network #define VIRTIO_NET_HASH_TYPE_UDP_EX (1 << 8) \end{lstlisting} +If the feature VIRTIO_NET_F_HASH_TUNNEL is negotiated, the encapsulation +hash type below indicates that the hash is calculated over the inner +header of the encapsulated packet: +Hash type applicable for inner payload of the gre-encapsulated packet +\begin{lstlisting} +#define VIRTIO_NET_HASH_TYPE_GRE_INNER (1 << 9) +\end{lstlisting} +Hash type applicable for inner payload of the vxlan-encapsulated packet +\begin{lstlisting} +#define VIRTIO_NET_HASH_TYPE_VXLAN_INNER (1 << 10) +\end{lstlisting} +Hash type applicable for inner payload of the geneve-encapsulated packet +\begin{lstlisting} +#define VIRTIO_NET_HASH_TYPE_GENEVE_INNER (1 << 11) +\end{lstlisting} + \subparagraph{IPv4 packets} \label{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / IPv4 packets} The device calculates the hash on IPv4 packets according to 'Enabled hash types' bitmask as follows:@@ -3975,15 +4026,125 @@ \subsubsection{Processing of Incoming Packets}\label{sec:Device Types / Network (see \ref{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / IPv6 packets without extension header}). \end{itemize} +\subparagraph{Inner payload of an encapsulated packet} +\label{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / Inner payload of the encapsulated packet} +If the feature VIRTIO_NET_F_HASH_TUNNEL is negotiated and the corresponding +encapsulation hash type is set in \field{hash_types}, the device calculates the +inner header hash on an encapsulated packet (See \ref{sec:Device Types +/ Network Device / Device Operation / Processing of Incoming Packets / +Hash calculation for incoming packets / Tunnel/Encapsulated packet}) as follows: + +The device calculates the hash on the inner IPv4 packet of an encapsulated packet +according to 'Enabled hash types' bitmask as follows: +\begin{itemize} + \item If VIRTIO_NET_HASH_TYPE_TCPv4 is set and the encapsulated packet has an inner + TCPv4 header, the hash is calculated over the following fields: + \begin{itemsize} + \item inner Source IP address + \item inner Destination IP address + \item inner Source TCP port + \item inner Destination TCP port + \end{itemsize} + \item Else if VIRTIO_NET_HASH_TYPE_UDPv4 is set and the encapsulated packet has an + inner UDPv4 header, the hash is calculated over the following fields: + \begin{itemsize} + \item inner Source IP address + \item inner Destination IP address + \item inner Source UDP port + \item inner Destination UDP port + \end{itemize} + \item Else if VIRTIO_NET_HASH_TYPE_IPv4 is set, the hash is calculated over the + following fields: + \begin{itemsize} + \item inner Source IP address + \item inner Destination IP address + \end{itemsize} + \item Else the device does not calculate the hash +\end{itemize} + +The device calculates the hash on the inner IPv6 packet without an extension header +of an encapsulated packet according to 'Enabled hash types' bitmask as follows: +\begin{itemize} + \item If VIRTIO_NET_HASH_TYPE_TCPv6 is set and the encapsulated packet has an inner + TCPv6 header, the hash is calculated over the following fields: + \begin{itemsize} + \item inner Source IPv6 address + \item inner Destination IPv6 address + \item inner Source TCP port + \item inner Destination TCP port + \end{itemsize} + \item Else if VIRTIO_NET_HASH_TYPE_UDPv6 is set and the encapsulated packet has an + inner UDPv6 header, the hash is calculated over the following fields: + \begin{itemsize} + \item inner Source IPv6 address + \item inner Destination IPv6 address + \item inner Source UDP port + \item inner Destination UDP port + \end{itemize} + \item Else if VIRTIO_NET_HASH_TYPE_IPv6 is set, the hash is calculated over the + following fields: + \begin{itemsize} + \item inner Source IPv6 address + \item inner Destination IPv6 address + \end{itemsize} + \item Else the device does not calculate the hash +\end{itemize} + +The device calculates the hash on the inner IPv6 packet with an extension header +of an encapsulated packet according to 'Enabled hash types' bitmask as follows: +\begin{itemsize} + \item If VIRTIO_NET_HASH_TYPE_TCP_EX is set and the encapsulated packet has an inner + TCPv6 header, the hash is calculated over the following fields: + \begin{itemize} + \item Home address from the home address option in the inner IPv6 destination + options header. If the inner extension header is not present, use the + inner Source IPv6 address. + \item IPv6 address that is contained in the Routing-Header-Type-2 from the + associated inner extension header. If the inner extension header is not + present, use the inner Destination IPv6 address. + \item inner Source TCP port + \item inner Destination TCP port + \end{itemize} + \item Else if VIRTIO_NET_HASH_TYPE_UDP_EX is set and the encapsulated packet has an inner + UDPv6 header, the hash is calculated over the following fields: + \begin{itemsize} + \item Home address from the home address option in the inner IPv6 destination + options header. If the inner extension header is not present, use the + inner Source IPv6 address. + \item IPv6 address that is contained in the Routing-Header-Type-2 from the + associated inner extension header. If the inner extension header is not + present, use the inner Destination IPv6 address. + \item inner Source UDP port + \item inner Destination UDP port + \end{itemize} + \item Else if VIRTIO_NET_HASH_TYPE_IP_EX is set, the hash is calculated over the + following fields: + \begin{itemsize} + \item Home address from the home address option in the inner IPv6 destination + options header. If the inner extension header is not present, use the + inner Source IPv6 address. + \item IPv6 address that is contained in the Routing-Header-Type-2 from the + associated inner extension header. If the inner extension header is not + present, use the inner Destination IPv6 address. + \end{itemize} + \item Else skip the inner IPv6 extension header and calculate the inner header hash as + defined for an encapsulated packet whose inner payload is an IPv6 packet without + an extension header. +\end{itemsize} + \paragraph{Hash reporting for incoming packets} \label{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash reporting for incoming packets} -If VIRTIO_NET_F_HASH_REPORT was negotiated and - the device has calculated the hash for the packet, the device fills \field{hash_report} with the report type of calculated hash -and \field{hash_value} with the value of calculated hash. +If VIRTIO_NET_F_HASH_REPORT was negotiated and the device has calculated the +hash for the packet, the device fills \field{hash_report} with the report type +of calculated hash, and \field{hash_value} with the value of calculated hash. +Also, if VIRTIO_NET_F_HASH_TUNNEL was negotiated, the device needs to fill +\field{hash_report_tunnel} with the report type of the encapsulated packet, and +it is set to VIRTIO_NET_HASH_REPORT_TUNNEL_NONE for the unencapsulated packet. If VIRTIO_NET_F_HASH_REPORT was negotiated but due to any reason the -hash was not calculated, the device sets \field{hash_report} to VIRTIO_NET_HASH_REPORT_NONE. +hash was not calculated, the device sets \field{hash_report} to VIRTIO_NET_HASH_REPORT_NONE, +and sets \field{hash_report_tunnel} to VIRTIO_NET_HASH_REPORT_TUNNEL_NONE. Possible values that the device can report in \field{hash_report} are defined below. They correspond to supported hash types defined in@@ -4005,6 +4166,22 @@ \subsubsection{Processing of Incoming Packets}\label{sec:Device Types / Network #define VIRTIO_NET_HASH_REPORT_UDPv6_EX 9 \end{lstlisting} +If \field{hash_report} differs from VIRTIO_NET_HASH_REPORT_NONE, +\field{hash_report_tunnel} can report the type of the encapsulated +packet to the driver over the inner header hash calculation. +Possible values that the device can report in \field{hash_report_tunnel} +are defined below: + +\begin{lstlisting} +#define VIRTIO_NET_HASH_REPORT_TUNNEL_NONE 0 +#define VIRTIO_NET_HASH_REPORT_GRE 1 +#define VIRTIO_NET_HASH_REPORT_VXLAN 2 +#define VIRTIO_NET_HASH_REPORT_GENEVE 3 +\end{lstlisting} + +They correspond to supported hash types defined in +\ref{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / Supported/enabled hash types}. + \subsubsection{Control Virtqueue}\label{sec:Device Types / Network Device / Device Operation / Control Virtqueue} The driver uses the control virtqueue (if VIRTIO_NET_F_CTRL_VQ isintroduction.tex +19
-0@@ -98,6 +98,25 @@ \section{Normative References}\label{sec:Normative References} \phantomsection\label{intro:SEC1}\textbf{[SEC1]} & Standards for Efficient Cryptography Group(SECG), ``SEC1: Elliptic Cureve Cryptography'', Version 1.0, September 2000. \newline\url{https://www.secg.org/sec1-v2.pdf}\\ + \phantomsection\label{intro:GRE}\textbf{[GRE]} & + Generic Routing Encapsulation + \newline\url{https://datatracker.ietf.org/doc/rfc2784/}\\ + \phantomsection\label{intro:VXLAN}\textbf{[VXLAN]} & + Virtual eXtensible Local Area Network + \newline\url{https://datatracker.ietf.org/doc/rfc7348/}\\ + \phantomsection\label{intro:GENEVE}\textbf{[GENEVE]} & + Generic Network Virtualization Encapsulation + \newline\url{https://datatracker.ietf.org/doc/rfc8926/}\\ + \phantomsection\label{intro:IP}\textbf{[IP]} & + INTERNET PROTOCOL + \newline\url{https://www.rfc-editor.org/rfc/rfc791}\\ + \phantomsection\label{intro:UDP}\textbf{[UDP]} & + User Datagram Protocol + \newline\url{https://www.rfc-editor.org/rfc/rfc768}\\ + \phantomsection\label{intro:TCP}\textbf{[TCP]} & + TRANSMISSION CONTROL PROTOCOL + \newline\url{https://www.rfc-editor.org/rfc/rfc793}\\ + \end{longtable}Signature · 1 line
2.19.1.6.gb485710b
Quoted 80 lines · Michael S. Tsirkin
Quoted 79 lines · Heng Qi
If the tunnel is used to encapsulate the packets, the hash calculated using the outer header of the receive packets is always fixed for the same flow packets, i.e. they will be steered to the same receive queue.
We add a feature bit
VIRTIO_NET_F_HASH_TUNNELand related bitmasks in \field{hash_types}, which instructs the device to calculate the hash using the inner headers of tunnel-encapsulated packets. Besides, values in \field{hash_report_tunnel} are added to report tunnel types.Reviewed-by Jason Wang <[email protected]>Signed-off-by Heng Qi <[email protected]>Signed-off-by Xuan Zhuo <[email protected]>---
v6:
1. Modify the wording of some sentences for clarity. @Michael S. Tsirkin
2. Fix some syntax issues. @Michael S. Tsirkinv5:
1. Fix some syntax and capitalization issues. @Michael S. Tsirkin
2. Use encapsulated/encaptulation uniformly. @Michael S. Tsirkin
3. Move the links to introduction section. @Michael S. Tsirkin
4. Clarify some sentences. @Michael S. Tsirkinv4:
1. Clarify some paragraphs. @Cornelia Huck
2. Fix the u8 type. @Cornelia Huckv3:
1. RenameVIRTIO_NET_F_HASH_GRE_VXLAN_GENEVE_INNERtoVIRTIO_NET_F_HASH_TUNNEL. @Jason Wang
2. Make things clearer. @Jason Wang @Michael S. Tsirkin
3. Keep the possibility to use inner hash for automatic receive steering. @Jason Wang
4. Add the "Tunnel packet" paragraph to avoid repeating the GRE etc. many times. @Michael S. Tsirkinv2:
1. Add a feature bit for GRE/VXLAN/GENEVE inner hash. @Jason Wang
2. Chang \field{hash_tunnel} to \field{hash_report_tunnel}. @Jason Wang, @Michael S. Tsirkinv1:
1. Remove the patch for the bitmask fix. @Michael S. Tsirkin
2. Clarify some paragraphs. @Jason Wang
3. Add \field{hash_tunnel} andVIRTIO_NET_HASH_REPORT_GRE. @Yuri Benditovichcontent.tex | 191 +++++++++++++++++++++++++++++++++++++++++++++-- introduction.tex | 19 +++++ 2 files changed, 203 insertions(+), 7 deletions(-)content.tex +6
-1@@ -3084,6 +3084,9 @@ \subsection{Feature bits}\label{sec:Device Types / Network Device / Feature bits \item[VIRTIO_NET_F_CTRL_MAC_ADDR(23)] Set MAC address through control channel. +\item[VIRTIO_NET_F_HASH_TUNNEL(52)] Device supports inner + header hash for GRE, VXLAN and GENEVE tunnel-encapsulated packets. + \item[VIRTIO_NET_F_NOTF_COAL(53)] Device supports notifications coalescing. \item[VIRTIO_NET_F_GUEST_USO4 (54)] Driver can receive USOv4 packets.@@ -3095,7 +3098,8 @@ \subsection{Feature bits}\label{sec:Device Types / Network Device / Feature bits to several segments when each of these smaller packets has UDP header. \item[VIRTIO_NET_F_HASH_REPORT(57)] Device can report per-packet hash - value and a type of calculated hash. + value, a type of calculated hash, and, if VIRTIO_NET_F_HASH_TUNNEL + is negotiated, an encapsulation packet type. \item[VIRTIO_NET_F_GUEST_HDRLEN(59)] Driver can provide the exact \field{hdr_len} value. Device benefits from knowing the exact header length.@@ -3140,6 +3144,7 @@ \subsubsection{Feature bit requirements}\label{sec:Device Types / Network Device \item[VIRTIO_NET_F_NOTF_COAL] Requires VIRTIO_NET_F_CTRL_VQ. \item[VIRTIO_NET_F_RSC_EXT] Requires VIRTIO_NET_F_HOST_TSO4 or VIRTIO_NET_F_HOST_TSO6. \item[VIRTIO_NET_F_RSS] Requires VIRTIO_NET_F_CTRL_VQ. +\item[VIRTIO_NET_F_HASH_TUNNEL] Requires VIRTIO_NET_F_CTRL_VQ. \end{description}
Quoted 9 lines
content.tex \subsection{Device Operation}\label{sec:Device Types / Network Device / Device O · line 3394
Quoted 7 lines
content.tex +1
-1\subsubsection{Legacy Interface: Feature bits}\label{sec:Device Types / Network Device / Feature bits / Legacy Interface: Feature bits}@@ -3386,7 +3391,8 @@ \subsection{Device Operation}\label{sec:Device Types / Network Device / Device O le16 num_buffers; le32 hash_value; (Only if VIRTIO_NET_F_HASH_REPORT negotiated) le16 hash_report; (Only if VIRTIO_NET_F_HASH_REPORT negotiated) - le16 padding_reserved; (Only if VIRTIO_NET_F_HASH_REPORT negotiated) + u8 hash_report_tunnel; (Only if VIRTIO_NET_F_HASH_REPORT negotiated, only valid of VIRTIO_NET_F_HASH_TUNNEL negotiated)of -> if?
Sorry for the late reply.
It's Cornelia's suggestion, and 'of' seems to work fine.
Quoted 2 lines
also let's add: otherwise reserved?
Sure.
Quoted 42 lines
Quoted 42 lines
content.tex +23
-1+ u8 padding_reserved; (Only if VIRTIO_NET_F_HASH_REPORT negotiated) }; \end{lstlisting}@@ -3837,7 +3843,8 @@ \subsubsection{Processing of Incoming Packets}\label{sec:Device Types / Network A device attempts to calculate a per-packet hash in the following cases: \begin{itemize} \item The feature VIRTIO_NET_F_RSS was negotiated. The device uses the hash to determine the receive virtqueue to place incoming packets. -\item The feature VIRTIO_NET_F_HASH_REPORT was negotiated. The device reports the hash value and the hash type with the packet. +\item The feature VIRTIO_NET_F_HASH_REPORT was negotiated. The device reports the hash value and the hash type. If additionally +VIRTIO_NET_F_HASH_TUNNEL was negotiated, the device reports the encapsulation type as well. \end{itemize} If the feature VIRTIO_NET_F_RSS was negotiated:@@ -3863,8 +3870,36 @@ \subsubsection{Processing of Incoming Packets}\label{sec:Device Types / Network \ref{sec:Device Types / Network Device / Device Operation / Control Virtqueue / Automatic receive steering in multiqueue mode / Hash calculation}. \end{itemize} +\subparagraph{Tunnel/Encapsulated packet} +\label{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / Tunnel/Encapsulated packet} +A tunnel packet is encapsulated from the original packet based on the tunneling +protocol (only a single level of encapsulation is currently supported). The +encapsulated packet contains an outer header and an inner header, and the device +calculates the hash over either the inner header or the outer header. + +When the feature VIRTIO_NET_F_HASH_TUNNEL is negotiated and the corresponding +encapsulation type is set in \field{hash_types}, the hash for a specific type of +encapsulated packet is calculated over the inner as opposed to outer header. +Supported encapsulation types are listed in \ref{sec:Device Types / Network Device / +Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / +Supported/enabled hash types}. + +If both VIRTIO_NET_F_HASH_REPORT and VIRTIO_NET_F_HASH_TUNNEL are negotiated, +the device can support inner hash calculation for \hyperref[intro:GRE]{[GRE]}, +\hyperref[intro:VXLAN]{[VXLAN]} and \hyperref[intro:GENEVE]{[GENEVE]} +encapsulated packets, and the corresponding encapsulation type in \field{hash_types} +is VIRTIO_NET_HASH_TYPE_{GRE, VXLAN, GENEVE}_INNER respectively. The value in +\field{hash_report_tunnel} is VIRTIO_NET_HASH_REPORT_{NONE, GRE, VXLAN, GENEVE} respectively.
Quoted 9 lines
content.tex \subsubsection{Processing of Incoming Packets}\label{sec:Device Types / Network · line 3897
Quoted 5 lines
content.tex +5
-0+ +If VIRTIO_NET_F_HASH_REPORT is negotiated but VIRTIO_NET_F_HASH_TUNNEL is not +negotiated, the device calculates the hash over the outer header, and +\field{hash_report} reports the hash type. \field{hash_report_tunnel} +is no longer valid.Does this mean that VIRTIO_NET_HASH_TYPE_{GRE, VXLAN, GENEVE}_INNER must not be set without
VIRTIO_NET_F_HASH_TUNNEL?
Yes.
Quoted 4 lines
If yes then we can add conformance statements to this end and then drop the
"If the featureVIRTIO_NET_F_HASH_TUNNELis negotiated" all over the place just relying on hash type being set instead.
Please see the response below.
Quoted 8 lines
Also, note that it has to be ok for device to expose hash types without negotiating
VIRTIO_NET_F_HASH_TUNNELsince they are in config space.
Idea: do we want to decouple these completely? haveVIRTIO_NET_F_HASH_TUNNELjust addhash_report_tunneland have hash types speak for themselves? It makes things nicely orthogonal in that if one does not care about knowing encapsulation type (e.g. for RSS) one can disableVIRTIO_NET_F_HASH_TUNNEL.
This seems to have the following 3 problems:
1. This will break the specification. As described in the specification, any field
in the configuration space only exists when there is a corresponding feature bit:
"Device configuration space is generally used for rarely-changing or initialization-time parameters. Where configuration fields are optional, their existence is indicated by feature bits: ", which also applies to supported_hash_types.
2. If I'm not wrong, this seems to make the encapsulation hash types dependent on VIRTIO_NET_F_RSS, can the driver read supported_hash_types (including encapsulation hash types) if VIRTIO_NET_F_RSS is not negotiated? But that shouldn't be the case, we only use encapsulation hash types to provide guidance for hash calculation, but not necessarily use hash values to select receive queues (unless VIRTIO_NET_F_RSS is negotiated).
3. If all supported_hash_types speak for themselves, why do we need the VIRTIO_NET_F_HASH_TUNNEL feature, because as long as the encapsulation hash types exist and VIRTIO_NET_F_HASH_REPORT is negotiated, hash_report_tunnel exists correspondingly. "have VIRTIO_NET_F_HASH_TUNNEL just add hash_report_tunnel"? It seems like hash_report_tunnel will only work if the encapsulation hash types are present.
So, if I'm not wrong, the reasonable logic should be that VIRTIO_NET_F_HASH_TUNNEL is only used to declare that the device supports inner header hash and let the corresponding encapsulation hash type exist, when VIRTIO_NET_F_RSS is negotiated, we can use this hash value to select the receive queue. Considering the migration at the same time, if the dst device does not negotiate VIRTIO_NET_F_HASH_TUNNEL, the migration will fail; otherwise, use encapsulation hash types to determine whether the migration can succeed.
Thanks.
Quoted 236 lines
Objections? Jason?
Quoted 216 lines
content.tex +3
-0+ \subparagraph{Supported/enabled hash types} \label{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / Supported/enabled hash types} +This paragraph relies on definitions from \hyperref[intro:IP]{[IP]}, +\hyperref[intro:UDP]{[UDP]} and \hyperref[intro:TCP]{[TCP]}. Hash types applicable for IPv4 packets:\begin{lstlisting}
#defineVIRTIO_NET_HASH_TYPE_IPv4(1 << 0)content.tex +146
-4@@ -3884,6 +3919,22 @@ \subsubsection{Processing of Incoming Packets}\label{sec:Device Types / Network #define VIRTIO_NET_HASH_TYPE_UDP_EX (1 << 8) \end{lstlisting} +If the feature VIRTIO_NET_F_HASH_TUNNEL is negotiated, the encapsulation +hash type below indicates that the hash is calculated over the inner +header of the encapsulated packet: +Hash type applicable for inner payload of the gre-encapsulated packet +\begin{lstlisting} +#define VIRTIO_NET_HASH_TYPE_GRE_INNER (1 << 9) +\end{lstlisting} +Hash type applicable for inner payload of the vxlan-encapsulated packet +\begin{lstlisting} +#define VIRTIO_NET_HASH_TYPE_VXLAN_INNER (1 << 10) +\end{lstlisting} +Hash type applicable for inner payload of the geneve-encapsulated packet +\begin{lstlisting} +#define VIRTIO_NET_HASH_TYPE_GENEVE_INNER (1 << 11) +\end{lstlisting} + \subparagraph{IPv4 packets} \label{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / IPv4 packets} The device calculates the hash on IPv4 packets according to 'Enabled hash types' bitmask as follows:@@ -3975,15 +4026,125 @@ \subsubsection{Processing of Incoming Packets}\label{sec:Device Types / Network (see \ref{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / IPv6 packets without extension header}). \end{itemize} +\subparagraph{Inner payload of an encapsulated packet} +\label{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / Inner payload of the encapsulated packet} +If the feature VIRTIO_NET_F_HASH_TUNNEL is negotiated and the corresponding +encapsulation hash type is set in \field{hash_types}, the device calculates the +inner header hash on an encapsulated packet (See \ref{sec:Device Types +/ Network Device / Device Operation / Processing of Incoming Packets / +Hash calculation for incoming packets / Tunnel/Encapsulated packet}) as follows: + +The device calculates the hash on the inner IPv4 packet of an encapsulated packet +according to 'Enabled hash types' bitmask as follows: +\begin{itemize} + \item If VIRTIO_NET_HASH_TYPE_TCPv4 is set and the encapsulated packet has an inner + TCPv4 header, the hash is calculated over the following fields: + \begin{itemsize} + \item inner Source IP address + \item inner Destination IP address + \item inner Source TCP port + \item inner Destination TCP port + \end{itemsize} + \item Else if VIRTIO_NET_HASH_TYPE_UDPv4 is set and the encapsulated packet has an + inner UDPv4 header, the hash is calculated over the following fields: + \begin{itemsize} + \item inner Source IP address + \item inner Destination IP address + \item inner Source UDP port + \item inner Destination UDP port + \end{itemize} + \item Else if VIRTIO_NET_HASH_TYPE_IPv4 is set, the hash is calculated over the + following fields: + \begin{itemsize} + \item inner Source IP address + \item inner Destination IP address + \end{itemsize} + \item Else the device does not calculate the hash +\end{itemize} + +The device calculates the hash on the inner IPv6 packet without an extension header +of an encapsulated packet according to 'Enabled hash types' bitmask as follows: +\begin{itemize} + \item If VIRTIO_NET_HASH_TYPE_TCPv6 is set and the encapsulated packet has an inner + TCPv6 header, the hash is calculated over the following fields: + \begin{itemsize} + \item inner Source IPv6 address + \item inner Destination IPv6 address + \item inner Source TCP port + \item inner Destination TCP port + \end{itemsize} + \item Else if VIRTIO_NET_HASH_TYPE_UDPv6 is set and the encapsulated packet has an + inner UDPv6 header, the hash is calculated over the following fields: + \begin{itemsize} + \item inner Source IPv6 address + \item inner Destination IPv6 address + \item inner Source UDP port + \item inner Destination UDP port + \end{itemize} + \item Else if VIRTIO_NET_HASH_TYPE_IPv6 is set, the hash is calculated over the + following fields: + \begin{itemsize} + \item inner Source IPv6 address + \item inner Destination IPv6 address + \end{itemsize} + \item Else the device does not calculate the hash +\end{itemize} + +The device calculates the hash on the inner IPv6 packet with an extension header +of an encapsulated packet according to 'Enabled hash types' bitmask as follows: +\begin{itemsize} + \item If VIRTIO_NET_HASH_TYPE_TCP_EX is set and the encapsulated packet has an inner + TCPv6 header, the hash is calculated over the following fields: + \begin{itemize} + \item Home address from the home address option in the inner IPv6 destination + options header. If the inner extension header is not present, use the + inner Source IPv6 address. + \item IPv6 address that is contained in the Routing-Header-Type-2 from the + associated inner extension header. If the inner extension header is not + present, use the inner Destination IPv6 address. + \item inner Source TCP port + \item inner Destination TCP port + \end{itemize} + \item Else if VIRTIO_NET_HASH_TYPE_UDP_EX is set and the encapsulated packet has an inner + UDPv6 header, the hash is calculated over the following fields: + \begin{itemsize} + \item Home address from the home address option in the inner IPv6 destination + options header. If the inner extension header is not present, use the + inner Source IPv6 address. + \item IPv6 address that is contained in the Routing-Header-Type-2 from the + associated inner extension header. If the inner extension header is not + present, use the inner Destination IPv6 address. + \item inner Source UDP port + \item inner Destination UDP port + \end{itemize} + \item Else if VIRTIO_NET_HASH_TYPE_IP_EX is set, the hash is calculated over the + following fields: + \begin{itemsize} + \item Home address from the home address option in the inner IPv6 destination + options header. If the inner extension header is not present, use the + inner Source IPv6 address. + \item IPv6 address that is contained in the Routing-Header-Type-2 from the + associated inner extension header. If the inner extension header is not + present, use the inner Destination IPv6 address. + \end{itemize} + \item Else skip the inner IPv6 extension header and calculate the inner header hash as + defined for an encapsulated packet whose inner payload is an IPv6 packet without + an extension header. +\end{itemsize} + \paragraph{Hash reporting for incoming packets} \label{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash reporting for incoming packets} -If VIRTIO_NET_F_HASH_REPORT was negotiated and - the device has calculated the hash for the packet, the device fills \field{hash_report} with the report type of calculated hash -and \field{hash_value} with the value of calculated hash. +If VIRTIO_NET_F_HASH_REPORT was negotiated and the device has calculated the +hash for the packet, the device fills \field{hash_report} with the report type +of calculated hash, and \field{hash_value} with the value of calculated hash. +Also, if VIRTIO_NET_F_HASH_TUNNEL was negotiated, the device needs to fill +\field{hash_report_tunnel} with the report type of the encapsulated packet, and +it is set to VIRTIO_NET_HASH_REPORT_TUNNEL_NONE for the unencapsulated packet. If VIRTIO_NET_F_HASH_REPORT was negotiated but due to any reason the -hash was not calculated, the device sets \field{hash_report} to VIRTIO_NET_HASH_REPORT_NONE. +hash was not calculated, the device sets \field{hash_report} to VIRTIO_NET_HASH_REPORT_NONE, +and sets \field{hash_report_tunnel} to VIRTIO_NET_HASH_REPORT_TUNNEL_NONE. Possible values that the device can report in \field{hash_report} are defined below. They correspond to supported hash types defined in@@ -4005,6 +4166,22 @@ \subsubsection{Processing of Incoming Packets}\label{sec:Device Types / Network #define VIRTIO_NET_HASH_REPORT_UDPv6_EX 9 \end{lstlisting} +If \field{hash_report} differs from VIRTIO_NET_HASH_REPORT_NONE, +\field{hash_report_tunnel} can report the type of the encapsulated +packet to the driver over the inner header hash calculation. +Possible values that the device can report in \field{hash_report_tunnel} +are defined below: + +\begin{lstlisting} +#define VIRTIO_NET_HASH_REPORT_TUNNEL_NONE 0 +#define VIRTIO_NET_HASH_REPORT_GRE 1 +#define VIRTIO_NET_HASH_REPORT_VXLAN 2 +#define VIRTIO_NET_HASH_REPORT_GENEVE 3 +\end{lstlisting} + +They correspond to supported hash types defined in +\ref{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / Supported/enabled hash types}. + \subsubsection{Control Virtqueue}\label{sec:Device Types / Network Device / Device Operation / Control Virtqueue} The driver uses the control virtqueue (if VIRTIO_NET_F_CTRL_VQ isintroduction.tex +19
-0@@ -98,6 +98,25 @@ \section{Normative References}\label{sec:Normative References} \phantomsection\label{intro:SEC1}\textbf{[SEC1]} & Standards for Efficient Cryptography Group(SECG), ``SEC1: Elliptic Cureve Cryptography'', Version 1.0, September 2000. \newline\url{https://www.secg.org/sec1-v2.pdf}\\ + \phantomsection\label{intro:GRE}\textbf{[GRE]} & + Generic Routing Encapsulation + \newline\url{https://datatracker.ietf.org/doc/rfc2784/}\\ + \phantomsection\label{intro:VXLAN}\textbf{[VXLAN]} & + Virtual eXtensible Local Area Network + \newline\url{https://datatracker.ietf.org/doc/rfc7348/}\\ + \phantomsection\label{intro:GENEVE}\textbf{[GENEVE]} & + Generic Network Virtualization Encapsulation + \newline\url{https://datatracker.ietf.org/doc/rfc8926/}\\ + \phantomsection\label{intro:IP}\textbf{[IP]} & + INTERNET PROTOCOL + \newline\url{https://www.rfc-editor.org/rfc/rfc791}\\ + \phantomsection\label{intro:UDP}\textbf{[UDP]} & + User Datagram Protocol + \newline\url{https://www.rfc-editor.org/rfc/rfc768}\\ + \phantomsection\label{intro:TCP}\textbf{[TCP]} & + TRANSMISSION CONTROL PROTOCOL + \newline\url{https://www.rfc-editor.org/rfc/rfc793}\\ + \end{longtable}Signature · 1 line
2.19.1.6.gb485710bThis publicly archived list offers a means to provide input to the OASIS Virtual I/O Device (VIRTIO) TC.
In order to verify user consent to the Feedback License terms and to minimize spam in the list archive, subscription is required before posting.
Subscribe: [email protected]
Unsubscribe: [email protected]
List help: [email protected]
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/
Quoted 11 lines · Heng Qi
Quoted 9 lines · Michael S. Tsirkin
content.tex \subsection{Device Operation}\label{sec:Device Types / Network Device / Device O · line 3394
Quoted 6 lines · Heng Qi
content.tex +1
-1@@ -3386,7 +3391,8 @@ \subsection{Device Operation}\label{sec:Device Types / Network Device / Device O le16 num_buffers; le32 hash_value; (Only if VIRTIO_NET_F_HASH_REPORT negotiated) le16 hash_report; (Only if VIRTIO_NET_F_HASH_REPORT negotiated) - le16 padding_reserved; (Only if VIRTIO_NET_F_HASH_REPORT negotiated) + u8 hash_report_tunnel; (Only if VIRTIO_NET_F_HASH_REPORT negotiated, only valid of VIRTIO_NET_F_HASH_TUNNEL negotiated)of -> if?
Quoted 2 lines
Sorry for the late reply.
It's Cornelia's suggestion, and 'of' seems to work fine.
I think that was simply a typo, should have been 'if'.
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Quoted 13 lines · Cornelia Huck
Quoted 11 lines · Heng Qi
Quoted 9 lines · Michael S. Tsirkin
content.tex \subsection{Device Operation}\label{sec:Device Types / Network Device / Device O · line 3394
Quoted 6 lines · Heng Qi
content.tex +1
-1@@ -3386,7 +3391,8 @@ \subsection{Device Operation}\label{sec:Device Types / Network Device / Device O le16 num_buffers; le32 hash_value; (Only if VIRTIO_NET_F_HASH_REPORT negotiated) le16 hash_report; (Only if VIRTIO_NET_F_HASH_REPORT negotiated) - le16 padding_reserved; (Only if VIRTIO_NET_F_HASH_REPORT negotiated) + u8 hash_report_tunnel; (Only if VIRTIO_NET_F_HASH_REPORT negotiated, only valid of VIRTIO_NET_F_HASH_TUNNEL negotiated)of -> if?
Quoted 4 lines
Quoted 2 lines
Sorry for the late reply.
It's Cornelia's suggestion, and 'of' seems to work fine.I think that was simply a typo, should have been 'if'.
Sorry for the late reply due to vacation.
It is my mistake, sorry.
Thanks.
Quoted 5 lines
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Quoted 178 lines · Heng Qi
Quoted 80 lines · Michael S. Tsirkin
Quoted 79 lines · Heng Qi
If the tunnel is used to encapsulate the packets, the hash calculated using the outer header of the receive packets is always fixed for the same flow packets, i.e. they will be steered to the same receive queue.
We add a feature bit
VIRTIO_NET_F_HASH_TUNNELand related bitmasks in \field{hash_types}, which instructs the device to calculate the hash using the inner headers of tunnel-encapsulated packets. Besides, values in \field{hash_report_tunnel} are added to report tunnel types.Reviewed-by Jason Wang <[email protected]>Signed-off-by Heng Qi <[email protected]>Signed-off-by Xuan Zhuo <[email protected]>---
v6:
1. Modify the wording of some sentences for clarity. @Michael S. Tsirkin
2. Fix some syntax issues. @Michael S. Tsirkinv5:
1. Fix some syntax and capitalization issues. @Michael S. Tsirkin
2. Use encapsulated/encaptulation uniformly. @Michael S. Tsirkin
3. Move the links to introduction section. @Michael S. Tsirkin
4. Clarify some sentences. @Michael S. Tsirkinv4:
1. Clarify some paragraphs. @Cornelia Huck
2. Fix the u8 type. @Cornelia Huckv3:
1. RenameVIRTIO_NET_F_HASH_GRE_VXLAN_GENEVE_INNERtoVIRTIO_NET_F_HASH_TUNNEL. @Jason Wang
2. Make things clearer. @Jason Wang @Michael S. Tsirkin
3. Keep the possibility to use inner hash for automatic receive steering. @Jason Wang
4. Add the "Tunnel packet" paragraph to avoid repeating the GRE etc. many times. @Michael S. Tsirkinv2:
1. Add a feature bit for GRE/VXLAN/GENEVE inner hash. @Jason Wang
2. Chang \field{hash_tunnel} to \field{hash_report_tunnel}. @Jason Wang, @Michael S. Tsirkinv1:
1. Remove the patch for the bitmask fix. @Michael S. Tsirkin
2. Clarify some paragraphs. @Jason Wang
3. Add \field{hash_tunnel} andVIRTIO_NET_HASH_REPORT_GRE. @Yuri Benditovichcontent.tex | 191 +++++++++++++++++++++++++++++++++++++++++++++-- introduction.tex | 19 +++++ 2 files changed, 203 insertions(+), 7 deletions(-)content.tex +6
-1@@ -3084,6 +3084,9 @@ \subsection{Feature bits}\label{sec:Device Types / Network Device / Feature bits \item[VIRTIO_NET_F_CTRL_MAC_ADDR(23)] Set MAC address through control channel. +\item[VIRTIO_NET_F_HASH_TUNNEL(52)] Device supports inner + header hash for GRE, VXLAN and GENEVE tunnel-encapsulated packets. + \item[VIRTIO_NET_F_NOTF_COAL(53)] Device supports notifications coalescing. \item[VIRTIO_NET_F_GUEST_USO4 (54)] Driver can receive USOv4 packets.@@ -3095,7 +3098,8 @@ \subsection{Feature bits}\label{sec:Device Types / Network Device / Feature bits to several segments when each of these smaller packets has UDP header. \item[VIRTIO_NET_F_HASH_REPORT(57)] Device can report per-packet hash - value and a type of calculated hash. + value, a type of calculated hash, and, if VIRTIO_NET_F_HASH_TUNNEL + is negotiated, an encapsulation packet type. \item[VIRTIO_NET_F_GUEST_HDRLEN(59)] Driver can provide the exact \field{hdr_len} value. Device benefits from knowing the exact header length.@@ -3140,6 +3144,7 @@ \subsubsection{Feature bit requirements}\label{sec:Device Types / Network Device \item[VIRTIO_NET_F_NOTF_COAL] Requires VIRTIO_NET_F_CTRL_VQ. \item[VIRTIO_NET_F_RSC_EXT] Requires VIRTIO_NET_F_HOST_TSO4 or VIRTIO_NET_F_HOST_TSO6. \item[VIRTIO_NET_F_RSS] Requires VIRTIO_NET_F_CTRL_VQ. +\item[VIRTIO_NET_F_HASH_TUNNEL] Requires VIRTIO_NET_F_CTRL_VQ. \end{description}Quoted 8 lines
content.tex \subsection{Device Operation}\label{sec:Device Types / Network Device / Device O · line 3394
Quoted 7 lines
content.tex +1
-1\subsubsection{Legacy Interface: Feature bits}\label{sec:Device Types / Network Device / Feature bits / Legacy Interface: Feature bits}@@ -3386,7 +3391,8 @@ \subsection{Device Operation}\label{sec:Device Types / Network Device / Device O le16 num_buffers; le32 hash_value; (Only if VIRTIO_NET_F_HASH_REPORT negotiated) le16 hash_report; (Only if VIRTIO_NET_F_HASH_REPORT negotiated) - le16 padding_reserved; (Only if VIRTIO_NET_F_HASH_REPORT negotiated) + u8 hash_report_tunnel; (Only if VIRTIO_NET_F_HASH_REPORT negotiated, only valid of VIRTIO_NET_F_HASH_TUNNEL negotiated)of -> if?
Sorry for the late reply.
It's Cornelia's suggestion, and 'of' seems to work fine.Quoted 1 line
also let's add: otherwise reserved?
Sure.
Quoted 39 lines
Quoted 39 lines
content.tex +22
-1+ u8 padding_reserved; (Only if VIRTIO_NET_F_HASH_REPORT negotiated) }; \end{lstlisting}@@ -3837,7 +3843,8 @@ \subsubsection{Processing of Incoming Packets}\label{sec:Device Types / Network A device attempts to calculate a per-packet hash in the following cases: \begin{itemize} \item The feature VIRTIO_NET_F_RSS was negotiated. The device uses the hash to determine the receive virtqueue to place incoming packets. -\item The feature VIRTIO_NET_F_HASH_REPORT was negotiated. The device reports the hash value and the hash type with the packet. +\item The feature VIRTIO_NET_F_HASH_REPORT was negotiated. The device reports the hash value and the hash type. If additionally +VIRTIO_NET_F_HASH_TUNNEL was negotiated, the device reports the encapsulation type as well. \end{itemize} If the feature VIRTIO_NET_F_RSS was negotiated:@@ -3863,8 +3870,36 @@ \subsubsection{Processing of Incoming Packets}\label{sec:Device Types / Network \ref{sec:Device Types / Network Device / Device Operation / Control Virtqueue / Automatic receive steering in multiqueue mode / Hash calculation}. \end{itemize} +\subparagraph{Tunnel/Encapsulated packet} +\label{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / Tunnel/Encapsulated packet} +A tunnel packet is encapsulated from the original packet based on the tunneling +protocol (only a single level of encapsulation is currently supported). The +encapsulated packet contains an outer header and an inner header, and the device +calculates the hash over either the inner header or the outer header. + +When the feature VIRTIO_NET_F_HASH_TUNNEL is negotiated and the corresponding +encapsulation type is set in \field{hash_types}, the hash for a specific type of +encapsulated packet is calculated over the inner as opposed to outer header. +Supported encapsulation types are listed in \ref{sec:Device Types / Network Device / +Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / +Supported/enabled hash types}. + +If both VIRTIO_NET_F_HASH_REPORT and VIRTIO_NET_F_HASH_TUNNEL are negotiated, +the device can support inner hash calculation for \hyperref[intro:GRE]{[GRE]}, +\hyperref[intro:VXLAN]{[VXLAN]} and \hyperref[intro:GENEVE]{[GENEVE]} +encapsulated packets, and the corresponding encapsulation type in \field{hash_types} +is VIRTIO_NET_HASH_TYPE_{GRE, VXLAN, GENEVE}_INNER respectively. The value inQuoted 9 lines
content.tex \subsubsection{Processing of Incoming Packets}\label{sec:Device Types / Network · line 3897
Quoted 6 lines
content.tex +6
-0+\field{hash_report_tunnel} is VIRTIO_NET_HASH_REPORT_{NONE, GRE, VXLAN, GENEVE} respectively. + +If VIRTIO_NET_F_HASH_REPORT is negotiated but VIRTIO_NET_F_HASH_TUNNEL is not +negotiated, the device calculates the hash over the outer header, and +\field{hash_report} reports the hash type. \field{hash_report_tunnel} +is no longer valid.Does this mean that VIRTIO_NET_HASH_TYPE_{GRE, VXLAN, GENEVE}_INNER must not be set without
VIRTIO_NET_F_HASH_TUNNEL?Yes.
Quoted 4 lines
If yes then we can add conformance statements to this end and then drop the
"If the featureVIRTIO_NET_F_HASH_TUNNELis negotiated" all over the place just relying on hash type being set instead.Please see the response below.
Quoted 7 lines
Also, note that it has to be ok for device to expose hash types without negotiating
VIRTIO_NET_F_HASH_TUNNELsince they are in config space.
Idea: do we want to decouple these completely? haveVIRTIO_NET_F_HASH_TUNNELjust addhash_report_tunneland have hash types speak for themselves? It makes things nicely orthogonal in that if one does not care about knowing encapsulation type (e.g. for RSS) one can disableVIRTIO_NET_F_HASH_TUNNEL.This seems to have the following 3 problems:
1. This will break the specification. As described in the specification, any field
in the configuration space only exists when there is a corresponding feature bit:
"Device configuration space is generally used for rarely-changing or initialization-time parameters. Where configuration fields are optional, their existence is indicated by feature bits: ", which also applies tosupported_hash_types.2. If I'm not wrong, this seems to make the encapsulation hash types dependent on
VIRTIO_NET_F_RSS, can the driver readsupported_hash_types(including encapsulation hash types) ifVIRTIO_NET_F_RSSis not negotiated? But that shouldn't be the case, we only use encapsulation hash types to provide guidance for hash calculation, but not necessarily use hash values to select receive queues (unlessVIRTIO_NET_F_RSSis negotiated).3. If all
supported_hash_typesspeak for themselves, why do we need theVIRTIO_NET_F_HASH_TUNNELfeature, because as long as the encapsulation hash types exist andVIRTIO_NET_F_HASH_REPORTis negotiated,hash_report_tunnelexists correspondingly. "haveVIRTIO_NET_F_HASH_TUNNELjust addhash_report_tunnel"? It seems likehash_report_tunnelwill only work if the encapsulation hash types are present.
Quoted 6 lines
So, if I'm not wrong, the reasonable logic should be that
VIRTIO_NET_F_HASH_TUNNELis only used to declare that the device supports inner header hash and let the corresponding encapsulation hash type exist, whenVIRTIO_NET_F_RSSis negotiated, we can use this hash value to select the receive queue. Considering the migration at the same time, if the dst device does not negotiateVIRTIO_NET_F_HASH_TUNNEL, the migration will fail; otherwise, use encapsulation hash types to determine whether the migration can succeed.
I think it might be better to have consistency:
1) If VIRTIO_NET_F_HASH_TUNNEL introduces a new field in vnet header, is it better to have a new config filed in the config space?
or
2) If VIRTIO_NET_F_HASH_TUNNEL doesn't introduce new config file, should we try to reuse hash_report?
1) seems better and cleaner to me.
More below
Quoted 18 lines
Thanks.
Quoted 15 lines
Objections? Jason?
Quoted 12 lines
content.tex +3
-0+ \subparagraph{Supported/enabled hash types} \label{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / Supported/enabled hash types} +This paragraph relies on definitions from \hyperref[intro:IP]{[IP]}, +\hyperref[intro:UDP]{[UDP]} and \hyperref[intro:TCP]{[TCP]}. Hash types applicable for IPv4 packets: \begin{lstlisting}#define VIRTIO_NET_HASH_TYPE_IPv4 (1 << 0)content.tex +0
-0@@ -3884,6 +3919,22 @@ \subsubsection{Processing of Incoming Packets}\label{sec:Device Types / Network #define VIRTIO_NET_HASH_TYPE_UDP_EX (1 << 8) \end{lstlisting}
Quoted 15 lines
Quoted 7 lines
Quoted 7 lines
content.tex +7
-0+If the feature VIRTIO_NET_F_HASH_TUNNEL is negotiated, the encapsulation +hash type below indicates that the hash is calculated over the inner +header of the encapsulated packet: +Hash type applicable for inner payload of the gre-encapsulated packet +\begin{lstlisting} +#define VIRTIO_NET_HASH_TYPE_GRE_INNER (1 << 9) +\end{lstlisting}Quoted 8 lines
content.tex \subsubsection{Processing of Incoming Packets}\label{sec:Device Types / Network · line 3936
Quoted 8 lines
content.tex +8
-0+Hash type applicable for inner payload of the vxlan-encapsulated packet +\begin{lstlisting} +#define VIRTIO_NET_HASH_TYPE_VXLAN_INNER (1 << 10) +\end{lstlisting} +Hash type applicable for inner payload of the geneve-encapsulated packet +\begin{lstlisting} +#define VIRTIO_NET_HASH_TYPE_GENEVE_INNER (1 << 11) +\end{lstlisting}
Just notice that this seems to break the equation:
VIRTIO_NET_HASH_TYPE_XXX = 1 « (VIRTIO_NET_HASH_REPORT_XXX - 1)
?
Thanks
Quoted 205 lines
Quoted 205 lines
Quoted 189 lines
content.tex +131
-4+ \subparagraph{IPv4 packets} \label{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / IPv4 packets} The device calculates the hash on IPv4 packets according to 'Enabled hash types' bitmask as follows:@@ -3975,15 +4026,125 @@ \subsubsection{Processing of Incoming Packets}\label{sec:Device Types / Network (see \ref{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / IPv6 packets without extension header}). \end{itemize} +\subparagraph{Inner payload of an encapsulated packet} +\label{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / Inner payload of the encapsulated packet} +If the feature VIRTIO_NET_F_HASH_TUNNEL is negotiated and the corresponding +encapsulation hash type is set in \field{hash_types}, the device calculates the +inner header hash on an encapsulated packet (See \ref{sec:Device Types +/ Network Device / Device Operation / Processing of Incoming Packets / +Hash calculation for incoming packets / Tunnel/Encapsulated packet}) as follows: + +The device calculates the hash on the inner IPv4 packet of an encapsulated packet +according to 'Enabled hash types' bitmask as follows: +\begin{itemize} + \item If VIRTIO_NET_HASH_TYPE_TCPv4 is set and the encapsulated packet has an inner + TCPv4 header, the hash is calculated over the following fields: + \begin{itemsize} + \item inner Source IP address + \item inner Destination IP address + \item inner Source TCP port + \item inner Destination TCP port + \end{itemsize} + \item Else if VIRTIO_NET_HASH_TYPE_UDPv4 is set and the encapsulated packet has an + inner UDPv4 header, the hash is calculated over the following fields: + \begin{itemsize} + \item inner Source IP address + \item inner Destination IP address + \item inner Source UDP port + \item inner Destination UDP port + \end{itemize} + \item Else if VIRTIO_NET_HASH_TYPE_IPv4 is set, the hash is calculated over the + following fields: + \begin{itemsize} + \item inner Source IP address + \item inner Destination IP address + \end{itemsize} + \item Else the device does not calculate the hash +\end{itemize} + +The device calculates the hash on the inner IPv6 packet without an extension header +of an encapsulated packet according to 'Enabled hash types' bitmask as follows: +\begin{itemize} + \item If VIRTIO_NET_HASH_TYPE_TCPv6 is set and the encapsulated packet has an inner + TCPv6 header, the hash is calculated over the following fields: + \begin{itemsize} + \item inner Source IPv6 address + \item inner Destination IPv6 address + \item inner Source TCP port + \item inner Destination TCP port + \end{itemsize} + \item Else if VIRTIO_NET_HASH_TYPE_UDPv6 is set and the encapsulated packet has an + inner UDPv6 header, the hash is calculated over the following fields: + \begin{itemsize} + \item inner Source IPv6 address + \item inner Destination IPv6 address + \item inner Source UDP port + \item inner Destination UDP port + \end{itemize} + \item Else if VIRTIO_NET_HASH_TYPE_IPv6 is set, the hash is calculated over the + following fields: + \begin{itemsize} + \item inner Source IPv6 address + \item inner Destination IPv6 address + \end{itemsize} + \item Else the device does not calculate the hash +\end{itemize} + +The device calculates the hash on the inner IPv6 packet with an extension header +of an encapsulated packet according to 'Enabled hash types' bitmask as follows: +\begin{itemsize} + \item If VIRTIO_NET_HASH_TYPE_TCP_EX is set and the encapsulated packet has an inner + TCPv6 header, the hash is calculated over the following fields: + \begin{itemize} + \item Home address from the home address option in the inner IPv6 destination + options header. If the inner extension header is not present, use the + inner Source IPv6 address. + \item IPv6 address that is contained in the Routing-Header-Type-2 from the + associated inner extension header. If the inner extension header is not + present, use the inner Destination IPv6 address. + \item inner Source TCP port + \item inner Destination TCP port + \end{itemize} + \item Else if VIRTIO_NET_HASH_TYPE_UDP_EX is set and the encapsulated packet has an inner + UDPv6 header, the hash is calculated over the following fields: + \begin{itemsize} + \item Home address from the home address option in the inner IPv6 destination + options header. If the inner extension header is not present, use the + inner Source IPv6 address. + \item IPv6 address that is contained in the Routing-Header-Type-2 from the + associated inner extension header. If the inner extension header is not + present, use the inner Destination IPv6 address. + \item inner Source UDP port + \item inner Destination UDP port + \end{itemize} + \item Else if VIRTIO_NET_HASH_TYPE_IP_EX is set, the hash is calculated over the + following fields: + \begin{itemsize} + \item Home address from the home address option in the inner IPv6 destination + options header. If the inner extension header is not present, use the + inner Source IPv6 address. + \item IPv6 address that is contained in the Routing-Header-Type-2 from the + associated inner extension header. If the inner extension header is not + present, use the inner Destination IPv6 address. + \end{itemize} + \item Else skip the inner IPv6 extension header and calculate the inner header hash as + defined for an encapsulated packet whose inner payload is an IPv6 packet without + an extension header. +\end{itemsize} + \paragraph{Hash reporting for incoming packets} \label{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash reporting for incoming packets} -If VIRTIO_NET_F_HASH_REPORT was negotiated and - the device has calculated the hash for the packet, the device fills \field{hash_report} with the report type of calculated hash -and \field{hash_value} with the value of calculated hash. +If VIRTIO_NET_F_HASH_REPORT was negotiated and the device has calculated the +hash for the packet, the device fills \field{hash_report} with the report type +of calculated hash, and \field{hash_value} with the value of calculated hash. +Also, if VIRTIO_NET_F_HASH_TUNNEL was negotiated, the device needs to fill +\field{hash_report_tunnel} with the report type of the encapsulated packet, and +it is set to VIRTIO_NET_HASH_REPORT_TUNNEL_NONE for the unencapsulated packet. If VIRTIO_NET_F_HASH_REPORT was negotiated but due to any reason the -hash was not calculated, the device sets \field{hash_report} to VIRTIO_NET_HASH_REPORT_NONE. +hash was not calculated, the device sets \field{hash_report} to VIRTIO_NET_HASH_REPORT_NONE, +and sets \field{hash_report_tunnel} to VIRTIO_NET_HASH_REPORT_TUNNEL_NONE. Possible values that the device can report in \field{hash_report} are defined below. They correspond to supported hash types defined in@@ -4005,6 +4166,22 @@ \subsubsection{Processing of Incoming Packets}\label{sec:Device Types / Network #define VIRTIO_NET_HASH_REPORT_UDPv6_EX 9 \end{lstlisting} +If \field{hash_report} differs from VIRTIO_NET_HASH_REPORT_NONE, +\field{hash_report_tunnel} can report the type of the encapsulated +packet to the driver over the inner header hash calculation. +Possible values that the device can report in \field{hash_report_tunnel} +are defined below: + +\begin{lstlisting} +#define VIRTIO_NET_HASH_REPORT_TUNNEL_NONE 0 +#define VIRTIO_NET_HASH_REPORT_GRE 1 +#define VIRTIO_NET_HASH_REPORT_VXLAN 2 +#define VIRTIO_NET_HASH_REPORT_GENEVE 3 +\end{lstlisting} + +They correspond to supported hash types defined in +\ref{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / Supported/enabled hash types}. + \subsubsection{Control Virtqueue}\label{sec:Device Types / Network Device / Device Operation / Control Virtqueue} The driver uses the control virtqueue (if VIRTIO_NET_F_CTRL_VQ isintroduction.tex +19
-0@@ -98,6 +98,25 @@ \section{Normative References}\label{sec:Normative References} \phantomsection\label{intro:SEC1}\textbf{[SEC1]} & Standards for Efficient Cryptography Group(SECG), ``SEC1: Elliptic Cureve Cryptography'', Version 1.0, September 2000. \newline\url{https://www.secg.org/sec1-v2.pdf}\\ + \phantomsection\label{intro:GRE}\textbf{[GRE]} & + Generic Routing Encapsulation + \newline\url{https://datatracker.ietf.org/doc/rfc2784/}\\ + \phantomsection\label{intro:VXLAN}\textbf{[VXLAN]} & + Virtual eXtensible Local Area Network + \newline\url{https://datatracker.ietf.org/doc/rfc7348/}\\ + \phantomsection\label{intro:GENEVE}\textbf{[GENEVE]} & + Generic Network Virtualization Encapsulation + \newline\url{https://datatracker.ietf.org/doc/rfc8926/}\\ + \phantomsection\label{intro:IP}\textbf{[IP]} & + INTERNET PROTOCOL + \newline\url{https://www.rfc-editor.org/rfc/rfc791}\\ + \phantomsection\label{intro:UDP}\textbf{[UDP]} & + User Datagram Protocol + \newline\url{https://www.rfc-editor.org/rfc/rfc768}\\ + \phantomsection\label{intro:TCP}\textbf{[TCP]} & + TRANSMISSION CONTROL PROTOCOL + \newline\url{https://www.rfc-editor.org/rfc/rfc793}\\ + \end{longtable}Signature · 1 line
2.19.1.6.gb485710bThis publicly archived list offers a means to provide input to the OASIS Virtual I/O Device (VIRTIO) TC.
In order to verify user consent to the Feedback License terms and to minimize spam in the list archive, subscription is required before posting.
Subscribe: [email protected]
Unsubscribe: [email protected]
List help: [email protected]
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/
Quoted 185 lines · Jason Wang
Quoted 170 lines · Heng Qi
Quoted 75 lines · Michael S. Tsirkin
Quoted 74 lines · Heng Qi
If the tunnel is used to encapsulate the packets, the hash calculated using the outer header of the receive packets is always fixed for the same flow packets, i.e. they will be steered to the same receive queue.
We add a feature bit
VIRTIO_NET_F_HASH_TUNNELand related bitmasks in \field{hash_types}, which instructs the device to calculate the hash using the inner headers of tunnel-encapsulated packets. Besides, values in \field{hash_report_tunnel} are added to report tunnel types.Reviewed-by Jason Wang <[email protected]>Signed-off-by Heng Qi <[email protected]>Signed-off-by Xuan Zhuo <[email protected]>---
v6:
1. Modify the wording of some sentences for clarity. @Michael S. Tsirkin
2. Fix some syntax issues. @Michael S. Tsirkinv5:
1. Fix some syntax and capitalization issues. @Michael S. Tsirkin
2. Use encapsulated/encaptulation uniformly. @Michael S. Tsirkin
3. Move the links to introduction section. @Michael S. Tsirkin
4. Clarify some sentences. @Michael S. Tsirkinv4:
1. Clarify some paragraphs. @Cornelia Huck
2. Fix the u8 type. @Cornelia Huckv3:
1. RenameVIRTIO_NET_F_HASH_GRE_VXLAN_GENEVE_INNERtoVIRTIO_NET_F_HASH_TUNNEL. @Jason Wang
2. Make things clearer. @Jason Wang @Michael S. Tsirkin
3. Keep the possibility to use inner hash for automatic receive steering. @Jason Wang
4. Add the "Tunnel packet" paragraph to avoid repeating the GRE etc. many times. @Michael S. Tsirkinv2:
1. Add a feature bit for GRE/VXLAN/GENEVE inner hash. @Jason Wang
2. Chang \field{hash_tunnel} to \field{hash_report_tunnel}. @Jason Wang, @Michael S. Tsirkinv1:
1. Remove the patch for the bitmask fix. @Michael S. Tsirkin
2. Clarify some paragraphs. @Jason Wang
3. Add \field{hash_tunnel} andVIRTIO_NET_HASH_REPORT_GRE. @Yuri Benditovichcontent.tex | 191 +++++++++++++++++++++++++++++++++++++++++++++-- introduction.tex | 19 +++++ 2 files changed, 203 insertions(+), 7 deletions(-)content.tex +6
-1@@ -3084,6 +3084,9 @@ \subsection{Feature bits}\label{sec:Device Types / Network Device / Feature bits \item[VIRTIO_NET_F_CTRL_MAC_ADDR(23)] Set MAC address through control channel. +\item[VIRTIO_NET_F_HASH_TUNNEL(52)] Device supports inner + header hash for GRE, VXLAN and GENEVE tunnel-encapsulated packets. + \item[VIRTIO_NET_F_NOTF_COAL(53)] Device supports notifications coalescing. \item[VIRTIO_NET_F_GUEST_USO4 (54)] Driver can receive USOv4 packets.@@ -3095,7 +3098,8 @@ \subsection{Feature bits}\label{sec:Device Types / Network Device / Feature bits to several segments when each of these smaller packets has UDP header. \item[VIRTIO_NET_F_HASH_REPORT(57)] Device can report per-packet hash - value and a type of calculated hash. + value, a type of calculated hash, and, if VIRTIO_NET_F_HASH_TUNNEL + is negotiated, an encapsulation packet type. \item[VIRTIO_NET_F_GUEST_HDRLEN(59)] Driver can provide the exact \field{hdr_len} value. Device benefits from knowing the exact header length.@@ -3140,6 +3144,7 @@ \subsubsection{Feature bit requirements}\label{sec:Device Types / Network Device \item[VIRTIO_NET_F_NOTF_COAL] Requires VIRTIO_NET_F_CTRL_VQ. \item[VIRTIO_NET_F_RSC_EXT] Requires VIRTIO_NET_F_HOST_TSO4 or VIRTIO_NET_F_HOST_TSO6. \item[VIRTIO_NET_F_RSS] Requires VIRTIO_NET_F_CTRL_VQ. +\item[VIRTIO_NET_F_HASH_TUNNEL] Requires VIRTIO_NET_F_CTRL_VQ. \end{description}Quoted 8 lines
content.tex \subsection{Device Operation}\label{sec:Device Types / Network Device / Device O · line 3394
Quoted 7 lines
content.tex +1
-1\subsubsection{Legacy Interface: Feature bits}\label{sec:Device Types / Network Device / Feature bits / Legacy Interface: Feature bits}@@ -3386,7 +3391,8 @@ \subsection{Device Operation}\label{sec:Device Types / Network Device / Device O le16 num_buffers; le32 hash_value; (Only if VIRTIO_NET_F_HASH_REPORT negotiated) le16 hash_report; (Only if VIRTIO_NET_F_HASH_REPORT negotiated) - le16 padding_reserved; (Only if VIRTIO_NET_F_HASH_REPORT negotiated) + u8 hash_report_tunnel; (Only if VIRTIO_NET_F_HASH_REPORT negotiated, only valid of VIRTIO_NET_F_HASH_TUNNEL negotiated)of -> if?
Sorry for the late reply.
It's Cornelia's suggestion, and 'of' seems to work fine.Quoted 1 line
also let's add: otherwise reserved?
Sure.
Quoted 36 lines
Quoted 36 lines
content.tex +22
-1+ u8 padding_reserved; (Only if VIRTIO_NET_F_HASH_REPORT negotiated) }; \end{lstlisting}@@ -3837,7 +3843,8 @@ \subsubsection{Processing of Incoming Packets}\label{sec:Device Types / Network A device attempts to calculate a per-packet hash in the following cases: \begin{itemize} \item The feature VIRTIO_NET_F_RSS was negotiated. The device uses the hash to determine the receive virtqueue to place incoming packets. -\item The feature VIRTIO_NET_F_HASH_REPORT was negotiated. The device reports the hash value and the hash type with the packet. +\item The feature VIRTIO_NET_F_HASH_REPORT was negotiated. The device reports the hash value and the hash type. If additionally +VIRTIO_NET_F_HASH_TUNNEL was negotiated, the device reports the encapsulation type as well. \end{itemize} If the feature VIRTIO_NET_F_RSS was negotiated:@@ -3863,8 +3870,36 @@ \subsubsection{Processing of Incoming Packets}\label{sec:Device Types / Network \ref{sec:Device Types / Network Device / Device Operation / Control Virtqueue / Automatic receive steering in multiqueue mode / Hash calculation}. \end{itemize} +\subparagraph{Tunnel/Encapsulated packet} +\label{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / Tunnel/Encapsulated packet} +A tunnel packet is encapsulated from the original packet based on the tunneling +protocol (only a single level of encapsulation is currently supported). The +encapsulated packet contains an outer header and an inner header, and the device +calculates the hash over either the inner header or the outer header. + +When the feature VIRTIO_NET_F_HASH_TUNNEL is negotiated and the corresponding +encapsulation type is set in \field{hash_types}, the hash for a specific type of +encapsulated packet is calculated over the inner as opposed to outer header. +Supported encapsulation types are listed in \ref{sec:Device Types / Network Device / +Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / +Supported/enabled hash types}. + +If both VIRTIO_NET_F_HASH_REPORT and VIRTIO_NET_F_HASH_TUNNEL are negotiated, +the device can support inner hash calculation for \hyperref[intro:GRE]{[GRE]}, +\hyperref[intro:VXLAN]{[VXLAN]} and \hyperref[intro:GENEVE]{[GENEVE]} +encapsulated packets, and the corresponding encapsulation type in \field{hash_types} +is VIRTIO_NET_HASH_TYPE_{GRE, VXLAN, GENEVE}_INNER respectively. The value inQuoted 9 lines
content.tex \subsubsection{Processing of Incoming Packets}\label{sec:Device Types / Network · line 3897
Quoted 6 lines
content.tex +6
-0+\field{hash_report_tunnel} is VIRTIO_NET_HASH_REPORT_{NONE, GRE, VXLAN, GENEVE} respectively. + +If VIRTIO_NET_F_HASH_REPORT is negotiated but VIRTIO_NET_F_HASH_TUNNEL is not +negotiated, the device calculates the hash over the outer header, and +\field{hash_report} reports the hash type. \field{hash_report_tunnel} +is no longer valid.Does this mean that VIRTIO_NET_HASH_TYPE_{GRE, VXLAN, GENEVE}_INNER must not be set without
VIRTIO_NET_F_HASH_TUNNEL?Yes.
Quoted 4 lines
If yes then we can add conformance statements to this end and then drop the
"If the featureVIRTIO_NET_F_HASH_TUNNELis negotiated" all over the place just relying on hash type being set instead.Please see the response below.
Quoted 7 lines
Also, note that it has to be ok for device to expose hash types without negotiating
VIRTIO_NET_F_HASH_TUNNELsince they are in config space.
Idea: do we want to decouple these completely? haveVIRTIO_NET_F_HASH_TUNNELjust addhash_report_tunneland have hash types speak for themselves? It makes things nicely orthogonal in that if one does not care about knowing encapsulation type (e.g. for RSS) one can disableVIRTIO_NET_F_HASH_TUNNEL.This seems to have the following 3 problems:
1. This will break the specification. As described in the specification, any field
in the configuration space only exists when there is a corresponding feature bit:
"Device configuration space is generally used for rarely-changing or initialization-time parameters. Where configuration fields are optional, their existence is indicated by feature bits: ", which also applies tosupported_hash_types.2. If I'm not wrong, this seems to make the encapsulation hash types dependent on
VIRTIO_NET_F_RSS, can the driver readsupported_hash_types(including encapsulation hash types) ifVIRTIO_NET_F_RSSis not negotiated? But that shouldn't be the case, we only use encapsulation hash types to provide guidance for hash calculation, but not necessarily use hash values to select receive queues (unlessVIRTIO_NET_F_RSSis negotiated).3. If all
supported_hash_typesspeak for themselves, why do we need theVIRTIO_NET_F_HASH_TUNNELfeature, because as long as the encapsulation hash types exist andVIRTIO_NET_F_HASH_REPORTis negotiated,hash_report_tunnelexists correspondingly. "haveVIRTIO_NET_F_HASH_TUNNELjust addhash_report_tunnel"? It seems likehash_report_tunnelwill only work if the encapsulation hash types are present.Quoted 6 lines
So, if I'm not wrong, the reasonable logic should be that
VIRTIO_NET_F_HASH_TUNNELis only used to declare that the device supports inner header hash and let the corresponding encapsulation hash type exist, whenVIRTIO_NET_F_RSSis negotiated, we can use this hash value to select the receive queue. Considering the migration at the same time, if the dst device does not negotiateVIRTIO_NET_F_HASH_TUNNEL, the migration will fail; otherwise, use encapsulation hash types to determine whether the migration can succeed.I think it might be better to have consistency:
1) If
VIRTIO_NET_F_HASH_TUNNELintroduces a new field in vnet header, is it better to have a new config filed in the config space?
Quoted 6 lines
or
2) If
VIRTIO_NET_F_HASH_TUNNELdoesn't introduce new config file, should we try to reusehash_report?1) seems better and cleaner to me.
Sorry for the late reply due to vacation.
Good point, it's clearer to use the VIRTIO_NET_F_HASH_TUNNEL feature to control all information related to inner header hash.
Quoted 40 lines
More below
Quoted 24 lines
Thanks.
Quoted 21 lines
Objections? Jason?
Quoted 18 lines
content.tex +10
-0+ \subparagraph{Supported/enabled hash types} \label{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / Supported/enabled hash types} +This paragraph relies on definitions from \hyperref[intro:IP]{[IP]}, +\hyperref[intro:UDP]{[UDP]} and \hyperref[intro:TCP]{[TCP]}. Hash types applicable for IPv4 packets: \begin{lstlisting} #define VIRTIO_NET_HASH_TYPE_IPv4 (1 << 0)@@ -3884,6 +3919,22 @@ \subsubsection{Processing of Incoming Packets}\label{sec:Device Types / Network #define VIRTIO_NET_HASH_TYPE_UDP_EX (1 << 8) \end{lstlisting} +If the feature VIRTIO_NET_F_HASH_TUNNEL is negotiated, the encapsulation +hash type below indicates that the hash is calculated over the inner +header of the encapsulated packet: +Hash type applicable for inner payload of the gre-encapsulated packet +\begin{lstlisting} +#define VIRTIO_NET_HASH_TYPE_GRE_INNER (1 << 9) +\end{lstlisting}Quoted 8 lines
Quoted 8 lines
content.tex \subsubsection{Processing of Incoming Packets}\label{sec:Device Types / Network · line 3936
Quoted 8 lines
content.tex +8
-0+Hash type applicable for inner payload of the vxlan-encapsulated packet +\begin{lstlisting} +#define VIRTIO_NET_HASH_TYPE_VXLAN_INNER (1 << 10) +\end{lstlisting} +Hash type applicable for inner payload of the geneve-encapsulated packet +\begin{lstlisting} +#define VIRTIO_NET_HASH_TYPE_GENEVE_INNER (1 << 11) +\end{lstlisting}Just notice that this seems to break the equation:
Quoted 4 lines
VIRTIO_NET_HASH_TYPE_XXX= 1 « (VIRTIO_NET_HASH_REPORT_XXX- 1)?
Yes, we have discussed it here https://lists.oasis-open.org/archives/virtio-dev/202211/msg00189.html , as you suggested above, it seems better to use the new config space field.
Thanks.
Quoted 205 lines
Thanks
Quoted 189 lines
Quoted 189 lines
Quoted 181 lines
content.tex +131
-4+ \subparagraph{IPv4 packets} \label{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / IPv4 packets} The device calculates the hash on IPv4 packets according to 'Enabled hash types' bitmask as follows:@@ -3975,15 +4026,125 @@ \subsubsection{Processing of Incoming Packets}\label{sec:Device Types / Network (see \ref{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / IPv6 packets without extension header}). \end{itemize} +\subparagraph{Inner payload of an encapsulated packet} +\label{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / Inner payload of the encapsulated packet} +If the feature VIRTIO_NET_F_HASH_TUNNEL is negotiated and the corresponding +encapsulation hash type is set in \field{hash_types}, the device calculates the +inner header hash on an encapsulated packet (See \ref{sec:Device Types +/ Network Device / Device Operation / Processing of Incoming Packets / +Hash calculation for incoming packets / Tunnel/Encapsulated packet}) as follows: + +The device calculates the hash on the inner IPv4 packet of an encapsulated packet +according to 'Enabled hash types' bitmask as follows: +\begin{itemize} + \item If VIRTIO_NET_HASH_TYPE_TCPv4 is set and the encapsulated packet has an inner + TCPv4 header, the hash is calculated over the following fields: + \begin{itemsize} + \item inner Source IP address + \item inner Destination IP address + \item inner Source TCP port + \item inner Destination TCP port + \end{itemsize} + \item Else if VIRTIO_NET_HASH_TYPE_UDPv4 is set and the encapsulated packet has an + inner UDPv4 header, the hash is calculated over the following fields: + \begin{itemsize} + \item inner Source IP address + \item inner Destination IP address + \item inner Source UDP port + \item inner Destination UDP port + \end{itemize} + \item Else if VIRTIO_NET_HASH_TYPE_IPv4 is set, the hash is calculated over the + following fields: + \begin{itemsize} + \item inner Source IP address + \item inner Destination IP address + \end{itemsize} + \item Else the device does not calculate the hash +\end{itemize} + +The device calculates the hash on the inner IPv6 packet without an extension header +of an encapsulated packet according to 'Enabled hash types' bitmask as follows: +\begin{itemize} + \item If VIRTIO_NET_HASH_TYPE_TCPv6 is set and the encapsulated packet has an inner + TCPv6 header, the hash is calculated over the following fields: + \begin{itemsize} + \item inner Source IPv6 address + \item inner Destination IPv6 address + \item inner Source TCP port + \item inner Destination TCP port + \end{itemsize} + \item Else if VIRTIO_NET_HASH_TYPE_UDPv6 is set and the encapsulated packet has an + inner UDPv6 header, the hash is calculated over the following fields: + \begin{itemsize} + \item inner Source IPv6 address + \item inner Destination IPv6 address + \item inner Source UDP port + \item inner Destination UDP port + \end{itemize} + \item Else if VIRTIO_NET_HASH_TYPE_IPv6 is set, the hash is calculated over the + following fields: + \begin{itemsize} + \item inner Source IPv6 address + \item inner Destination IPv6 address + \end{itemsize} + \item Else the device does not calculate the hash +\end{itemize} + +The device calculates the hash on the inner IPv6 packet with an extension header +of an encapsulated packet according to 'Enabled hash types' bitmask as follows: +\begin{itemsize} + \item If VIRTIO_NET_HASH_TYPE_TCP_EX is set and the encapsulated packet has an inner + TCPv6 header, the hash is calculated over the following fields: + \begin{itemize} + \item Home address from the home address option in the inner IPv6 destination + options header. If the inner extension header is not present, use the + inner Source IPv6 address. + \item IPv6 address that is contained in the Routing-Header-Type-2 from the + associated inner extension header. If the inner extension header is not + present, use the inner Destination IPv6 address. + \item inner Source TCP port + \item inner Destination TCP port + \end{itemize} + \item Else if VIRTIO_NET_HASH_TYPE_UDP_EX is set and the encapsulated packet has an inner + UDPv6 header, the hash is calculated over the following fields: + \begin{itemsize} + \item Home address from the home address option in the inner IPv6 destination + options header. If the inner extension header is not present, use the + inner Source IPv6 address. + \item IPv6 address that is contained in the Routing-Header-Type-2 from the + associated inner extension header. If the inner extension header is not + present, use the inner Destination IPv6 address. + \item inner Source UDP port + \item inner Destination UDP port + \end{itemize} + \item Else if VIRTIO_NET_HASH_TYPE_IP_EX is set, the hash is calculated over the + following fields: + \begin{itemsize} + \item Home address from the home address option in the inner IPv6 destination + options header. If the inner extension header is not present, use the + inner Source IPv6 address. + \item IPv6 address that is contained in the Routing-Header-Type-2 from the + associated inner extension header. If the inner extension header is not + present, use the inner Destination IPv6 address. + \end{itemize} + \item Else skip the inner IPv6 extension header and calculate the inner header hash as + defined for an encapsulated packet whose inner payload is an IPv6 packet without + an extension header. +\end{itemsize} + \paragraph{Hash reporting for incoming packets} \label{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash reporting for incoming packets} -If VIRTIO_NET_F_HASH_REPORT was negotiated and - the device has calculated the hash for the packet, the device fills \field{hash_report} with the report type of calculated hash -and \field{hash_value} with the value of calculated hash. +If VIRTIO_NET_F_HASH_REPORT was negotiated and the device has calculated the +hash for the packet, the device fills \field{hash_report} with the report type +of calculated hash, and \field{hash_value} with the value of calculated hash. +Also, if VIRTIO_NET_F_HASH_TUNNEL was negotiated, the device needs to fill +\field{hash_report_tunnel} with the report type of the encapsulated packet, and +it is set to VIRTIO_NET_HASH_REPORT_TUNNEL_NONE for the unencapsulated packet. If VIRTIO_NET_F_HASH_REPORT was negotiated but due to any reason the -hash was not calculated, the device sets \field{hash_report} to VIRTIO_NET_HASH_REPORT_NONE. +hash was not calculated, the device sets \field{hash_report} to VIRTIO_NET_HASH_REPORT_NONE, +and sets \field{hash_report_tunnel} to VIRTIO_NET_HASH_REPORT_TUNNEL_NONE. Possible values that the device can report in \field{hash_report} are defined below. They correspond to supported hash types defined in@@ -4005,6 +4166,22 @@ \subsubsection{Processing of Incoming Packets}\label{sec:Device Types / Network #define VIRTIO_NET_HASH_REPORT_UDPv6_EX 9 \end{lstlisting} +If \field{hash_report} differs from VIRTIO_NET_HASH_REPORT_NONE, +\field{hash_report_tunnel} can report the type of the encapsulated +packet to the driver over the inner header hash calculation. +Possible values that the device can report in \field{hash_report_tunnel} +are defined below: + +\begin{lstlisting} +#define VIRTIO_NET_HASH_REPORT_TUNNEL_NONE 0 +#define VIRTIO_NET_HASH_REPORT_GRE 1 +#define VIRTIO_NET_HASH_REPORT_VXLAN 2 +#define VIRTIO_NET_HASH_REPORT_GENEVE 3 +\end{lstlisting} + +They correspond to supported hash types defined in +\ref{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / Supported/enabled hash types}. + \subsubsection{Control Virtqueue}\label{sec:Device Types / Network Device / Device Operation / Control Virtqueue} The driver uses the control virtqueue (if VIRTIO_NET_F_CTRL_VQ isintroduction.tex +19
-0@@ -98,6 +98,25 @@ \section{Normative References}\label{sec:Normative References} \phantomsection\label{intro:SEC1}\textbf{[SEC1]} & Standards for Efficient Cryptography Group(SECG), ``SEC1: Elliptic Cureve Cryptography'', Version 1.0, September 2000. \newline\url{https://www.secg.org/sec1-v2.pdf}\\ + \phantomsection\label{intro:GRE}\textbf{[GRE]} & + Generic Routing Encapsulation + \newline\url{https://datatracker.ietf.org/doc/rfc2784/}\\ + \phantomsection\label{intro:VXLAN}\textbf{[VXLAN]} & + Virtual eXtensible Local Area Network + \newline\url{https://datatracker.ietf.org/doc/rfc7348/}\\ + \phantomsection\label{intro:GENEVE}\textbf{[GENEVE]} & + Generic Network Virtualization Encapsulation + \newline\url{https://datatracker.ietf.org/doc/rfc8926/}\\ + \phantomsection\label{intro:IP}\textbf{[IP]} & + INTERNET PROTOCOL + \newline\url{https://www.rfc-editor.org/rfc/rfc791}\\ + \phantomsection\label{intro:UDP}\textbf{[UDP]} & + User Datagram Protocol + \newline\url{https://www.rfc-editor.org/rfc/rfc768}\\ + \phantomsection\label{intro:TCP}\textbf{[TCP]} & + TRANSMISSION CONTROL PROTOCOL + \newline\url{https://www.rfc-editor.org/rfc/rfc793}\\ + \end{longtable}Signature · 1 line
2.19.1.6.gb485710bThis publicly archived list offers a means to provide input to the OASIS Virtual I/O Device (VIRTIO) TC.
In order to verify user consent to the Feedback License terms and to minimize spam in the list archive, subscription is required before posting.
Quoted 8 lines
Quoted 8 lines
Subscribe: [email protected]
Unsubscribe: [email protected]
List help: [email protected]
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Quoted 184 lines · Heng Qi
Quoted 183 lines · Jason Wang
Quoted 168 lines · Heng Qi
Quoted 75 lines · Michael S. Tsirkin
Quoted 74 lines · Heng Qi
If the tunnel is used to encapsulate the packets, the hash calculated using the outer header of the receive packets is always fixed for the same flow packets, i.e. they will be steered to the same receive queue.
We add a feature bit
VIRTIO_NET_F_HASH_TUNNELand related bitmasks in \field{hash_types}, which instructs the device to calculate the hash using the inner headers of tunnel-encapsulated packets. Besides, values in \field{hash_report_tunnel} are added to report tunnel types.Reviewed-by Jason Wang <[email protected]>Signed-off-by Heng Qi <[email protected]>Signed-off-by Xuan Zhuo <[email protected]>---
v6:
1. Modify the wording of some sentences for clarity. @Michael S. Tsirkin
2. Fix some syntax issues. @Michael S. Tsirkinv5:
1. Fix some syntax and capitalization issues. @Michael S. Tsirkin
2. Use encapsulated/encaptulation uniformly. @Michael S. Tsirkin
3. Move the links to introduction section. @Michael S. Tsirkin
4. Clarify some sentences. @Michael S. Tsirkinv4:
1. Clarify some paragraphs. @Cornelia Huck
2. Fix the u8 type. @Cornelia Huckv3:
1. RenameVIRTIO_NET_F_HASH_GRE_VXLAN_GENEVE_INNERtoVIRTIO_NET_F_HASH_TUNNEL. @Jason Wang
2. Make things clearer. @Jason Wang @Michael S. Tsirkin
3. Keep the possibility to use inner hash for automatic receive steering. @Jason Wang
4. Add the "Tunnel packet" paragraph to avoid repeating the GRE etc. many times. @Michael S. Tsirkinv2:
1. Add a feature bit for GRE/VXLAN/GENEVE inner hash. @Jason Wang
2. Chang \field{hash_tunnel} to \field{hash_report_tunnel}. @Jason Wang, @Michael S. Tsirkinv1:
1. Remove the patch for the bitmask fix. @Michael S. Tsirkin
2. Clarify some paragraphs. @Jason Wang
3. Add \field{hash_tunnel} andVIRTIO_NET_HASH_REPORT_GRE. @Yuri Benditovichcontent.tex | 191 +++++++++++++++++++++++++++++++++++++++++++++-- introduction.tex | 19 +++++ 2 files changed, 203 insertions(+), 7 deletions(-)content.tex +6
-1@@ -3084,6 +3084,9 @@ \subsection{Feature bits}\label{sec:Device Types / Network Device / Feature bits \item[VIRTIO_NET_F_CTRL_MAC_ADDR(23)] Set MAC address through control channel. +\item[VIRTIO_NET_F_HASH_TUNNEL(52)] Device supports inner + header hash for GRE, VXLAN and GENEVE tunnel-encapsulated packets. + \item[VIRTIO_NET_F_NOTF_COAL(53)] Device supports notifications coalescing. \item[VIRTIO_NET_F_GUEST_USO4 (54)] Driver can receive USOv4 packets.@@ -3095,7 +3098,8 @@ \subsection{Feature bits}\label{sec:Device Types / Network Device / Feature bits to several segments when each of these smaller packets has UDP header. \item[VIRTIO_NET_F_HASH_REPORT(57)] Device can report per-packet hash - value and a type of calculated hash. + value, a type of calculated hash, and, if VIRTIO_NET_F_HASH_TUNNEL + is negotiated, an encapsulation packet type. \item[VIRTIO_NET_F_GUEST_HDRLEN(59)] Driver can provide the exact \field{hdr_len} value. Device benefits from knowing the exact header length.@@ -3140,6 +3144,7 @@ \subsubsection{Feature bit requirements}\label{sec:Device Types / Network Device \item[VIRTIO_NET_F_NOTF_COAL] Requires VIRTIO_NET_F_CTRL_VQ. \item[VIRTIO_NET_F_RSC_EXT] Requires VIRTIO_NET_F_HOST_TSO4 or VIRTIO_NET_F_HOST_TSO6. \item[VIRTIO_NET_F_RSS] Requires VIRTIO_NET_F_CTRL_VQ. +\item[VIRTIO_NET_F_HASH_TUNNEL] Requires VIRTIO_NET_F_CTRL_VQ. \end{description}Quoted 8 lines
content.tex \subsection{Device Operation}\label{sec:Device Types / Network Device / Device O · line 3394
Quoted 7 lines
content.tex +1
-1\subsubsection{Legacy Interface: Feature bits}\label{sec:Device Types / Network Device / Feature bits / Legacy Interface: Feature bits}@@ -3386,7 +3391,8 @@ \subsection{Device Operation}\label{sec:Device Types / Network Device / Device O le16 num_buffers; le32 hash_value; (Only if VIRTIO_NET_F_HASH_REPORT negotiated) le16 hash_report; (Only if VIRTIO_NET_F_HASH_REPORT negotiated) - le16 padding_reserved; (Only if VIRTIO_NET_F_HASH_REPORT negotiated) + u8 hash_report_tunnel; (Only if VIRTIO_NET_F_HASH_REPORT negotiated, only valid of VIRTIO_NET_F_HASH_TUNNEL negotiated)of -> if?
Sorry for the late reply.
It's Cornelia's suggestion, and 'of' seems to work fine.Quoted 1 line
also let's add: otherwise reserved?
Sure.
Quoted 34 lines
Quoted 34 lines
content.tex +22
-1+ u8 padding_reserved; (Only if VIRTIO_NET_F_HASH_REPORT negotiated) }; \end{lstlisting}@@ -3837,7 +3843,8 @@ \subsubsection{Processing of Incoming Packets}\label{sec:Device Types / Network A device attempts to calculate a per-packet hash in the following cases: \begin{itemize} \item The feature VIRTIO_NET_F_RSS was negotiated. The device uses the hash to determine the receive virtqueue to place incoming packets. -\item The feature VIRTIO_NET_F_HASH_REPORT was negotiated. The device reports the hash value and the hash type with the packet. +\item The feature VIRTIO_NET_F_HASH_REPORT was negotiated. The device reports the hash value and the hash type. If additionally +VIRTIO_NET_F_HASH_TUNNEL was negotiated, the device reports the encapsulation type as well. \end{itemize} If the feature VIRTIO_NET_F_RSS was negotiated:@@ -3863,8 +3870,36 @@ \subsubsection{Processing of Incoming Packets}\label{sec:Device Types / Network \ref{sec:Device Types / Network Device / Device Operation / Control Virtqueue / Automatic receive steering in multiqueue mode / Hash calculation}. \end{itemize} +\subparagraph{Tunnel/Encapsulated packet} +\label{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / Tunnel/Encapsulated packet} +A tunnel packet is encapsulated from the original packet based on the tunneling +protocol (only a single level of encapsulation is currently supported). The +encapsulated packet contains an outer header and an inner header, and the device +calculates the hash over either the inner header or the outer header. + +When the feature VIRTIO_NET_F_HASH_TUNNEL is negotiated and the corresponding +encapsulation type is set in \field{hash_types}, the hash for a specific type of +encapsulated packet is calculated over the inner as opposed to outer header. +Supported encapsulation types are listed in \ref{sec:Device Types / Network Device / +Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / +Supported/enabled hash types}. + +If both VIRTIO_NET_F_HASH_REPORT and VIRTIO_NET_F_HASH_TUNNEL are negotiated, +the device can support inner hash calculation for \hyperref[intro:GRE]{[GRE]}, +\hyperref[intro:VXLAN]{[VXLAN]} and \hyperref[intro:GENEVE]{[GENEVE]} +encapsulated packets, and the corresponding encapsulation type in \field{hash_types} +is VIRTIO_NET_HASH_TYPE_{GRE, VXLAN, GENEVE}_INNER respectively. The value inQuoted 9 lines
content.tex \subsubsection{Processing of Incoming Packets}\label{sec:Device Types / Network · line 3897
Quoted 6 lines
content.tex +6
-0+\field{hash_report_tunnel} is VIRTIO_NET_HASH_REPORT_{NONE, GRE, VXLAN, GENEVE} respectively. + +If VIRTIO_NET_F_HASH_REPORT is negotiated but VIRTIO_NET_F_HASH_TUNNEL is not +negotiated, the device calculates the hash over the outer header, and +\field{hash_report} reports the hash type. \field{hash_report_tunnel} +is no longer valid.Does this mean that VIRTIO_NET_HASH_TYPE_{GRE, VXLAN, GENEVE}_INNER must not be set without
VIRTIO_NET_F_HASH_TUNNEL?Yes.
Quoted 4 lines
If yes then we can add conformance statements to this end and then drop the
"If the featureVIRTIO_NET_F_HASH_TUNNELis negotiated" all over the place just relying on hash type being set instead.Please see the response below.
Quoted 7 lines
Also, note that it has to be ok for device to expose hash types without negotiating
VIRTIO_NET_F_HASH_TUNNELsince they are in config space.
Idea: do we want to decouple these completely? haveVIRTIO_NET_F_HASH_TUNNELjust addhash_report_tunneland have hash types speak for themselves? It makes things nicely orthogonal in that if one does not care about knowing encapsulation type (e.g. for RSS) one can disableVIRTIO_NET_F_HASH_TUNNEL.This seems to have the following 3 problems:
1. This will break the specification. As described in the specification, any field
in the configuration space only exists when there is a corresponding feature bit:
"Device configuration space is generally used for rarely-changing or initialization-time parameters. Where configuration fields are optional, their existence is indicated by feature bits: ", which also applies tosupported_hash_types.2. If I'm not wrong, this seems to make the encapsulation hash types dependent on
VIRTIO_NET_F_RSS, can the driver readsupported_hash_types(including encapsulation hash types) ifVIRTIO_NET_F_RSSis not negotiated? But that shouldn't be the case, we only use encapsulation hash types to provide guidance for hash calculation, but not necessarily use hash values to select receive queues (unlessVIRTIO_NET_F_RSSis negotiated).3. If all
supported_hash_typesspeak for themselves, why do we need theVIRTIO_NET_F_HASH_TUNNELfeature, because as long as the encapsulation hash types exist andVIRTIO_NET_F_HASH_REPORTis negotiated,hash_report_tunnelexists correspondingly. "haveVIRTIO_NET_F_HASH_TUNNELjust addhash_report_tunnel"? It seems likehash_report_tunnelwill only work if the encapsulation hash types are present.Quoted 6 lines
So, if I'm not wrong, the reasonable logic should be that
VIRTIO_NET_F_HASH_TUNNELis only used to declare that the device supports inner header hash and let the corresponding encapsulation hash type exist, whenVIRTIO_NET_F_RSSis negotiated, we can use this hash value to select the receive queue. Considering the migration at the same time, if the dst device does not negotiateVIRTIO_NET_F_HASH_TUNNEL, the migration will fail; otherwise, use encapsulation hash types to determine whether the migration can succeed.I think it might be better to have consistency:
1) If
VIRTIO_NET_F_HASH_TUNNELintroduces a new field in vnet header, is it better to have a new config filed in the config space?or
Quoted 8 lines
Quoted 4 lines
2) If
VIRTIO_NET_F_HASH_TUNNELdoesn't introduce new config file, should we try to reusehash_report?1) seems better and cleaner to me.
Sorry for the late reply due to vacation.
Good point, it's clearer to use the
VIRTIO_NET_F_HASH_TUNNELfeature to control all information related to inner header hash.
Hi, Parav.
Do you think we need both hash_types and hash_tunnel_types?
Thanks.
Quoted 253 lines
Quoted 38 lines
More below
Quoted 23 lines
Thanks.
Quoted 21 lines
Objections? Jason?
Quoted 18 lines
content.tex +10
-0+ \subparagraph{Supported/enabled hash types} \label{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / Supported/enabled hash types} +This paragraph relies on definitions from \hyperref[intro:IP]{[IP]}, +\hyperref[intro:UDP]{[UDP]} and \hyperref[intro:TCP]{[TCP]}. Hash types applicable for IPv4 packets: \begin{lstlisting} #define VIRTIO_NET_HASH_TYPE_IPv4 (1 << 0)@@ -3884,6 +3919,22 @@ \subsubsection{Processing of Incoming Packets}\label{sec:Device Types / Network #define VIRTIO_NET_HASH_TYPE_UDP_EX (1 << 8) \end{lstlisting} +If the feature VIRTIO_NET_F_HASH_TUNNEL is negotiated, the encapsulation +hash type below indicates that the hash is calculated over the inner +header of the encapsulated packet: +Hash type applicable for inner payload of the gre-encapsulated packet +\begin{lstlisting} +#define VIRTIO_NET_HASH_TYPE_GRE_INNER (1 << 9) +\end{lstlisting}Quoted 8 lines
Quoted 8 lines
content.tex \subsubsection{Processing of Incoming Packets}\label{sec:Device Types / Network · line 3936
Quoted 8 lines
content.tex +8
-0+Hash type applicable for inner payload of the vxlan-encapsulated packet +\begin{lstlisting} +#define VIRTIO_NET_HASH_TYPE_VXLAN_INNER (1 << 10) +\end{lstlisting} +Hash type applicable for inner payload of the geneve-encapsulated packet +\begin{lstlisting} +#define VIRTIO_NET_HASH_TYPE_GENEVE_INNER (1 << 11) +\end{lstlisting}Just notice that this seems to break the equation:
Quoted 4 lines
VIRTIO_NET_HASH_TYPE_XXX= 1 « (VIRTIO_NET_HASH_REPORT_XXX- 1)?
Yes, we have discussed it here https://lists.oasis-open.org/archives/virtio-dev/202211/msg00189.html , as you suggested above, it seems better to use the new config space field.
Thanks.
Quoted 200 lines
Thanks
Quoted 188 lines
Quoted 188 lines
Quoted 181 lines
content.tex +131
-4+ \subparagraph{IPv4 packets} \label{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / IPv4 packets} The device calculates the hash on IPv4 packets according to 'Enabled hash types' bitmask as follows:@@ -3975,15 +4026,125 @@ \subsubsection{Processing of Incoming Packets}\label{sec:Device Types / Network (see \ref{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / IPv6 packets without extension header}). \end{itemize} +\subparagraph{Inner payload of an encapsulated packet} +\label{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / Inner payload of the encapsulated packet} +If the feature VIRTIO_NET_F_HASH_TUNNEL is negotiated and the corresponding +encapsulation hash type is set in \field{hash_types}, the device calculates the +inner header hash on an encapsulated packet (See \ref{sec:Device Types +/ Network Device / Device Operation / Processing of Incoming Packets / +Hash calculation for incoming packets / Tunnel/Encapsulated packet}) as follows: + +The device calculates the hash on the inner IPv4 packet of an encapsulated packet +according to 'Enabled hash types' bitmask as follows: +\begin{itemize} + \item If VIRTIO_NET_HASH_TYPE_TCPv4 is set and the encapsulated packet has an inner + TCPv4 header, the hash is calculated over the following fields: + \begin{itemsize} + \item inner Source IP address + \item inner Destination IP address + \item inner Source TCP port + \item inner Destination TCP port + \end{itemsize} + \item Else if VIRTIO_NET_HASH_TYPE_UDPv4 is set and the encapsulated packet has an + inner UDPv4 header, the hash is calculated over the following fields: + \begin{itemsize} + \item inner Source IP address + \item inner Destination IP address + \item inner Source UDP port + \item inner Destination UDP port + \end{itemize} + \item Else if VIRTIO_NET_HASH_TYPE_IPv4 is set, the hash is calculated over the + following fields: + \begin{itemsize} + \item inner Source IP address + \item inner Destination IP address + \end{itemsize} + \item Else the device does not calculate the hash +\end{itemize} + +The device calculates the hash on the inner IPv6 packet without an extension header +of an encapsulated packet according to 'Enabled hash types' bitmask as follows: +\begin{itemize} + \item If VIRTIO_NET_HASH_TYPE_TCPv6 is set and the encapsulated packet has an inner + TCPv6 header, the hash is calculated over the following fields: + \begin{itemsize} + \item inner Source IPv6 address + \item inner Destination IPv6 address + \item inner Source TCP port + \item inner Destination TCP port + \end{itemsize} + \item Else if VIRTIO_NET_HASH_TYPE_UDPv6 is set and the encapsulated packet has an + inner UDPv6 header, the hash is calculated over the following fields: + \begin{itemsize} + \item inner Source IPv6 address + \item inner Destination IPv6 address + \item inner Source UDP port + \item inner Destination UDP port + \end{itemize} + \item Else if VIRTIO_NET_HASH_TYPE_IPv6 is set, the hash is calculated over the + following fields: + \begin{itemsize} + \item inner Source IPv6 address + \item inner Destination IPv6 address + \end{itemsize} + \item Else the device does not calculate the hash +\end{itemize} + +The device calculates the hash on the inner IPv6 packet with an extension header +of an encapsulated packet according to 'Enabled hash types' bitmask as follows: +\begin{itemsize} + \item If VIRTIO_NET_HASH_TYPE_TCP_EX is set and the encapsulated packet has an inner + TCPv6 header, the hash is calculated over the following fields: + \begin{itemize} + \item Home address from the home address option in the inner IPv6 destination + options header. If the inner extension header is not present, use the + inner Source IPv6 address. + \item IPv6 address that is contained in the Routing-Header-Type-2 from the + associated inner extension header. If the inner extension header is not + present, use the inner Destination IPv6 address. + \item inner Source TCP port + \item inner Destination TCP port + \end{itemize} + \item Else if VIRTIO_NET_HASH_TYPE_UDP_EX is set and the encapsulated packet has an inner + UDPv6 header, the hash is calculated over the following fields: + \begin{itemsize} + \item Home address from the home address option in the inner IPv6 destination + options header. If the inner extension header is not present, use the + inner Source IPv6 address. + \item IPv6 address that is contained in the Routing-Header-Type-2 from the + associated inner extension header. If the inner extension header is not + present, use the inner Destination IPv6 address. + \item inner Source UDP port + \item inner Destination UDP port + \end{itemize} + \item Else if VIRTIO_NET_HASH_TYPE_IP_EX is set, the hash is calculated over the + following fields: + \begin{itemsize} + \item Home address from the home address option in the inner IPv6 destination + options header. If the inner extension header is not present, use the + inner Source IPv6 address. + \item IPv6 address that is contained in the Routing-Header-Type-2 from the + associated inner extension header. If the inner extension header is not + present, use the inner Destination IPv6 address. + \end{itemize} + \item Else skip the inner IPv6 extension header and calculate the inner header hash as + defined for an encapsulated packet whose inner payload is an IPv6 packet without + an extension header. +\end{itemsize} + \paragraph{Hash reporting for incoming packets} \label{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash reporting for incoming packets} -If VIRTIO_NET_F_HASH_REPORT was negotiated and - the device has calculated the hash for the packet, the device fills \field{hash_report} with the report type of calculated hash -and \field{hash_value} with the value of calculated hash. +If VIRTIO_NET_F_HASH_REPORT was negotiated and the device has calculated the +hash for the packet, the device fills \field{hash_report} with the report type +of calculated hash, and \field{hash_value} with the value of calculated hash. +Also, if VIRTIO_NET_F_HASH_TUNNEL was negotiated, the device needs to fill +\field{hash_report_tunnel} with the report type of the encapsulated packet, and +it is set to VIRTIO_NET_HASH_REPORT_TUNNEL_NONE for the unencapsulated packet. If VIRTIO_NET_F_HASH_REPORT was negotiated but due to any reason the -hash was not calculated, the device sets \field{hash_report} to VIRTIO_NET_HASH_REPORT_NONE. +hash was not calculated, the device sets \field{hash_report} to VIRTIO_NET_HASH_REPORT_NONE, +and sets \field{hash_report_tunnel} to VIRTIO_NET_HASH_REPORT_TUNNEL_NONE. Possible values that the device can report in \field{hash_report} are defined below. They correspond to supported hash types defined in@@ -4005,6 +4166,22 @@ \subsubsection{Processing of Incoming Packets}\label{sec:Device Types / Network #define VIRTIO_NET_HASH_REPORT_UDPv6_EX 9 \end{lstlisting} +If \field{hash_report} differs from VIRTIO_NET_HASH_REPORT_NONE, +\field{hash_report_tunnel} can report the type of the encapsulated +packet to the driver over the inner header hash calculation. +Possible values that the device can report in \field{hash_report_tunnel} +are defined below: + +\begin{lstlisting} +#define VIRTIO_NET_HASH_REPORT_TUNNEL_NONE 0 +#define VIRTIO_NET_HASH_REPORT_GRE 1 +#define VIRTIO_NET_HASH_REPORT_VXLAN 2 +#define VIRTIO_NET_HASH_REPORT_GENEVE 3 +\end{lstlisting} + +They correspond to supported hash types defined in +\ref{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / Supported/enabled hash types}. + \subsubsection{Control Virtqueue}\label{sec:Device Types / Network Device / Device Operation / Control Virtqueue} The driver uses the control virtqueue (if VIRTIO_NET_F_CTRL_VQ isintroduction.tex +19
-0@@ -98,6 +98,25 @@ \section{Normative References}\label{sec:Normative References} \phantomsection\label{intro:SEC1}\textbf{[SEC1]} & Standards for Efficient Cryptography Group(SECG), ``SEC1: Elliptic Cureve Cryptography'', Version 1.0, September 2000. \newline\url{https://www.secg.org/sec1-v2.pdf}\\ + \phantomsection\label{intro:GRE}\textbf{[GRE]} & + Generic Routing Encapsulation + \newline\url{https://datatracker.ietf.org/doc/rfc2784/}\\ + \phantomsection\label{intro:VXLAN}\textbf{[VXLAN]} & + Virtual eXtensible Local Area Network + \newline\url{https://datatracker.ietf.org/doc/rfc7348/}\\ + \phantomsection\label{intro:GENEVE}\textbf{[GENEVE]} & + Generic Network Virtualization Encapsulation + \newline\url{https://datatracker.ietf.org/doc/rfc8926/}\\ + \phantomsection\label{intro:IP}\textbf{[IP]} & + INTERNET PROTOCOL + \newline\url{https://www.rfc-editor.org/rfc/rfc791}\\ + \phantomsection\label{intro:UDP}\textbf{[UDP]} & + User Datagram Protocol + \newline\url{https://www.rfc-editor.org/rfc/rfc768}\\ + \phantomsection\label{intro:TCP}\textbf{[TCP]} & + TRANSMISSION CONTROL PROTOCOL + \newline\url{https://www.rfc-editor.org/rfc/rfc793}\\ + \end{longtable}Signature · 1 line
2.19.1.6.gb485710bThis publicly archived list offers a means to provide input to the OASIS Virtual I/O Device (VIRTIO) TC.
In order to verify user consent to the Feedback License terms and to minimize spam in the list archive, subscription is required before posting.
Quoted 8 lines
Quoted 8 lines
Subscribe: [email protected]
Unsubscribe: [email protected]
List help: [email protected]
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/Quoted 3 lines
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Quoted 272 lines
From: Heng Qi <[email protected]>
Sent: Tuesday, February 7, 2023 9:31 PMQuoted 7 lines · Heng Qi
Quoted 6 lines · Jason Wang
Quoted 5 lines · Heng Qi
Quoted 4 lines · Michael S. Tsirkin
Quoted 3 lines · Heng Qi
If the tunnel is used to encapsulate the packets, the hash calculated using the outer header of the receive packets is always fixed for the same flow packets, i.e. they will be steered to the same
receive queue.
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
We add a feature bit
VIRTIO_NET_F_HASH_TUNNELand relatedbitmasks
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
in \field{hash_types}, which instructs the device to calculate the hash using the inner headers of tunnel-encapsulated packets. Besides, values in \field{hash_report_tunnel} are added to report tunnel
types.
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Fixes:
https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2F
github.com%2Foasis-tcs%2Fvirtio-spec%2Fissues%2F151&data=05%7C01%7
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Cparav%40nvidia.com%7C12be6c51b3c04860c01608db097c796a%7C43083d15 7
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
27340c1b7db39efd9ccc17a%7C0%7C0%7C638114202991221369%7CUnknown %7CT
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
WFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJ
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
XVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=6feFgy94LMCYfG0NKPlHCr1AvkC OH4v9
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
%2BnLomXklNqs%3D&reserved=0
Reviewed-by Jason Wang <[email protected]>Signed-off-by Heng Qi <[email protected]>Signed-off-by Xuan Zhuo <[email protected]>---
v6:
1. Modify the wording of some sentences for clarity. @Michael S.Tsirkin
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
2. Fix some syntax issues. @Michael S. Tsirkin
v5:
1. Fix some syntax and capitalization issues. @Michael S. Tsirkin
2. Use encapsulated/encaptulation uniformly. @Michael S. Tsirkin
3. Move the links to introduction section. @Michael S. Tsirkin
4. Clarify some sentences. @Michael S. TsirkinQuoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
v4:
1. Clarify some paragraphs. @Cornelia Huck
2. Fix the u8 type. @Cornelia Huckv3:
1. RenameVIRTIO_NET_F_HASH_GRE_VXLAN_GENEVE_INNERto
VIRTIO_NET_F_HASH_TUNNEL. @Jason WangQuoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
2. Make things clearer. @Jason Wang @Michael S. Tsirkin
3. Keep the possibility to use inner hash for automatic receive steering.@Jason Wang
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
4. Add the "Tunnel packet" paragraph to avoid repeating the GRE
etc. many times. @Michael S. Tsirkinv2:
1. Add a feature bit for GRE/VXLAN/GENEVE inner hash. @Jason Wang
2. Chang \field{hash_tunnel} to \field{hash_report_tunnel}.
@Jason Wang, @Michael S. TsirkinQuoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
v1:
1. Remove the patch for the bitmask fix. @Michael S. Tsirkin
2. Clarify some paragraphs. @Jason Wang
3. Add \field{hash_tunnel} andVIRTIO_NET_HASH_REPORT_GRE.@Yuri
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Benditovich
content.tex | 191+++++++++++++++++++++++++++++++++++++++++++++--
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
content.tex \subsection{Feature bits}\label{sec:Device
Quoted 8 lines
introduction.tex | 19 +++++ 2 files changed, 203 insertions(+), 7 deletions(-)content.tex index e863709..7845f6c renamed from content.tex +0
-0100644
content.tex +0
-0@@ -3084,6 +3084,9 @@ \subsection{Feature bits}\label{sec:DeviceTypes / Network Device / Feature bits
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
content.tex \subsection{Feature bits}\label{sec:Device · line 3084
Quoted 1 line
content.tex +0
-0\item[VIRTIO_NET_F_CTRL_MAC_ADDR(23)] Set MAC address throughcontrol
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
content.tex \subsection{Feature bits}\label{sec:Device Types / Network Device / Feature bits
Quoted 3 lines
content.tex +2
-0channel. +\item[VIRTIO_NET_F_HASH_TUNNEL(52)] Device supports inner + header hash for GRE, VXLAN and GENEVE tunnel-encapsulatedpackets.
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
content.tex \subsection{Feature bits}\label{sec:Device · line 3089
Quoted 2 lines
content.tex +1
-0+ \item[VIRTIO_NET_F_NOTF_COAL(53)] Device supports notificationscoalescing.
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
content.tex \subsection{Feature bits}\label{sec:Device · line 3090
Quoted 1 line
content.tex +0
-0\item[VIRTIO_NET_F_GUEST_USO4 (54)] Driver can receive USOv4packets.
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
content.tex \subsection{Feature bits}\label{sec:Device
Quoted 1 line
content.tex +0
-0@@ -3095,7 +3098,8 @@ \subsection{Feature bits}\label{sec:DeviceTypes / Network Device / Feature bits
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
content.tex \subsection{Feature bits}\label{sec:Device · line 3098
Quoted 1 line
content.tex +0
-0to several segments when each of these smaller packets has UDPheader.
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
content.tex \subsection{Feature bits}\label{sec:Device · line 3099
Quoted 1 line
content.tex +0
-0\item[VIRTIO_NET_F_HASH_REPORT(57)] Device can report per-packethash
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
content.tex \subsection{Feature bits}\label{sec:Device · line 3100
Quoted 2 lines
content.tex +1
-1- value and a type of calculated hash. + value, a type of calculated hash, and, if
VIRTIO_NET_F_HASH_TUNNELQuoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
content.tex \subsection{Feature bits}\label{sec:Device · line 3102
Quoted 2 lines
content.tex +1
-0+ is negotiated, an encapsulation packet type. \item[VIRTIO_NET_F_GUEST_HDRLEN(59)] Driver can provide the exact\field{hdr_len}
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
content.tex \subsubsection{Feature bit
Quoted 2 lines
content.tex +0
-0value. Device benefits from knowing the exact header length.@@ -3140,6 +3144,7 @@ \subsubsection{Feature bitrequirements}\label{sec:Device Types / Network Device
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
content.tex \subsubsection{Feature bit · line 3145
Quoted 2 lines
content.tex +0
-0\item[VIRTIO_NET_F_NOTF_COAL] Requires VIRTIO_NET_F_CTRL_VQ. \item[VIRTIO_NET_F_RSC_EXT] Requires VIRTIO_NET_F_HOST_TSO4 or
VIRTIO_NET_F_HOST_TSO6.Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
content.tex \subsubsection{Feature bit · line 3147
Quoted 2 lines
content.tex +1
-0\item[VIRTIO_NET_F_RSS] Requires VIRTIO_NET_F_CTRL_VQ. +\item[VIRTIO_NET_F_HASH_TUNNEL] Requires
VIRTIO_NET_F_CTRL_VQ.Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
content.tex +0
-0\end{description} \subsubsection{Legacy Interface: Feature bits}\label{sec:Device Types / Network Device / Feature bits / Legacy Interface: Feature bits}(quoted diff) +0
-0@@ -3386,7 +3391,8 @@ \subsection{Device Operation}\label{sec:DeviceTypes / Network Device / Device O
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
le16
num_buffers;
le32hash_value; (Only ifVIRTIO_NET_F_HASH_REPORTnegotiated)
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
le16
hash_report; (Only ifVIRTIO_NET_F_HASH_REPORTnegotiated)
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
- le16
padding_reserved; (Only ifVIRTIO_NET_F_HASH_REPORTnegotiated)
Quoted 10 lines
Quoted 10 lines
Quoted 10 lines
Quoted 3 lines
Quoted 2 lines
+ u8
hash_report_tunnel; (Only ifVIRTIO_NET_F_HASH_REPORT
+ negotiated, only valid ofVIRTIO_NET_F_HASH_TUNNELnegotiated)of -> if?
Sorry for the late reply.
It's Cornelia's suggestion, and 'of' seems to work fine.Quoted 1 line
also let's add: otherwise reserved?
Sure.
Quoted 1 line
Quoted 1 line
+ u8
padding_reserved; (Only ifVIRTIO_NET_F_HASH_REPORTnegotiated)
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
content.tex \subsubsection{Processing of Incoming
Quoted 3 lines
}; \end{lstlisting}content.tex +0
-0@@ -3837,7 +3843,8 @@ \subsubsection{Processing of IncomingPackets}\label{sec:Device Types / Network
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
content.tex \subsubsection{Processing of Incoming Packets}\label{sec:Device Types / Network
Quoted 3 lines
content.tex +0
-0A device attempts to calculate a per-packet hash in the following cases: \begin{itemize} \item The feature VIRTIO_NET_F_RSS was negotiated. The device usesthe hash to determine the receive virtqueue to place incoming packets.
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
content.tex \subsubsection{Processing of Incoming · line 3840
Quoted 1 line
content.tex +0
-1-\item The feature VIRTIO_NET_F_HASH_REPORT was negotiated. Thedevice reports the hash value and the hash type with the packet.
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
content.tex \subsubsection{Processing of Incoming · line 3847
Quoted 2 lines
content.tex +2
-0+\item The feature VIRTIO_NET_F_HASH_REPORT was negotiated. The +device reports the hash value and the hash type. If additionally
VIRTIO_NET_F_HASH_TUNNELwas negotiated, the device reports the encapsulation type as well.Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
content.tex \subsubsection{Processing of Incoming
Quoted 3 lines
content.tex +0
-0\end{itemize} If the feature VIRTIO_NET_F_RSS was negotiated:@@ -3863,8 +3870,36 @@ \subsubsection{Processing of IncomingPackets}\label{sec:Device Types / Network
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
content.tex \subsubsection{Processing of Incoming · line 3870
Quoted 1 line
content.tex +0
-0\ref{sec:Device Types / Network Device / Device Operation / ControlVirtqueue / Automatic receive steering in multiqueue mode / Hash calculation}.
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
content.tex \subsubsection{Processing of Incoming · line 3877
Quoted 7 lines
content.tex +6
-0\end{itemize} +\subparagraph{Tunnel/Encapsulated packet} \label{sec:Device Types +/ Network Device / Device Operation / Processing of Incoming +Packets / Hash calculation for incoming packets / +Tunnel/Encapsulated packet} A tunnel packet is encapsulated from +the original packet based on the tunneling protocol (only a +single level of encapsulation is currently supported). The encapsulatedpacket contains an outer header and an inner header, and the device calculates the hash over either the inner header or the outer header.
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
content.tex \subsubsection{Processing of Incoming · line 3881
Quoted 4 lines
content.tex +4
-0+ +When the feature VIRTIO_NET_F_HASH_TUNNEL is negotiated and the +corresponding encapsulation type is set in \field{hash_types}, +the hash for a specific type of encapsulated packet is calculated overthe inner as opposed to outer header.
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
content.tex \subsubsection{Processing of Incoming · line 3884
Quoted 3 lines
content.tex +3
-0+Supported encapsulation types are listed in \ref{sec:Device Types +/ Network Device / Device Operation / Processing of Incoming +Packets / Hash calculation for incoming packets / Supported/enabledhash types}.
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
content.tex \subsubsection{Processing of Incoming · line 3886
Quoted 2 lines
content.tex +2
-0+ +If both VIRTIO_NET_F_HASH_REPORT and
VIRTIO_NET_F_HASH_TUNNELareQuoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
content.tex \subsubsection{Processing of Incoming · line 3891
Quoted 5 lines
content.tex +5
-0+negotiated, the device can support inner hash calculation for +\hyperref[intro:GRE]{[GRE]}, \hyperref[intro:VXLAN]{[VXLAN]} and +\hyperref[intro:GENEVE]{[GENEVE]} encapsulated packets, and the +corresponding encapsulation type in \field{hash_types} is +VIRTIO_NET_HASH_TYPE_{GRE, VXLAN, GENEVE}_INNER respectively.The value in \field{hash_report_tunnel} is VIRTIO_NET_HASH_REPORT_{NONE, GRE, VXLAN, GENEVE} respectively.
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
content.tex \subsubsection{Processing of Incoming · line 3896
Quoted 5 lines
content.tex +5
-0+ +If VIRTIO_NET_F_HASH_REPORT is negotiated but +VIRTIO_NET_F_HASH_TUNNEL is not negotiated, the device calculates +the hash over the outer header, and \field{hash_report} reports +the hash type. \field{hash_report_tunnel} is no longer valid.Does this mean that VIRTIO_NET_HASH_TYPE_{GRE, VXLAN,
GENEVE}_INNER
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 2 lines
must not be set without
VIRTIO_NET_F_HASH_TUNNEL?Yes.
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 3 lines
If yes then we can add conformance statements to this end and then drop the "If the feature
VIRTIO_NET_F_HASH_TUNNELis negotiated" all over the place just relying on hash type being set instead.Please see the response below.
Quoted 3 lines
Also, note that it has to be ok for device to expose hash types without negotiating
VIRTIO_NET_F_HASH_TUNNELsince they are in config space.
Idea: do we want to decouple these completely? have
VIRTIO_NET_F_HASH_TUNNELQuoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 4 lines
just add
hash_report_tunneland have hash types speak for themselves? It makes things nicely orthogonal in that if one does not care about knowing encapsulation type (e.g. for RSS) one can disableVIRTIO_NET_F_HASH_TUNNEL.This seems to have the following 3 problems:
1. This will break the specification. As described in the specification, any
field
Quoted 1 line
Quoted 1 line
Quoted 1 line
in the configuration space only exists when there is a corresponding
feature bit:
Quoted 1 line
Quoted 1 line
Quoted 1 line
"Device configuration space is generally used for rarely-changing or
initialization-time parameters.
Quoted 1 line
Quoted 1 line
Quoted 1 line
Where configuration fields are optional, their existence is indicated by
feature bits: ",
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
which also applies to
supported_hash_types.2. If I'm not wrong, this seems to make the encapsulation hash types
dependent on
Quoted 1 line
Quoted 1 line
Quoted 1 line
VIRTIO_NET_F_RSS, can the driver readsupported_hash_types(including encapsulation
Quoted 1 line
Quoted 1 line
Quoted 1 line
hash types) if
VIRTIO_NET_F_RSSis not negotiated? But that shouldn't bethe case,
Quoted 1 line
Quoted 1 line
Quoted 1 line
we only use encapsulation hash types to provide guidance for hash
calculation, but not
Quoted 1 line
Quoted 1 line
Quoted 1 line
necessarily use hash values to select receive queues (unless
VIRTIO_NET_F_RSSis negotiated).Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
3. If all
supported_hash_typesspeak for themselves, why do we need the
VIRTIO_NET_F_HASH_TUNNELQuoted 1 line
Quoted 1 line
Quoted 1 line
feature, because as long as the encapsulation hash types exist and
VIRTIO_NET_F_HASH_REPORTisQuoted 1 line
Quoted 1 line
Quoted 1 line
negotiated,
hash_report_tunnelexists correspondingly. "have
VIRTIO_NET_F_HASH_TUNNELjust addQuoted 1 line
Quoted 1 line
Quoted 1 line
hash_report_tunnel"? It seems likehash_report_tunnelwill only work ifthe encapsulation hash
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
types are present.
So, if I'm not wrong, the reasonable logic should be that
VIRTIO_NET_F_HASH_TUNNELis only usedQuoted 1 line
Quoted 1 line
Quoted 1 line
to declare that the device supports inner header hash and let the
corresponding encapsulation
Quoted 1 line
Quoted 1 line
Quoted 1 line
hash type exist, when
VIRTIO_NET_F_RSSis negotiated, we can use thishash value to select the
Quoted 1 line
Quoted 1 line
Quoted 1 line
receive queue. Considering the migration at the same time, if the dst device
does not negotiate
Quoted 1 line
Quoted 1 line
Quoted 1 line
VIRTIO_NET_F_HASH_TUNNEL, the migration will fail; otherwise, useencapsulation hash types to
Quoted 13 lines
Quoted 13 lines
Quoted 1 line
determine whether the migration can succeed.
I think it might be better to have consistency:
1) If
VIRTIO_NET_F_HASH_TUNNELintroduces a new field in vnet header, is it better to have a new config filed in the config space?or
2) If
VIRTIO_NET_F_HASH_TUNNELdoesn't introduce new config file, should we try to reusehash_report?1) seems better and cleaner to me.
Quoted 8 lines
Quoted 4 lines
Sorry for the late reply due to vacation.
Good point, it's clearer to use the
VIRTIO_NET_F_HASH_TUNNELfeature to control all information related to inner header hash.Hi, Parav.
Do you think we need both
hash_typesandhash_tunnel_types?
In struct virtio_net_config we need two fields.
a. supported_hash_types (already exists)
b. supported_hash_tunnel_type -> bitmap indicating for which outer headers, inner hash calculation is supported.
In struct virtio_net_hdr we need two fields.
a. hash_report (already exists)
b. hash_tunnel_type 8 bits -> absolute value indicating which outer header exists when inner header hash calculated. You already have it in your patch named as hash_report_tunnel.
May be better to name as hash_report_tunnel_type to make it clearer that its type.
Quoted 265 lines · Parav Pandit
Quoted 265 lines
From: Heng Qi <[email protected]>
Sent: Tuesday, February 7, 2023 9:31 PMQuoted 7 lines · Heng Qi
Quoted 6 lines · Jason Wang
Quoted 5 lines · Heng Qi
Quoted 4 lines · Michael S. Tsirkin
Quoted 3 lines · Heng Qi
If the tunnel is used to encapsulate the packets, the hash calculated using the outer header of the receive packets is always fixed for the same flow packets, i.e. they will be steered to the same
receive queue.
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
We add a feature bit
VIRTIO_NET_F_HASH_TUNNELand relatedbitmasks
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
in \field{hash_types}, which instructs the device to calculate the hash using the inner headers of tunnel-encapsulated packets. Besides, values in \field{hash_report_tunnel} are added to report tunnel
types.
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Fixes:
https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2F
github.com%2Foasis-tcs%2Fvirtio-spec%2Fissues%2F151&data=05%7C01%7 Cparav%40nvidia.com%7C12be6c51b3c04860c01608db097c796a%7C43083d15 7 27340c1b7db39efd9ccc17a%7C0%7C0%7C638114202991221369%7CUnknown %7CT WFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJ XVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=6feFgy94LMCYfG0NKPlHCr1AvkC OH4v9
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
%2BnLomXklNqs%3D&reserved=0
Reviewed-by Jason Wang <[email protected]>Signed-off-by Heng Qi <[email protected]>Signed-off-by Xuan Zhuo <[email protected]>---
v6:
1. Modify the wording of some sentences for clarity. @Michael S.Tsirkin
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
2. Fix some syntax issues. @Michael S. Tsirkin
v5:
1. Fix some syntax and capitalization issues. @Michael S. Tsirkin
2. Use encapsulated/encaptulation uniformly. @Michael S. Tsirkin
3. Move the links to introduction section. @Michael S. Tsirkin
4. Clarify some sentences. @Michael S. TsirkinQuoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
v4:
1. Clarify some paragraphs. @Cornelia Huck
2. Fix the u8 type. @Cornelia Huckv3:
1. RenameVIRTIO_NET_F_HASH_GRE_VXLAN_GENEVE_INNERto
VIRTIO_NET_F_HASH_TUNNEL. @Jason WangQuoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
2. Make things clearer. @Jason Wang @Michael S. Tsirkin
3. Keep the possibility to use inner hash for automatic receive steering.@Jason Wang
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
4. Add the "Tunnel packet" paragraph to avoid repeating the GRE
etc. many times. @Michael S. Tsirkinv2:
1. Add a feature bit for GRE/VXLAN/GENEVE inner hash. @Jason Wang
2. Chang \field{hash_tunnel} to \field{hash_report_tunnel}.
@Jason Wang, @Michael S. TsirkinQuoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
v1:
1. Remove the patch for the bitmask fix. @Michael S. Tsirkin
2. Clarify some paragraphs. @Jason Wang
3. Add \field{hash_tunnel} andVIRTIO_NET_HASH_REPORT_GRE.@Yuri
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Benditovich
content.tex | 191+++++++++++++++++++++++++++++++++++++++++++++--
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
content.tex \subsection{Feature bits}\label{sec:Device
Quoted 8 lines
introduction.tex | 19 +++++ 2 files changed, 203 insertions(+), 7 deletions(-)content.tex index e863709..7845f6c renamed from content.tex +0
-0100644
content.tex +0
-0@@ -3084,6 +3084,9 @@ \subsection{Feature bits}\label{sec:DeviceTypes / Network Device / Feature bits
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
content.tex \subsection{Feature bits}\label{sec:Device · line 3084
Quoted 1 line
content.tex +0
-0\item[VIRTIO_NET_F_CTRL_MAC_ADDR(23)] Set MAC address throughcontrol
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
content.tex \subsection{Feature bits}\label{sec:Device · line 3087
Quoted 3 lines
content.tex +2
-0channel. +\item[VIRTIO_NET_F_HASH_TUNNEL(52)] Device supports inner + header hash for GRE, VXLAN and GENEVE tunnel-encapsulatedpackets.
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
content.tex \subsection{Feature bits}\label{sec:Device · line 3089
Quoted 2 lines
content.tex +1
-0+ \item[VIRTIO_NET_F_NOTF_COAL(53)] Device supports notificationscoalescing.
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
content.tex \subsection{Feature bits}\label{sec:Device · line 3090
Quoted 1 line
content.tex +0
-0\item[VIRTIO_NET_F_GUEST_USO4 (54)] Driver can receive USOv4packets.
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
content.tex \subsection{Feature bits}\label{sec:Device
Quoted 1 line
content.tex +0
-0@@ -3095,7 +3098,8 @@ \subsection{Feature bits}\label{sec:DeviceTypes / Network Device / Feature bits
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
content.tex \subsection{Feature bits}\label{sec:Device · line 3098
Quoted 1 line
content.tex +0
-0to several segments when each of these smaller packets has UDPheader.
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
content.tex \subsection{Feature bits}\label{sec:Device · line 3099
Quoted 1 line
content.tex +0
-0\item[VIRTIO_NET_F_HASH_REPORT(57)] Device can report per-packethash
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
content.tex \subsection{Feature bits}\label{sec:Device · line 3100
Quoted 2 lines
content.tex +1
-1- value and a type of calculated hash. + value, a type of calculated hash, and, if
VIRTIO_NET_F_HASH_TUNNELQuoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
content.tex \subsection{Feature bits}\label{sec:Device · line 3102
Quoted 2 lines
content.tex +1
-0+ is negotiated, an encapsulation packet type. \item[VIRTIO_NET_F_GUEST_HDRLEN(59)] Driver can provide the exact\field{hdr_len}
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
content.tex \subsubsection{Feature bit
Quoted 2 lines
content.tex +0
-0value. Device benefits from knowing the exact header length.@@ -3140,6 +3144,7 @@ \subsubsection{Feature bitrequirements}\label{sec:Device Types / Network Device
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
content.tex \subsubsection{Feature bit · line 3145
Quoted 2 lines
content.tex +0
-0\item[VIRTIO_NET_F_NOTF_COAL] Requires VIRTIO_NET_F_CTRL_VQ. \item[VIRTIO_NET_F_RSC_EXT] Requires VIRTIO_NET_F_HOST_TSO4 or
VIRTIO_NET_F_HOST_TSO6.Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
content.tex \subsubsection{Feature bit · line 3147
Quoted 2 lines
content.tex +1
-0\item[VIRTIO_NET_F_RSS] Requires VIRTIO_NET_F_CTRL_VQ. +\item[VIRTIO_NET_F_HASH_TUNNEL] Requires
VIRTIO_NET_F_CTRL_VQ.Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
content.tex +0
-0\end{description} \subsubsection{Legacy Interface: Feature bits}\label{sec:Device Types / Network Device / Feature bits / Legacy Interface: Feature bits}(quoted diff) +0
-0@@ -3386,7 +3391,8 @@ \subsection{Device Operation}\label{sec:DeviceTypes / Network Device / Device O
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
le16
num_buffers;
le32hash_value; (Only ifVIRTIO_NET_F_HASH_REPORTnegotiated)
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
le16
hash_report; (Only ifVIRTIO_NET_F_HASH_REPORTnegotiated)
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
- le16
padding_reserved; (Only ifVIRTIO_NET_F_HASH_REPORTnegotiated)
Quoted 10 lines
Quoted 10 lines
Quoted 10 lines
Quoted 3 lines
Quoted 2 lines
+ u8
hash_report_tunnel; (Only ifVIRTIO_NET_F_HASH_REPORT
+ negotiated, only valid ofVIRTIO_NET_F_HASH_TUNNELnegotiated)of -> if?
Sorry for the late reply.
It's Cornelia's suggestion, and 'of' seems to work fine.Quoted 1 line
also let's add: otherwise reserved?
Sure.
Quoted 1 line
Quoted 1 line
+ u8
padding_reserved; (Only ifVIRTIO_NET_F_HASH_REPORTnegotiated)
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
content.tex \subsubsection{Processing of Incoming
Quoted 3 lines
}; \end{lstlisting}content.tex +0
-0@@ -3837,7 +3843,8 @@ \subsubsection{Processing of IncomingPackets}\label{sec:Device Types / Network
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
content.tex \subsubsection{Processing of Incoming · line 3845
Quoted 3 lines
content.tex +0
-0A device attempts to calculate a per-packet hash in the following cases: \begin{itemize} \item The feature VIRTIO_NET_F_RSS was negotiated. The device usesthe hash to determine the receive virtqueue to place incoming packets.
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
content.tex \subsubsection{Processing of Incoming · line 3840
Quoted 1 line
content.tex +0
-1-\item The feature VIRTIO_NET_F_HASH_REPORT was negotiated. Thedevice reports the hash value and the hash type with the packet.
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
content.tex \subsubsection{Processing of Incoming · line 3847
Quoted 2 lines
content.tex +2
-0+\item The feature VIRTIO_NET_F_HASH_REPORT was negotiated. The +device reports the hash value and the hash type. If additionally
VIRTIO_NET_F_HASH_TUNNELwas negotiated, the device reports the encapsulation type as well.Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
content.tex \subsubsection{Processing of Incoming
Quoted 3 lines
content.tex +0
-0\end{itemize} If the feature VIRTIO_NET_F_RSS was negotiated:@@ -3863,8 +3870,36 @@ \subsubsection{Processing of IncomingPackets}\label{sec:Device Types / Network
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
content.tex \subsubsection{Processing of Incoming · line 3870
Quoted 1 line
content.tex +0
-0\ref{sec:Device Types / Network Device / Device Operation / ControlVirtqueue / Automatic receive steering in multiqueue mode / Hash calculation}.
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
content.tex \subsubsection{Processing of Incoming · line 3877
Quoted 7 lines
content.tex +6
-0\end{itemize} +\subparagraph{Tunnel/Encapsulated packet} \label{sec:Device Types +/ Network Device / Device Operation / Processing of Incoming +Packets / Hash calculation for incoming packets / +Tunnel/Encapsulated packet} A tunnel packet is encapsulated from +the original packet based on the tunneling protocol (only a +single level of encapsulation is currently supported). The encapsulatedpacket contains an outer header and an inner header, and the device calculates the hash over either the inner header or the outer header.
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
content.tex \subsubsection{Processing of Incoming · line 3881
Quoted 4 lines
content.tex +4
-0+ +When the feature VIRTIO_NET_F_HASH_TUNNEL is negotiated and the +corresponding encapsulation type is set in \field{hash_types}, +the hash for a specific type of encapsulated packet is calculated overthe inner as opposed to outer header.
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
content.tex \subsubsection{Processing of Incoming · line 3884
Quoted 3 lines
content.tex +3
-0+Supported encapsulation types are listed in \ref{sec:Device Types +/ Network Device / Device Operation / Processing of Incoming +Packets / Hash calculation for incoming packets / Supported/enabledhash types}.
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
content.tex \subsubsection{Processing of Incoming · line 3886
Quoted 2 lines
content.tex +2
-0+ +If both VIRTIO_NET_F_HASH_REPORT and
VIRTIO_NET_F_HASH_TUNNELareQuoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
content.tex \subsubsection{Processing of Incoming · line 3891
Quoted 5 lines
content.tex +5
-0+negotiated, the device can support inner hash calculation for +\hyperref[intro:GRE]{[GRE]}, \hyperref[intro:VXLAN]{[VXLAN]} and +\hyperref[intro:GENEVE]{[GENEVE]} encapsulated packets, and the +corresponding encapsulation type in \field{hash_types} is +VIRTIO_NET_HASH_TYPE_{GRE, VXLAN, GENEVE}_INNER respectively.The value in \field{hash_report_tunnel} is VIRTIO_NET_HASH_REPORT_{NONE, GRE, VXLAN, GENEVE} respectively.
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
content.tex \subsubsection{Processing of Incoming · line 3896
Quoted 5 lines
content.tex +5
-0+ +If VIRTIO_NET_F_HASH_REPORT is negotiated but +VIRTIO_NET_F_HASH_TUNNEL is not negotiated, the device calculates +the hash over the outer header, and \field{hash_report} reports +the hash type. \field{hash_report_tunnel} is no longer valid.Does this mean that VIRTIO_NET_HASH_TYPE_{GRE, VXLAN,
GENEVE}_INNER
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 2 lines
must not be set without
VIRTIO_NET_F_HASH_TUNNEL?Yes.
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 3 lines
If yes then we can add conformance statements to this end and then drop the "If the feature
VIRTIO_NET_F_HASH_TUNNELis negotiated" all over the place just relying on hash type being set instead.Please see the response below.
Quoted 3 lines
Also, note that it has to be ok for device to expose hash types without negotiating
VIRTIO_NET_F_HASH_TUNNELsince they are in config space.
Idea: do we want to decouple these completely? have
VIRTIO_NET_F_HASH_TUNNELQuoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 4 lines
just add
hash_report_tunneland have hash types speak for themselves? It makes things nicely orthogonal in that if one does not care about knowing encapsulation type (e.g. for RSS) one can disableVIRTIO_NET_F_HASH_TUNNEL.This seems to have the following 3 problems:
1. This will break the specification. As described in the specification, any
field
Quoted 1 line
Quoted 1 line
Quoted 1 line
in the configuration space only exists when there is a corresponding
feature bit:
Quoted 1 line
Quoted 1 line
Quoted 1 line
"Device configuration space is generally used for rarely-changing or
initialization-time parameters.
Quoted 1 line
Quoted 1 line
Quoted 1 line
Where configuration fields are optional, their existence is indicated by
feature bits: ",
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
which also applies to
supported_hash_types.2. If I'm not wrong, this seems to make the encapsulation hash types
dependent on
Quoted 1 line
Quoted 1 line
Quoted 1 line
VIRTIO_NET_F_RSS, can the driver readsupported_hash_types(including encapsulation
Quoted 1 line
Quoted 1 line
Quoted 1 line
hash types) if
VIRTIO_NET_F_RSSis not negotiated? But that shouldn't bethe case,
Quoted 1 line
Quoted 1 line
Quoted 1 line
we only use encapsulation hash types to provide guidance for hash
calculation, but not
Quoted 1 line
Quoted 1 line
Quoted 1 line
necessarily use hash values to select receive queues (unless
VIRTIO_NET_F_RSSis negotiated).Quoted 1 line
Quoted 1 line
Quoted 1 line
3. If all
supported_hash_typesspeak for themselves, why do we need the
VIRTIO_NET_F_HASH_TUNNELQuoted 1 line
Quoted 1 line
Quoted 1 line
feature, because as long as the encapsulation hash types exist and
VIRTIO_NET_F_HASH_REPORTisQuoted 1 line
Quoted 1 line
Quoted 1 line
negotiated,
hash_report_tunnelexists correspondingly. "have
VIRTIO_NET_F_HASH_TUNNELjust addQuoted 1 line
Quoted 1 line
Quoted 1 line
hash_report_tunnel"? It seems likehash_report_tunnelwill only work ifthe encapsulation hash
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
types are present.
So, if I'm not wrong, the reasonable logic should be that
VIRTIO_NET_F_HASH_TUNNELis only usedQuoted 1 line
Quoted 1 line
Quoted 1 line
to declare that the device supports inner header hash and let the
corresponding encapsulation
Quoted 1 line
Quoted 1 line
Quoted 1 line
hash type exist, when
VIRTIO_NET_F_RSSis negotiated, we can use thishash value to select the
Quoted 1 line
Quoted 1 line
Quoted 1 line
receive queue. Considering the migration at the same time, if the dst device
does not negotiate
Quoted 1 line
Quoted 1 line
Quoted 1 line
VIRTIO_NET_F_HASH_TUNNEL, the migration will fail; otherwise, useencapsulation hash types to
Quoted 14 lines
Quoted 6 lines
Quoted 1 line
determine whether the migration can succeed.
I think it might be better to have consistency:
1) If
VIRTIO_NET_F_HASH_TUNNELintroduces a new field in vnet header, is it better to have a new config filed in the config space?Quoted 6 lines
or
2) If
VIRTIO_NET_F_HASH_TUNNELdoesn't introduce new config file, should we try to reusehash_report?1) seems better and cleaner to me.
Sorry for the late reply due to vacation.
Quoted 8 lines
Quoted 5 lines
Quoted 2 lines
Good point, it's clearer to use the
VIRTIO_NET_F_HASH_TUNNELfeature to control all information related to inner header hash.Hi, Parav.
Do you think we need both
hash_typesandhash_tunnel_types?In struct
virtio_net_configwe need two fields.
a.supported_hash_types(already exists)
b.supported_hash_tunnel_type-> bitmap indicating for which outer headers, inner hash calculation is supported.
Thanks for the suggestion, we seem to have reached an agreement.
Quoted 6 lines
In struct
virtio_net_hdrwe need two fields.
a.hash_report(already exists)
b.hash_tunnel_type8 bits -> absolute value indicating which outer header exists when inner header hash calculated. You already have it in your patch named ashash_report_tunnel.
May be better to name ashash_report_tunnel_typeto make it clearer that its type.
Sure.
Thanks for your reply.
Quoted 2 lines
From: Heng Qi <[email protected]>
Sent: Tuesday, February 7, 2023 10:25 PM
[..]
Quoted 13 lines
Quoted 5 lines
Quoted 2 lines
Do you think we need both
hash_typesandhash_tunnel_types?In struct
virtio_net_configwe need two fields.
a.supported_hash_types(already exists) b.supported_hash_tunnel_type-> bitmap indicating for which outer headers, inner hash calculation issupported.
Thanks for the suggestion, we seem to have reached an agreement.
Quoted 4 lines
In struct
virtio_net_hdrwe need two fields.
a.hash_report(already exists)
b.hash_tunnel_type8 bits -> absolute value indicating which outer header
Quoted 8 lines
exists when inner header hash calculated.
Quoted 2 lines
You already have it in your patch named as
hash_report_tunnel. May be better to name ashash_report_tunnel_typeto make it clearer that itstype.
Sure.
Thanks for your reply.
I had one last question. Why do we need to inform the hash_report_tunnel_type of the outer header in the virtio_net_hdr? Is this for debug? Or is there a use case that will process this value?
Quoted 17 lines · Parav Pandit
Quoted 2 lines
From: Heng Qi <[email protected]>
Sent: Tuesday, February 7, 2023 10:25 PM[..]
Quoted 14 lines
Quoted 4 lines
Quoted 1 line
Do you think we need both
hash_typesandhash_tunnel_types?In struct
virtio_net_configwe need two fields.
a.supported_hash_types(already exists) b.supported_hash_tunnel_type-> bitmap indicating for which outer headers, inner hash calculation issupported.
Thanks for the suggestion, we seem to have reached an agreement.
Quoted 3 lines
In struct
virtio_net_hdrwe need two fields.
a.hash_report(already exists)
b.hash_tunnel_type8 bits -> absolute value indicating which outer headerexists when inner header hash calculated.
Quoted 2 lines
You already have it in your patch named as
hash_report_tunnel. May be better to name ashash_report_tunnel_typeto make it clearer that its
Quoted 7 lines
Quoted 5 lines
type.
Sure.
Thanks for your reply.
I had one last question. Why do we need to inform the
hash_report_tunnel_typeof the outer header in thevirtio_net_hdr? Is this for debug? Or is there a use case that will process this value?
The driver may use it to do some statistical information, or do some rx classification based on the rx hash, and we'd better not hide information from the driver.
Thanks.
Quoted 31 lines
From: Heng Qi <[email protected]>
Sent: Wednesday, February 8, 2023 1:11 AMQuoted 25 lines · Parav Pandit
Quoted 2 lines
From: Heng Qi <[email protected]>
Sent: Tuesday, February 7, 2023 10:25 PM[..]
Quoted 22 lines
Quoted 6 lines
Quoted 1 line
Do you think we need both
hash_typesandhash_tunnel_types?In struct
virtio_net_configwe need two fields.
a.supported_hash_types(already exists) b.supported_hash_tunnel_type
-> bitmap indicating for which outer headers, inner hash calculation -> issupported.
Thanks for the suggestion, we seem to have reached an agreement.
Quoted 4 lines
In struct
virtio_net_hdrwe need two fields.
a.hash_report(already exists)
b.hash_tunnel_type8 bits -> absolute value indicating which outer headerexists when inner header hash calculated.
Quoted 3 lines
You already have it in your patch named as
hash_report_tunnel. May be better to name ashash_report_tunnel_typeto make it clearer that itstype.
Sure.
Quoted 9 lines
Quoted 2 lines
Quoted 1 line
Thanks for your reply.
I had one last question. Why do we need to inform the
hash_report_tunnel_typeof the outer header in thevirtio_net_hdr?Quoted 1 line
Is this for debug? Or is there a use case that will process this value?
The driver may use it to do some statistical information, or do some rx classification based on the rx hash, and we'd better not hide information from the driver.
Statistical information is better gathered via stats, instead of adding such code in driver data path.
It is not about hiding.
It has onus on the data path to detect the tunnel type and place that virtio_net_hdr. So lets find one use of it in data path which sw entity will use it.
Quoted 31 lines · Parav Pandit
Quoted 31 lines
From: Heng Qi <[email protected]>
Sent: Wednesday, February 8, 2023 1:11 AMQuoted 26 lines · Parav Pandit
Quoted 2 lines
From: Heng Qi <[email protected]>
Sent: Tuesday, February 7, 2023 10:25 PM[..]
Quoted 23 lines
Quoted 6 lines
Quoted 1 line
Do you think we need both
hash_typesandhash_tunnel_types?In struct
virtio_net_configwe need two fields.
a.supported_hash_types(already exists) b.supported_hash_tunnel_type
-> bitmap indicating for which outer headers, inner hash calculation -> issupported.
Thanks for the suggestion, we seem to have reached an agreement.
Quoted 4 lines
In struct
virtio_net_hdrwe need two fields.
a.hash_report(already exists)
b.hash_tunnel_type8 bits -> absolute value indicating which outer headerexists when inner header hash calculated.
Quoted 3 lines
You already have it in your patch named as
hash_report_tunnel. May be better to name ashash_report_tunnel_typeto make it clearer that itstype.
Sure.
Thanks for your reply.
Quoted 8 lines
Quoted 7 lines
Quoted 1 line
I had one last question. Why do we need to inform the
hash_report_tunnel_typeof the outer header in thevirtio_net_hdr?Quoted 1 line
Is this for debug? Or is there a use case that will process this value?
The driver may use it to do some statistical information, or do some rx classification based on the rx hash, and we'd better not hide information from the driver.
Statistical information is better gathered via stats, instead of adding such code in driver data path.
A cautionary note : the source of hash_report_tunnel comes from the discussion here https://lists.oasis-open.org/archives/virtio-dev/202211/msg00063.html . If I understand correctly, if virtio-net-hdr also has hash_report_tunnel, how does the driver do statistics?
CC Yuri for some opinions.
Thanks.
Quoted 6 lines
It is not about hiding.
It has onus on the data path to detect the tunnel type and place thatvirtio_net_hdr. So lets find one use of it in data path which sw entity will use it.
This publicly archived list offers a means to provide input to the OASIS Virtual I/O Device (VIRTIO) TC.
In order to verify user consent to the Feedback License terms and to minimize spam in the list archive, subscription is required before posting.
Subscribe: [email protected]
Unsubscribe: [email protected]
List help: [email protected]
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/
Quoted 13 lines
From: Heng Qi <[email protected]>
Sent: Thursday, February 9, 2023 12:21 AMQuoted 9 lines
Quoted 8 lines
Quoted 2 lines
Quoted 1 line
Thanks for your reply.
I had one last question. Why do we need to inform the
hash_report_tunnel_typeof the outer header in thevirtio_net_hdr?Quoted 1 line
Is this for debug? Or is there a use case that will process this value?
The driver may use it to do some statistical information, or do some rx classification based on the rx hash, and we'd better not hide information from the driver.
Statistical information is better gathered via stats, instead of adding such code
Quoted 8 lines
in driver data path.
A cautionary note : the source of
hash_report_tunnelcomes from the discussion here
https://lists.oasis-open.org/archives/virtio-dev/202211/msg00063.htmlIf I understand correctly, if virtio-net-hdr also has
hash_report_tunnel, how does the driver do statistics?
I am asking to avoid having hash_report_tunnel in virtio_net_hdr if it is only for the statistics purpose.
If it has more use than just statistics, I like to understand its use in rx packet processing flow, if there is any and to describe in commit log too.
Below [1] says about driver doing some rx classification. It is unclear on inner or outer. The response is not very clear to me, about how this field is useful.
[1] https://lists.oasis-open.org/archives/virtio-dev/202211/msg00063.html
If driver needs to know the statistics of it, it can query them from the device by introducing statistics cmd.
Quoted 11 lines · Parav Pandit
Quoted 11 lines
From: Heng Qi <[email protected]>
Sent: Thursday, February 9, 2023 12:21 AMQuoted 9 lines
Quoted 8 lines
Quoted 2 lines
Quoted 1 line
Thanks for your reply.
I had one last question. Why do we need to inform the
hash_report_tunnel_typeof the outer header in thevirtio_net_hdr?Quoted 1 line
Is this for debug? Or is there a use case that will process this value?
The driver may use it to do some statistical information, or do some rx classification based on the rx hash, and we'd better not hide information from the driver.
Statistical information is better gathered via stats, instead of adding such code
Quoted 8 lines
Quoted 7 lines
in driver data path.
A cautionary note : the source of
hash_report_tunnelcomes from the discussion here
https://lists.oasis-open.org/archives/virtio-dev/202211/msg00063.html
If I understand correctly, if virtio-net-hdr also hashash_report_tunnel, how does the driver do statistics?I am asking to avoid having
hash_report_tunnelinvirtio_net_hdrif it is only for the statistics purpose.
Parav, I don't seem to follow this.
Note that the hash_report(le16) and padding reserved(le16) fields only exist when VIRTIO_NET_F_HASH_REPORT is negotiated, and hash_report_tunnel only uses the first 8 bits of padding_reserved, which does not add any overhead.
Whether to reuse hash_report or use the new hash_report_tunnel initially has been discussed: https://lists.oasis-open.org/archives/virtio-dev/202211/msg00065.html .
Quoted 1 line
If it has more use than just statistics, I like to understand its use in rx packet processing flow, if there is any and to describe in commit log too.
I think there are still some important points, such as the kernel or driver forwards the packet,
and the hash value is used when selecting the queue, but at this time they want to know whether
the hash value comes from the inner or outer header.
What if it doesn't want to forward based on the hash calculated by the inner header, but only uses the hash calculated by the outer header?
Thanks.
Quoted 6 lines
Below [1] says about driver doing some rx classification. It is unclear on inner or outer. The response is not very clear to me, about how this field is useful.
[1] https://lists.oasis-open.org/archives/virtio-dev/202211/msg00063.html
If driver needs to know the statistics of it, it can query them from the device by introducing statistics cmd.
Any more discussions or comments? Please let me know if this is OK with you. and the v8 version is released at
[1] https://lists.oasis-open.org/archives/virtio-dev/202302/msg00094.html
Thanks.
Quoted 54 lines · Heng Qi
Quoted 11 lines · Parav Pandit
Quoted 11 lines
From: Heng Qi <[email protected]>
Sent: Thursday, February 9, 2023 12:21 AMQuoted 9 lines
Quoted 8 lines
Quoted 2 lines
Quoted 1 line
Thanks for your reply.
I had one last question. Why do we need to inform the
hash_report_tunnel_typeof the outer header in thevirtio_net_hdr?Quoted 1 line
Is this for debug? Or is there a use case that will process this value?
The driver may use it to do some statistical information, or do some rx classification based on the rx hash, and we'd better not hide information from the driver.
Statistical information is better gathered via stats, instead of adding such code
Quoted 8 lines
Quoted 7 lines
in driver data path.
A cautionary note : the source of
hash_report_tunnelcomes from the discussion here
https://lists.oasis-open.org/archives/virtio-dev/202211/msg00063.html
If I understand correctly, if virtio-net-hdr also hashash_report_tunnel, how does the driver do statistics?I am asking to avoid having
hash_report_tunnelinvirtio_net_hdrif it is only for the statistics purpose.Parav, I don't seem to follow this.
Note that the
hash_report(le16)and padding reserved(le16) fields only exist whenVIRTIO_NET_F_HASH_REPORTis negotiated, andhash_report_tunnelonly uses the first 8 bits ofpadding_reserved, which does not add any overhead.Whether to reuse
hash_reportor use the newhash_report_tunnelinitially has been discussed: https://lists.oasis-open.org/archives/virtio-dev/202211/msg00065.html .Quoted 1 line
If it has more use than just statistics, I like to understand its use in rx packet processing flow, if there is any and to describe in commit log too.
I think there are still some important points, such as the kernel or driver forwards the packet,
and the hash value is used when selecting the queue, but at this time they want to know whether
the hash value comes from the inner or outer header.
What if it doesn't want to forward based on the hash calculated by the inner header, but only uses the hash calculated by the outer header?Thanks.
Quoted 6 lines
Below [1] says about driver doing some rx classification. It is unclear on inner or outer. The response is not very clear to me, about how this field is useful.
[1] https://lists.oasis-open.org/archives/virtio-dev/202211/msg00063.html
If driver needs to know the statistics of it, it can query them from the device by introducing statistics cmd.
This publicly archived list offers a means to provide input to the OASIS Virtual I/O Device (VIRTIO) TC.
In order to verify user consent to the Feedback License terms and to minimize spam in the list archive, subscription is required before posting.
Quoted 8 lines
Subscribe: [email protected]
Unsubscribe: [email protected]
List help: [email protected]
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/
This publicly archived list offers a means to provide input to the OASIS Virtual I/O Device (VIRTIO) TC.
In order to verify user consent to the Feedback License terms and to minimize spam in the list archive, subscription is required before posting.
Subscribe: [email protected]
Unsubscribe: [email protected]
List help: [email protected]
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/
Quoted 20 lines · Parav Pandit
Quoted 2 lines
From: Heng Qi <[email protected]>
Sent: Tuesday, February 7, 2023 10:25 PM[..]
Quoted 16 lines
Quoted 5 lines
Quoted 2 lines
Do you think we need both
hash_typesandhash_tunnel_types?In struct
virtio_net_configwe need two fields.
a.supported_hash_types(already exists) b.supported_hash_tunnel_type-> bitmap indicating for which outer headers, inner hash calculation issupported.
Thanks for the suggestion, we seem to have reached an agreement.
Quoted 4 lines
In struct
virtio_net_hdrwe need two fields.
a.hash_report(already exists)
b.hash_tunnel_type8 bits -> absolute value indicating which outer headerexists when inner header hash calculated.
Quoted 2 lines
You already have it in your patch named as
hash_report_tunnel. May be better to name ashash_report_tunnel_typeto make it clearer that its
Quoted 8 lines
Quoted 5 lines
type.
Sure.
Thanks for your reply.
I had one last question. Why do we need to inform the
hash_report_tunnel_typeof the outer header in thevirtio_net_hdr? Is this for debug? Or is there a use case that will process this value?
Well we have hash_report which is kind of similar (and also kind of pointless but I think it's there because WHQL wants it). Maybe we can steal some bits from there instead of a new field?
I have a follow up question though: are we only hashing the inner header or both inner and outer header? Somewhat confused on this.
In fact, CC Yuri for thoughts and suggestions from windows side of things.
Signature · 1 line
MST
Quoted 33 lines
From: Michael S. Tsirkin <[email protected]>
Sent: Wednesday, February 8, 2023 8:32 AMQuoted 28 lines · Parav Pandit
Quoted 2 lines
From: Heng Qi <[email protected]>
Sent: Tuesday, February 7, 2023 10:25 PM[..]
Quoted 24 lines
Quoted 7 lines
Quoted 2 lines
Do you think we need both
hash_typesandhash_tunnel_types?In struct
virtio_net_configwe need two fields. a.supported_hash_types(already exists) b.supported_hash_tunnel_type
-> bitmap indicating for which outer headers, inner hash -> calculation issupported.
Thanks for the suggestion, we seem to have reached an agreement.
Quoted 5 lines
In struct
virtio_net_hdrwe need two fields.
a.hash_report(already exists)
b.hash_tunnel_type8 bits -> absolute value indicating which outer headerexists when inner header hash calculated.
Quoted 3 lines
You already have it in your patch named as
hash_report_tunnel. May be better to name ashash_report_tunnel_typeto make it clearer that itstype.
Sure.
Quoted 8 lines
Quoted 3 lines
Quoted 1 line
Thanks for your reply.
I had one last question. Why do we need to inform the
hash_report_tunnel_typeof the outer header in thevirtio_net_hdr?Quoted 1 line
Is this for debug? Or is there a use case that will process this value?
Well we have
hash_reportwhich is kind of similar (and also kind of pointless but I think it's there because WHQL wants it).
Hash_report is useful. It tells hash_value is in which namespace (ipv4-tcp/ipv4 udp etc). OS can use this value to find tcp connection in a given namespace.
Quoted 3 lines
Maybe we can steal some bits
from there instead of a new field?
I do not have problem adding extra bits. I just don't find that just telling that its vxlan or nvgre to the OS is useful. If OS needs to know about outer header details, it needs to know the VNI information than just telling vxlan.
Quoted 4 lines
I have a follow up question though: are we only hashing the inner header or both inner and outer header? Somewhat confused on this.
I understood as inner header. But worth to describe it. May be there. Need to read v8 patch.
This publicly archived list offers a means to provide input to the OASIS Virtual I/O Device (VIRTIO) TC.
In order to verify user consent to the Feedback License terms and to minimize spam in the list archive, subscription is required before posting.
Subscribe: [email protected]
Unsubscribe: [email protected]
List help: [email protected]
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/
Quoted 41 lines · Parav Pandit
Quoted 33 lines
From: Michael S. Tsirkin <[email protected]>
Sent: Wednesday, February 8, 2023 8:32 AMQuoted 28 lines · Parav Pandit
Quoted 2 lines
From: Heng Qi <[email protected]>
Sent: Tuesday, February 7, 2023 10:25 PM[..]
Quoted 24 lines
Quoted 7 lines
Quoted 2 lines
Do you think we need both
hash_typesandhash_tunnel_types?In struct
virtio_net_configwe need two fields. a.supported_hash_types(already exists) b.supported_hash_tunnel_type
-> bitmap indicating for which outer headers, inner hash -> calculation issupported.
Thanks for the suggestion, we seem to have reached an agreement.
Quoted 5 lines
In struct
virtio_net_hdrwe need two fields.
a.hash_report(already exists)
b.hash_tunnel_type8 bits -> absolute value indicating which outer headerexists when inner header hash calculated.
Quoted 3 lines
You already have it in your patch named as
hash_report_tunnel. May be better to name ashash_report_tunnel_typeto make it clearer that itstype.
Sure.
Quoted 8 lines
Quoted 3 lines
Quoted 1 line
Thanks for your reply.
I had one last question. Why do we need to inform the
hash_report_tunnel_typeof the outer header in thevirtio_net_hdr?Quoted 1 line
Is this for debug? Or is there a use case that will process this value?
Well we have
hash_reportwhich is kind of similar (and also kind of pointless but I think it's there because WHQL wants it).
Quoted 8 lines
Hash_reportis useful. It tellshash_valueis in which namespace (ipv4-tcp/ipv4 udp etc). OS can use this value to find tcp connection in a given namespace.Quoted 3 lines
Maybe we can steal some bits
from there instead of a new field?I do not have problem adding extra bits. I just don't find that just telling that its vxlan or nvgre to the OS is useful. If OS needs to know about outer header details, it needs to know the VNI information than just telling vxlan.
This does make sense.
Quoted 5 lines
Quoted 4 lines
I have a follow up question though: are we only hashing the inner header or both inner and outer header? Somewhat confused on this.
I understood as inner header. But worth to describe it. May be there. Need to read v8 patch.
Hmm. I just realized that there's a security problem with hashing just the inner header: it allow users inside the tunnel control queueing outside. By observing packet loss some information leaks between tunnels.
Ideas for solving this they all involve hashing both inner and outer header:
1- report two sets of hashes. overkill?
2- hash both headers together
2- add salt. can come from driver or device itself
More ideas?
Signature · 18 lines
MST This publicly archived list offers a means to provide input to the OASIS Virtual I/O Device (VIRTIO) TC. In order to verify user consent to the Feedback License terms and to minimize spam in the list archive, subscription is required before posting. Subscribe: [email protected] Unsubscribe: [email protected] List help: [email protected] List archive: https://lists.oasis-open.org/archives/virtio-comment/ Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists Committee: https://www.oasis-open.org/committees/virtio/ Join OASIS: https://www.oasis-open.org/join/
Quoted 63 lines
From: Michael S. Tsirkin <[email protected]>
Sent: Wednesday, February 8, 2023 8:52 AMQuoted 34 lines · Parav Pandit
Quoted 34 lines
From: Michael S. Tsirkin <[email protected]>
Sent: Wednesday, February 8, 2023 8:32 AMQuoted 29 lines · Parav Pandit
Quoted 2 lines
From: Heng Qi <[email protected]>
Sent: Tuesday, February 7, 2023 10:25 PM[..]
Quoted 24 lines
Quoted 7 lines
Quoted 2 lines
Do you think we need both
hash_typesandhash_tunnel_types?In struct
virtio_net_configwe need two fields. a.supported_hash_types(already exists) b.supported_hash_tunnel_type
-> bitmap indicating for which outer headers, inner hash -> calculation issupported.
Thanks for the suggestion, we seem to have reached an agreement.
Quoted 4 lines
In struct
virtio_net_hdrwe need two fields.
a.hash_report(already exists) b.hash_tunnel_type8 bits -> absolute value indicating which outer headerexists when inner header hash calculated.
Quoted 3 lines
You already have it in your patch named as
hash_report_tunnel. May be better to name ashash_report_tunnel_typeto make it clearer that itstype.
Sure.
Thanks for your reply.
Quoted 7 lines
Quoted 6 lines
Quoted 1 line
I had one last question. Why do we need to inform the
hash_report_tunnel_typeof the outer header in thevirtio_net_hdr?Quoted 1 line
Is this for debug? Or is there a use case that will process this value?
Well we have
hash_reportwhich is kind of similar (and also kind of pointless but I think it's there because WHQL wants it).
Hash_reportis useful. It tellshash_valueis in which namespace (ipv4-tcp/ipv4udp etc).
Quoted 6 lines
OS can use this value to find tcp connection in a given namespace.
Quoted 3 lines
Maybe we can steal some bits
from there instead of a new field?I do not have problem adding extra bits. I just don't find that just telling that
its vxlan or nvgre to the OS is useful.
Quoted 1 line
If OS needs to know about outer header details, it needs to know the VNI
information than just telling vxlan.
This does make sense.
Quoted 4 lines
Quoted 4 lines
I have a follow up question though: are we only hashing the inner header or both inner and outer header? Somewhat confused on this.
Quoted 7 lines
Quoted 1 line
I understood as inner header. But worth to describe it. May be there. Need to
read v8 patch.
Hmm. I just realized that there's a security problem with hashing just the inner
header: it allow users inside the tunnel control queueing outside. By observing packet loss some information leaks between tunnels.
I likely didn't understand. Can you please explain?
Queuing is always done on the inner header with/without encapsulation. Hash is always reported for inner header.
It is only adding the ability to hash even when outer header exists.
If queuing to be decided based on outer header (hash), then that is different. Hashing both inner and outer in a flat q structure unlikely works, right? Because both hashes can result in different q selection.
Quoted 11 lines
Ideas for solving this they all involve hashing both inner and outer header:
1- report two sets of hashes. overkill?
2- hash both headers together
2- add salt. can come from driver or device itselfMore ideas?
--
MST
Quoted 70 lines · Parav Pandit
Quoted 63 lines
From: Michael S. Tsirkin <[email protected]>
Sent: Wednesday, February 8, 2023 8:52 AMQuoted 34 lines · Parav Pandit
Quoted 34 lines
From: Michael S. Tsirkin <[email protected]>
Sent: Wednesday, February 8, 2023 8:32 AMQuoted 29 lines · Parav Pandit
Quoted 2 lines
From: Heng Qi <[email protected]>
Sent: Tuesday, February 7, 2023 10:25 PM[..]
Quoted 24 lines
Quoted 7 lines
Quoted 2 lines
Do you think we need both
hash_typesandhash_tunnel_types?In struct
virtio_net_configwe need two fields. a.supported_hash_types(already exists) b.supported_hash_tunnel_type
-> bitmap indicating for which outer headers, inner hash -> calculation issupported.
Thanks for the suggestion, we seem to have reached an agreement.
Quoted 4 lines
In struct
virtio_net_hdrwe need two fields.
a.hash_report(already exists) b.hash_tunnel_type8 bits -> absolute value indicating which outer headerexists when inner header hash calculated.
Quoted 3 lines
You already have it in your patch named as
hash_report_tunnel. May be better to name ashash_report_tunnel_typeto make it clearer that itstype.
Sure.
Thanks for your reply.
Quoted 7 lines
Quoted 6 lines
Quoted 1 line
I had one last question. Why do we need to inform the
hash_report_tunnel_typeof the outer header in thevirtio_net_hdr?Quoted 1 line
Is this for debug? Or is there a use case that will process this value?
Well we have
hash_reportwhich is kind of similar (and also kind of pointless but I think it's there because WHQL wants it).
Hash_reportis useful. It tellshash_valueis in which namespace (ipv4-tcp/ipv4udp etc).
Quoted 6 lines
OS can use this value to find tcp connection in a given namespace.
Quoted 3 lines
Maybe we can steal some bits
from there instead of a new field?I do not have problem adding extra bits. I just don't find that just telling that
its vxlan or nvgre to the OS is useful.
Quoted 1 line
If OS needs to know about outer header details, it needs to know the VNI
information than just telling vxlan.
This does make sense.
Quoted 4 lines
Quoted 4 lines
I have a follow up question though: are we only hashing the inner header or both inner and outer header? Somewhat confused on this.
Quoted 7 lines
Quoted 1 line
I understood as inner header. But worth to describe it. May be there. Need to
read v8 patch.
Hmm. I just realized that there's a security problem with hashing just the inner
header: it allow users inside the tunnel control queueing outside. By observing packet loss some information leaks between tunnels.
Quoted 5 lines
I likely didn't understand. Can you please explain?
Queuing is always done on the inner header with/without encapsulation. Hash is always reported for inner header.
It is only adding the ability to hash even when outer header exists.
If hashing just on outer header (currently the only option) then a given tunnel all lands in a given queue.
Just keep that queue separate and users of this tunnel can not learn whether other queues are overflowing, and can not overflow other queues.
If you hash inner header then user can flood device with
packets of a given connection and the same connection in a different tunnel hashes to the same queue. Now one tunnel can
- cause DoS for another tunnel
- cause packet loss or latency triggering possible security bugs within guest
- detect that another tunnel is using the connection by
detecting its own packet loss or increased latency
Quoted 3 lines
If queuing to be decided based on outer header (hash), then that is different. Hashing both inner and outer in a flat q structure unlikely works, right? Because both hashes can result in different q selection.
That's the point.
Is there any precedent in OSes for configuring things like this that we can look at?
Quoted 11 lines
Quoted 11 lines
Ideas for solving this they all involve hashing both inner and outer header:
1- report two sets of hashes. overkill?
2- hash both headers together
2- add salt. can come from driver or device itselfMore ideas?
--
MST
Quoted 19 lines
From: Michael S. Tsirkin <[email protected]>
Sent: Wednesday, February 8, 2023 9:09 AMQuoted 8 lines
Quoted 3 lines
header: it allow users inside the tunnel control queueing outside. By observing packet loss some information leaks between tunnels.
I likely didn't understand. Can you please explain?
Queuing is always done on the inner header with/without encapsulation. Hash is always reported for inner header.
It is only adding the ability to hash even when outer header exists.If hashing just on outer header (currently the only option) then a given tunnel all lands in a given queue.
Just keep that queue separate and users of this tunnel can not learn whether other queues are overflowing, and can not overflow other queues.
Quoted 8 lines
If you hash inner header then user can flood device with packets of a given connection and the same connection in a different tunnel hashes to the same queue. Now one tunnel can
- cause DoS for another tunnel
- cause packet loss or latency triggering possible security bugs within guest
- detect that another tunnel is using the connection by
detecting its own packet loss or increased latency
Yes. It can lead to above issues.
Steering on inner is on best effort based sw implementations running on top of net device.
To avoid above issues, a hierarchical model is needed.
I am not aware of any.
To my knowledge, usually who care for above issues end up using a different net device for each VNI and achieve the desired hierarchy.
Quoted 11 lines
Quoted 4 lines
If queuing to be decided based on outer header (hash), then that is different. Hashing both inner and outer in a flat q structure unlikely works, right? Because both hashes can result in different q selection.
That's the point.
Is there any precedent in OSes for configuring things like this that we can look at?
ethtool -N (not yet part of virtio) is the closest match that can steer based on inner and outer both, but it is not hierarchical, and it is orthogonal to this feature.
Quoted 12 lines
Quoted 12 lines
Quoted 12 lines
Ideas for solving this they all involve hashing both inner and outer header:
1- report two sets of hashes. overkill?
2- hash both headers together
2- add salt. can come from driver or device itselfMore ideas?
--
MST
Quoted 78 lines · Michael S. Tsirkin
Quoted 70 lines · Parav Pandit
Quoted 63 lines
From: Michael S. Tsirkin <[email protected]>
Sent: Wednesday, February 8, 2023 8:52 AMQuoted 34 lines · Parav Pandit
Quoted 34 lines
From: Michael S. Tsirkin <[email protected]>
Sent: Wednesday, February 8, 2023 8:32 AMQuoted 29 lines · Parav Pandit
Quoted 2 lines
From: Heng Qi <[email protected]>
Sent: Tuesday, February 7, 2023 10:25 PM[..]
Quoted 24 lines
Quoted 7 lines
Quoted 2 lines
Do you think we need both
hash_typesandhash_tunnel_types?In struct
virtio_net_configwe need two fields. a.supported_hash_types(already exists) b.supported_hash_tunnel_type
-> bitmap indicating for which outer headers, inner hash -> calculation issupported.
Thanks for the suggestion, we seem to have reached an agreement.
Quoted 4 lines
In struct
virtio_net_hdrwe need two fields.
a.hash_report(already exists) b.hash_tunnel_type8 bits -> absolute value indicating which outer headerexists when inner header hash calculated.
Quoted 3 lines
You already have it in your patch named as
hash_report_tunnel. May be better to name ashash_report_tunnel_typeto make it clearer that itstype.
Sure.
Thanks for your reply.
Quoted 7 lines
Quoted 6 lines
Quoted 1 line
I had one last question. Why do we need to inform the
hash_report_tunnel_typeof the outer header in thevirtio_net_hdr?Quoted 1 line
Is this for debug? Or is there a use case that will process this value?
Well we have
hash_reportwhich is kind of similar (and also kind of pointless but I think it's there because WHQL wants it).
Hash_reportis useful. It tellshash_valueis in which namespace (ipv4-tcp/ipv4udp etc).
Quoted 6 lines
OS can use this value to find tcp connection in a given namespace.
Quoted 3 lines
Maybe we can steal some bits
from there instead of a new field?I do not have problem adding extra bits. I just don't find that just telling that
its vxlan or nvgre to the OS is useful.
Quoted 1 line
If OS needs to know about outer header details, it needs to know the VNI
information than just telling vxlan.
This does make sense.
Quoted 4 lines
Quoted 4 lines
I have a follow up question though: are we only hashing the inner header or both inner and outer header? Somewhat confused on this.
Quoted 7 lines
Quoted 1 line
I understood as inner header. But worth to describe it. May be there. Need to
read v8 patch.
Hmm. I just realized that there's a security problem with hashing just the inner
header: it allow users inside the tunnel control queueing outside. By observing packet loss some information leaks between tunnels.Quoted 5 lines
I likely didn't understand. Can you please explain?
Queuing is always done on the inner header with/without encapsulation. Hash is always reported for inner header.
It is only adding the ability to hash even when outer header exists.
Quoted 6 lines
If hashing just on outer header (currently the only option) then a given tunnel all lands in a given queue.
Just keep that queue separate and users of this tunnel can not learn whether other queues are overflowing, and can not overflow other queues.
This is not a problem with the inner hash, it is a general problem with the outer hash.
I communicated with our people who are doing cloud security (they are also one of the demanders of inner hash), and it is a common problem for one tunnel to attack another tunnel, which has nothing to do with inner hash or outer hash.
For example, there is a tunnel t1; a tunnel t2; a tunnel endpoint VTEP0, and the vni id of t1 is id1, and the vni id of v2 is id2; a VM.
-----------
Tunnel t1 | |VTEP_1 <=======================> (vni id1) |
| |
| VTEP0 | ======================> VM
Tunnel t2 | |
VTEP_2 <=======================> (vni id2) |
-----------At this time, regardless of the inner hash or the outer hash, the traffic of tunnel t1 and tunnel t2 will reach the VM through VTEP0 (whether it is a single queue or multiple queues), and may be placed on the same queue to cause queue overflow. Some current forwarding tools such as DPDK have good forwarding performance, and it is difficult to fill up the queue; or switch the attack traffic to the attack clusters; or connect the traffic of different tunnels to different network card ports or network devices.
Thanks.
Quoted 43 lines
If you hash inner header then user can flood device with
packets of a given connection and the same connection in a different tunnel hashes to the same queue. Now one tunnel can
- cause DoS for another tunnel
- cause packet loss or latency triggering possible security bugs within guest
- detect that another tunnel is using the connection by
detecting its own packet loss or increased latencyQuoted 3 lines
If queuing to be decided based on outer header (hash), then that is different. Hashing both inner and outer in a flat q structure unlikely works, right? Because both hashes can result in different q selection.
That's the point.
Is there any precedent in OSes for configuring things like this that we can look at?
Quoted 11 lines
Quoted 11 lines
Ideas for solving this they all involve hashing both inner and outer header:
1- report two sets of hashes. overkill?
2- hash both headers together
2- add salt. can come from driver or device itselfMore ideas?
--
MSTThis publicly archived list offers a means to provide input to the OASIS Virtual I/O Device (VIRTIO) TC.
In order to verify user consent to the Feedback License terms and to minimize spam in the list archive, subscription is required before posting.
Quoted 8 lines
Subscribe: [email protected]
Unsubscribe: [email protected]
List help: [email protected]
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/
This publicly archived list offers a means to provide input to the OASIS Virtual I/O Device (VIRTIO) TC.
In order to verify user consent to the Feedback License terms and to minimize spam in the list archive, subscription is required before posting.
Subscribe: [email protected]
Unsubscribe: [email protected]
List help: [email protected]
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/
Quoted 93 lines · Heng Qi
Quoted 78 lines · Michael S. Tsirkin
Quoted 70 lines · Parav Pandit
Quoted 63 lines
From: Michael S. Tsirkin <[email protected]>
Sent: Wednesday, February 8, 2023 8:52 AMQuoted 34 lines · Parav Pandit
Quoted 34 lines
From: Michael S. Tsirkin <[email protected]>
Sent: Wednesday, February 8, 2023 8:32 AMQuoted 29 lines · Parav Pandit
Quoted 2 lines
From: Heng Qi <[email protected]>
Sent: Tuesday, February 7, 2023 10:25 PM[..]
Quoted 24 lines
Quoted 7 lines
Quoted 2 lines
Do you think we need both
hash_typesandhash_tunnel_types?In struct
virtio_net_configwe need two fields. a.supported_hash_types(already exists) b.supported_hash_tunnel_type
-> bitmap indicating for which outer headers, inner hash -> calculation issupported.
Thanks for the suggestion, we seem to have reached an agreement.
Quoted 4 lines
In struct
virtio_net_hdrwe need two fields.
a.hash_report(already exists) b.hash_tunnel_type8 bits -> absolute value indicating which outer headerexists when inner header hash calculated.
Quoted 3 lines
You already have it in your patch named as
hash_report_tunnel. May be better to name ashash_report_tunnel_typeto make it clearer that itstype.
Sure.
Thanks for your reply.
Quoted 7 lines
Quoted 6 lines
Quoted 1 line
I had one last question. Why do we need to inform the
hash_report_tunnel_typeof the outer header in thevirtio_net_hdr?Quoted 1 line
Is this for debug? Or is there a use case that will process this value?
Well we have
hash_reportwhich is kind of similar (and also kind of pointless but I think it's there because WHQL wants it).
Hash_reportis useful. It tellshash_valueis in which namespace (ipv4-tcp/ipv4udp etc).
Quoted 6 lines
OS can use this value to find tcp connection in a given namespace.
Quoted 3 lines
Maybe we can steal some bits
from there instead of a new field?I do not have problem adding extra bits. I just don't find that just telling that
its vxlan or nvgre to the OS is useful.
Quoted 1 line
If OS needs to know about outer header details, it needs to know the VNI
information than just telling vxlan.
This does make sense.
Quoted 4 lines
Quoted 4 lines
I have a follow up question though: are we only hashing the inner header or both inner and outer header? Somewhat confused on this.
Quoted 7 lines
Quoted 1 line
I understood as inner header. But worth to describe it. May be there. Need to
read v8 patch.
Hmm. I just realized that there's a security problem with hashing just the inner
header: it allow users inside the tunnel control queueing outside. By observing packet loss some information leaks between tunnels.Quoted 5 lines
I likely didn't understand. Can you please explain?
Queuing is always done on the inner header with/without encapsulation. Hash is always reported for inner header.
It is only adding the ability to hash even when outer header exists.Quoted 6 lines
If hashing just on outer header (currently the only option) then a given tunnel all lands in a given queue.
Just keep that queue separate and users of this tunnel can not learn whether other queues are overflowing, and can not overflow other queues.This is not a problem with the inner hash, it is a general problem with the outer hash.
I communicated with our people who are doing cloud security (they are also one of the demanders of inner hash), and it is a common problem for one tunnel to attack another tunnel, which has nothing to do with inner hash or outer hash.For example, there is a tunnel t1; a tunnel t2; a tunnel endpoint VTEP0, and the vni id of t1 is id1, and the vni id of v2 is id2; a VM.
Quoted 8 lines
-----------
Tunnel t1 | |VTEP_1<=======================> (vni id1) || | | VTEP0 | ======================> VM Tunnel t2 | | VTEP_2 <=======================> (vni id2) | -----------
The description graph should look like this:
-----------
Tunnel t1 | |
VTEP_1 <=======================> (vni id1) |
| |
| VTEP | ======================> VM
Tunnel t2 | |
VTEP_2 <=======================> (vni id2) |
-----------Quoted 63 lines
At this time, regardless of the inner hash or the outer hash, the traffic of tunnel t1 and tunnel t2 will reach the VM through VTEP0 (whether it is a single queue or multiple queues), and may be placed on the same queue to cause queue overflow. Some current forwarding tools such as DPDK have good forwarding performance, and it is difficult to fill up the queue; or switch the attack traffic to the attack clusters; or connect the traffic of different tunnels to different network card ports or network devices.
Thanks.
Quoted 43 lines
If you hash inner header then user can flood device with
packets of a given connection and the same connection in a different tunnel hashes to the same queue. Now one tunnel can
- cause DoS for another tunnel
- cause packet loss or latency triggering possible security bugs within guest
- detect that another tunnel is using the connection by
detecting its own packet loss or increased latencyQuoted 3 lines
If queuing to be decided based on outer header (hash), then that is different. Hashing both inner and outer in a flat q structure unlikely works, right? Because both hashes can result in different q selection.
That's the point.
Is there any precedent in OSes for configuring things like this that we can look at?
Quoted 11 lines
Quoted 11 lines
Ideas for solving this they all involve hashing both inner and outer header:
1- report two sets of hashes. overkill?
2- hash both headers together
2- add salt. can come from driver or device itselfMore ideas?
--
MSTThis publicly archived list offers a means to provide input to the OASIS Virtual I/O Device (VIRTIO) TC.
In order to verify user consent to the Feedback License terms and to minimize spam in the list archive, subscription is required before posting.
Quoted 8 lines
Subscribe: [email protected]
Unsubscribe: [email protected]
List help: [email protected]
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Quoted 63 lines
From: Michael S. Tsirkin <[email protected]>
Sent: Wednesday, February 8, 2023 8:52 AMQuoted 34 lines · Parav Pandit
Quoted 34 lines
From: Michael S. Tsirkin <[email protected]>
Sent: Wednesday, February 8, 2023 8:32 AMQuoted 29 lines · Parav Pandit
Quoted 2 lines
From: Heng Qi <[email protected]>
Sent: Tuesday, February 7, 2023 10:25 PM[..]
Quoted 24 lines
Quoted 7 lines
Quoted 2 lines
Do you think we need both
hash_typesandhash_tunnel_types?In struct
virtio_net_configwe need two fields. a.supported_hash_types(already exists) b.supported_hash_tunnel_type
-> bitmap indicating for which outer headers, inner hash -> calculation issupported.
Thanks for the suggestion, we seem to have reached an agreement.
Quoted 4 lines
In struct
virtio_net_hdrwe need two fields.
a.hash_report(already exists) b.hash_tunnel_type8 bits -> absolute value indicating which outer headerexists when inner header hash calculated.
Quoted 3 lines
You already have it in your patch named as
hash_report_tunnel. May be better to name ashash_report_tunnel_typeto make it clearer that itstype.
Sure.
Thanks for your reply.
Quoted 7 lines
Quoted 6 lines
Quoted 1 line
I had one last question. Why do we need to inform the
hash_report_tunnel_typeof the outer header in thevirtio_net_hdr?Quoted 1 line
Is this for debug? Or is there a use case that will process this value?
Well we have
hash_reportwhich is kind of similar (and also kind of pointless but I think it's there because WHQL wants it).
Hash_reportis useful. It tellshash_valueis in which namespace (ipv4-tcp/ipv4udp etc).
Quoted 6 lines
OS can use this value to find tcp connection in a given namespace.
Quoted 3 lines
Maybe we can steal some bits
from there instead of a new field?I do not have problem adding extra bits. I just don't find that just telling that
its vxlan or nvgre to the OS is useful.
Quoted 1 line
If OS needs to know about outer header details, it needs to know the VNI
information than just telling vxlan.
This does make sense.
Quoted 4 lines
Quoted 4 lines
I have a follow up question though: are we only hashing the inner header or both inner and outer header? Somewhat confused on this.
Quoted 7 lines
Quoted 1 line
I understood as inner header. But worth to describe it. May be there. Need to
read v8 patch.
Hmm. I just realized that there's a security problem with hashing just the inner
header: it allow users inside the tunnel control queueing outside. By observing packet loss some information leaks between tunnels.
Ah I know now.
We are leaking outer header information inside the virtio net hdr, and outer header might be already stripped off by a different entity.
I think the use case here is it's the same sw entity that owns the virtio net device does the encap/decap too.
Quoted 66 lines · Parav Pandit
Quoted 66 lines
From: Michael S. Tsirkin <[email protected]>
Sent: Wednesday, February 8, 2023 8:52 AMQuoted 34 lines · Parav Pandit
Quoted 34 lines
From: Michael S. Tsirkin <[email protected]>
Sent: Wednesday, February 8, 2023 8:32 AMQuoted 29 lines · Parav Pandit
Quoted 2 lines
From: Heng Qi <[email protected]>
Sent: Tuesday, February 7, 2023 10:25 PM[..]
Quoted 24 lines
Quoted 7 lines
Quoted 2 lines
Do you think we need both
hash_typesandhash_tunnel_types?In struct
virtio_net_configwe need two fields. a.supported_hash_types(already exists) b.supported_hash_tunnel_type
-> bitmap indicating for which outer headers, inner hash -> calculation issupported.
Thanks for the suggestion, we seem to have reached an agreement.
Quoted 4 lines
In struct
virtio_net_hdrwe need two fields.
a.hash_report(already exists) b.hash_tunnel_type8 bits -> absolute value indicating which outer headerexists when inner header hash calculated.
Quoted 3 lines
You already have it in your patch named as
hash_report_tunnel. May be better to name ashash_report_tunnel_typeto make it clearer that itstype.
Sure.
Thanks for your reply.
Quoted 7 lines
Quoted 6 lines
Quoted 1 line
I had one last question. Why do we need to inform the
hash_report_tunnel_typeof the outer header in thevirtio_net_hdr?Quoted 1 line
Is this for debug? Or is there a use case that will process this value?
Well we have
hash_reportwhich is kind of similar (and also kind of pointless but I think it's there because WHQL wants it).
Hash_reportis useful. It tellshash_valueis in which namespace (ipv4-tcp/ipv4udp etc).
Quoted 6 lines
OS can use this value to find tcp connection in a given namespace.
Quoted 3 lines
Maybe we can steal some bits
from there instead of a new field?I do not have problem adding extra bits. I just don't find that just telling that
its vxlan or nvgre to the OS is useful.
Quoted 1 line
If OS needs to know about outer header details, it needs to know the VNI
information than just telling vxlan.
This does make sense.
Quoted 5 lines
Quoted 4 lines
I have a follow up question though: are we only hashing the inner header or both inner and outer header? Somewhat confused on this.
I understood as inner header. But worth to describe it. May be there. Need to
read v8 patch.
Quoted 8 lines
Quoted 4 lines
Hmm. I just realized that there's a security problem with hashing just the inner
header: it allow users inside the tunnel control queueing outside. By observing packet loss some information leaks between tunnels.Ah I know now.
We are leaking outer header information inside the virtio net hdr, and outer header might be already stripped off by a different entity.I think the use case here is it's the same sw entity that owns the virtio net device does the encap/decap too.
No not exactly, we are leaking info between encap tunnels.
Signature · 1 line
MST
content.tex \subsection{Feature bits}\label{sec:Device Types / Network Device / Feature bits · line 3089
Quoted 12 lines
From: [email protected] <[email protected]>
Sent: Wednesday, January 4, 2023 2:14 AMIf the tunnel is used to encapsulate the packets, the hash calculated using the outer header of the receive packets is always fixed for the same flow packets, i.e. they will be steered to the same receive queue.
+\item[VIRTIO_NET_F_HASH_TUNNEL(52)] Device supports inner
content.tex +2
-0+ header hash for GRE, VXLAN and GENEVE tunnel-encapsulated packets. +
A device may not support all 3 at the same time.
Please remove mentioning tunneling protocols description from here. Just say device support inner header hash ...
An additional bit map somewhere else should say supported hash over different tunneling types.
[...]
Quoted 112 lines
(quoted diff) +4
-0+The device calculates the hash on the inner IPv4 packet of an +encapsulated packet according to 'Enabled hash types' bitmask as follows: +\begin{itemize} + \item If VIRTIO_NET_HASH_TYPE_TCPv4 is set and the encapsulated packethas an inner
content.tex +8
-0+ TCPv4 header, the hash is calculated over the following fields: + \begin{itemsize} + \item inner Source IP address + \item inner Destination IP address + \item inner Source TCP port + \item inner Destination TCP port + \end{itemsize} + \item Else if VIRTIO_NET_HASH_TYPE_UDPv4 is set and the encapsulatedpacket has an
content.tex +8
-0+ inner UDPv4 header, the hash is calculated over the following fields: + \begin{itemsize} + \item inner Source IP address + \item inner Destination IP address + \item inner Source UDP port + \item inner Destination UDP port + \end{itemize} + \item Else if VIRTIO_NET_HASH_TYPE_IPv4 is set, the hash is calculated overthe
content.tex +9
-0+ following fields: + \begin{itemsize} + \item inner Source IP address + \item inner Destination IP address + \end{itemsize} + \item Else the device does not calculate the hash \end{itemize} + +The device calculates the hash on the inner IPv6 packet without an +extension header of an encapsulated packet according to 'Enabled hash types'bitmask as follows:
+\begin{itemize}
+ \item IfVIRTIO_NET_HASH_TYPE_TCPv6is set and the encapsulated packet has an innercontent.tex +8
-0+ TCPv6 header, the hash is calculated over the following fields: + \begin{itemsize} + \item inner Source IPv6 address + \item inner Destination IPv6 address + \item inner Source TCP port + \item inner Destination TCP port + \end{itemsize} + \item Else if VIRTIO_NET_HASH_TYPE_UDPv6 is set and the encapsulatedpacket has an
content.tex +8
-0+ inner UDPv6 header, the hash is calculated over the following fields: + \begin{itemsize} + \item inner Source IPv6 address + \item inner Destination IPv6 address + \item inner Source UDP port + \item inner Destination UDP port + \end{itemize} + \item Else if VIRTIO_NET_HASH_TYPE_IPv6 is set, the hash is calculated overthe
content.tex +9
-0+ following fields: + \begin{itemsize} + \item inner Source IPv6 address + \item inner Destination IPv6 address + \end{itemsize} + \item Else the device does not calculate the hash \end{itemize} + +The device calculates the hash on the inner IPv6 packet with an +extension header of an encapsulated packet according to 'Enabled hash types'bitmask as follows:
+\begin{itemsize}
+ \item IfVIRTIO_NET_HASH_TYPE_TCP_EXis set and the encapsulated packet has an innercontent.tex +3
-0+ TCPv6 header, the hash is calculated over the following fields: + \begin{itemize} + \item Home address from the home address option in the inner IPv6destination
content.tex +3
-0+ options header. If the inner extension header is not present, use the + inner Source IPv6 address. + \item IPv6 address that is contained in the Routing-Header-Type-2 fromthe
content.tex +6
-0+ associated inner extension header. If the inner extension header is not + present, use the inner Destination IPv6 address. + \item inner Source TCP port + \item inner Destination TCP port + \end{itemize} + \item Else if VIRTIO_NET_HASH_TYPE_UDP_EX is set and the encapsulatedpacket has an inner
content.tex +3
-0+ UDPv6 header, the hash is calculated over the following fields: + \begin{itemsize} + \item Home address from the home address option in the inner IPv6destination
content.tex +3
-0+ options header. If the inner extension header is not present, use the + inner Source IPv6 address. + \item IPv6 address that is contained in the Routing-Header-Type-2 fromthe
content.tex +6
-0+ associated inner extension header. If the inner extension header is not + present, use the inner Destination IPv6 address. + \item inner Source UDP port + \item inner Destination UDP port + \end{itemize} + \item Else if VIRTIO_NET_HASH_TYPE_IP_EX is set, the hash is calculatedover the
content.tex +3
-0+ following fields: + \begin{itemsize} + \item Home address from the home address option in the inner IPv6destination
content.tex +3
-0+ options header. If the inner extension header is not present, use the + inner Source IPv6 address. + \item IPv6 address that is contained in the Routing-Header-Type-2 fromthe
content.tex +4
-0+ associated inner extension header. If the inner extension header is not + present, use the inner Destination IPv6 address. + \end{itemize} + \item Else skip the inner IPv6 extension header and calculate the innerheader hash as
(quoted diff) +1
-0+ defined for an encapsulated packet whose inner payload is an IPv6packet without
content.tex \subsubsection{Processing of Incoming Packets}\label{sec:Device Types / Network · line 4134
Quoted 3 lines
content.tex +3
-0+ an extension header. +\end{itemsize} +
With the inclusion of tunnel outer header, it doesn't need to redefine the hashing for inner packets which is exactly same without the tunnel. hash tunnel capability only indicates that hashing is done on the inner packets as_before.
Quoted 9 lines · Parav Pandit
Quoted 8 lines
From: [email protected] <[email protected]>
Sent: Wednesday, January 4, 2023 2:14 AMIf the tunnel is used to encapsulate the packets, the hash calculated using the outer header of the receive packets is always fixed for the same flow packets, i.e. they will be steered to the same receive queue.
Quoted 6 lines
content.tex \subsection{Feature bits}\label{sec:Device Types / Network Device / Feature bits · line 3089
Quoted 3 lines
+\item[VIRTIO_NET_F_HASH_TUNNEL(52)] Device supports inner
content.tex +2
-0+ header hash for GRE, VXLAN and GENEVE tunnel-encapsulated packets. +A device may not support all 3 at the same time.
Please remove mentioning tunneling protocols description from here. Just say device support inner header hash ...
Sorry for the late reply due to vacation.
Good idea, Michael suggested doing the same. But we also discussed this issue:
Early, we used a feature bit to force devices to support GRE and VXLAN (see
https://lists.oasis-open.org/archives/virtio-dev/202211/msg00183.html ).
Later Jason suggested to use VIRTIO_NET_F_HASH_TUNNEL instead of VIRTIO_NET_F_HASH_GRE_VXLAN_GENEVE_INNER (see
https://lists.oasis-open.org/archives/virtio-dev/202212/msg00014.html ). Now Michael proposes to remove this list (see
https://lists.oasis-open.org/archives/virtio-dev/202301/msg00079.html ), because the migration is based on the feature bit and hash types to determine whether the live migration is successful.
Quoted 3 lines
An additional bit map somewhere else should say supported hash over different tunneling types.
Yes, we use \field{supported_hash_types} to declare supported hash types.
Quoted 112 lines
[...]
Quoted 110 lines
(quoted diff) +4
-0+The device calculates the hash on the inner IPv4 packet of an +encapsulated packet according to 'Enabled hash types' bitmask as follows: +\begin{itemize} + \item If VIRTIO_NET_HASH_TYPE_TCPv4 is set and the encapsulated packethas an inner
content.tex +8
-0+ TCPv4 header, the hash is calculated over the following fields: + \begin{itemsize} + \item inner Source IP address + \item inner Destination IP address + \item inner Source TCP port + \item inner Destination TCP port + \end{itemsize} + \item Else if VIRTIO_NET_HASH_TYPE_UDPv4 is set and the encapsulatedpacket has an
content.tex +8
-0+ inner UDPv4 header, the hash is calculated over the following fields: + \begin{itemsize} + \item inner Source IP address + \item inner Destination IP address + \item inner Source UDP port + \item inner Destination UDP port + \end{itemize} + \item Else if VIRTIO_NET_HASH_TYPE_IPv4 is set, the hash is calculated overthe
content.tex +9
-0+ following fields: + \begin{itemsize} + \item inner Source IP address + \item inner Destination IP address + \end{itemsize} + \item Else the device does not calculate the hash \end{itemize} + +The device calculates the hash on the inner IPv6 packet without an +extension header of an encapsulated packet according to 'Enabled hash types'bitmask as follows:
+\begin{itemize}
+ \item IfVIRTIO_NET_HASH_TYPE_TCPv6is set and the encapsulated packet has an innercontent.tex +8
-0+ TCPv6 header, the hash is calculated over the following fields: + \begin{itemsize} + \item inner Source IPv6 address + \item inner Destination IPv6 address + \item inner Source TCP port + \item inner Destination TCP port + \end{itemsize} + \item Else if VIRTIO_NET_HASH_TYPE_UDPv6 is set and the encapsulatedpacket has an
content.tex +8
-0+ inner UDPv6 header, the hash is calculated over the following fields: + \begin{itemsize} + \item inner Source IPv6 address + \item inner Destination IPv6 address + \item inner Source UDP port + \item inner Destination UDP port + \end{itemize} + \item Else if VIRTIO_NET_HASH_TYPE_IPv6 is set, the hash is calculated overthe
content.tex +9
-0+ following fields: + \begin{itemsize} + \item inner Source IPv6 address + \item inner Destination IPv6 address + \end{itemsize} + \item Else the device does not calculate the hash \end{itemize} + +The device calculates the hash on the inner IPv6 packet with an +extension header of an encapsulated packet according to 'Enabled hash types'bitmask as follows:
+\begin{itemsize}
+ \item IfVIRTIO_NET_HASH_TYPE_TCP_EXis set and the encapsulated packet has an innercontent.tex +3
-0+ TCPv6 header, the hash is calculated over the following fields: + \begin{itemize} + \item Home address from the home address option in the inner IPv6destination
content.tex +3
-0+ options header. If the inner extension header is not present, use the + inner Source IPv6 address. + \item IPv6 address that is contained in the Routing-Header-Type-2 fromthe
content.tex +6
-0+ associated inner extension header. If the inner extension header is not + present, use the inner Destination IPv6 address. + \item inner Source TCP port + \item inner Destination TCP port + \end{itemize} + \item Else if VIRTIO_NET_HASH_TYPE_UDP_EX is set and the encapsulatedpacket has an inner
content.tex +3
-0+ UDPv6 header, the hash is calculated over the following fields: + \begin{itemsize} + \item Home address from the home address option in the inner IPv6destination
content.tex +3
-0+ options header. If the inner extension header is not present, use the + inner Source IPv6 address. + \item IPv6 address that is contained in the Routing-Header-Type-2 fromthe
content.tex +6
-0+ associated inner extension header. If the inner extension header is not + present, use the inner Destination IPv6 address. + \item inner Source UDP port + \item inner Destination UDP port + \end{itemize} + \item Else if VIRTIO_NET_HASH_TYPE_IP_EX is set, the hash is calculatedover the
content.tex +3
-0+ following fields: + \begin{itemsize} + \item Home address from the home address option in the inner IPv6destination
content.tex +3
-0+ options header. If the inner extension header is not present, use the + inner Source IPv6 address. + \item IPv6 address that is contained in the Routing-Header-Type-2 fromthe
content.tex +4
-0+ associated inner extension header. If the inner extension header is not + present, use the inner Destination IPv6 address. + \end{itemize} + \item Else skip the inner IPv6 extension header and calculate the innerheader hash as
Quoted 9 lines
content.tex \subsubsection{Processing of Incoming Packets}\label{sec:Device Types / Network · line 4134
Quoted 5 lines
(quoted diff) +1
-0+ defined for an encapsulated packet whose inner payload is an IPv6packet without
content.tex +3
-0+ an extension header. +\end{itemsize} +With the inclusion of tunnel outer header, it doesn't need to redefine the hashing for inner packets which is exactly same without the tunnel. hash tunnel capability only indicates that hashing is done on the inner packets
as_before.
This seems like a trade-off, and I can get rid of this if the simple statement "computing the hash for the inner header of the tunnel packet is the same as without tunnel" is also clear to the reader.
What do you think? Cc Jason and Michael.
Thanks.
Quoted 15 lines
This publicly archived list offers a means to provide input to the OASIS Virtual I/O Device (VIRTIO) TC.
In order to verify user consent to the Feedback License terms and to minimize spam in the list archive, subscription is required before posting.
Subscribe: [email protected]
Unsubscribe: [email protected]
List help: [email protected]
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/
Quoted 20 lines
From: Heng Qi <[email protected]>
Sent: Tuesday, January 31, 2023 12:57 AMQuoted 9 lines · Parav Pandit
Quoted 9 lines
From: [email protected]
<[email protected]>
Sent: Wednesday, January 4, 2023 2:14 AMIf the tunnel is used to encapsulate the packets, the hash calculated using the outer header of the receive packets is always fixed for the same flow packets, i.e. they will be steered to the same
receive queue.
Quoted 4 lines
Quoted 4 lines
+\item[VIRTIO_NET_F_HASH_TUNNEL(52)] Device supports inner
content.tex +2
-0+ header hash for GRE, VXLAN and GENEVE tunnel-encapsulated packets. +
Quoted 8 lines
Quoted 3 lines
A device may not support all 3 at the same time.
Please remove mentioning tunneling protocols description from here. Just say device support inner header hash ...Sorry for the late reply due to vacation.
Good idea, Michael suggested doing the same. But we also discussed this issue:
Early, we used a feature bit to force devices to support GRE and VXLAN
..
There is no need to force the device.
LM will work as it will use both he values - the feature bit and the supported/negotiated hash type in the config space.
Quoted 5 lines
Quoted 1 line
An additional bit map somewhere else should say supported hash over
different tunneling types.
Quoted 1 line
Yes, we use \field{supported_hash_types} to declare supported hash types.
Nice. So yes, just remove the description from the feature bits. [..]
Quoted 4 lines
Quoted 2 lines
With the inclusion of tunnel outer header, it doesn't need to redefine the
hashing for inner packets which is exactly same without the tunnel.
Quoted 1 line
hash tunnel capability only indicates that hashing is done on the inner packets
Quoted 8 lines
as_before.Quoted 1 line
This seems like a trade-off, and I can get rid of this if the simple statement "computing the hash for the inner header of the tunnel packet is the same as without tunnel"
is also clear to the reader.
What do you think? Cc Jason and Michael.
Not sure I follow your comment.
Inclusion of outer header has zero change in inner header hash.
The data structure etc is just fine in the patch you proposed.
So, there is no need to redefine all over it again.
Only description need to be simplified as below.
Please just change the text wording to indicate that when tunnel feature is negotiated, and if the hashing is performed by the device on the inner header, hash_report_tunnel will contain the valid outer tunnel type information.
Also, this patch is adding two functionalities.
1. Inner header hash calculation of existing already defined hash types
2. outer header hash for new type for GRE,VXLAN,GENEVE.
#1 should be in 1st patch.
#2 should be in 2nd patch.
This is better to review.
Overall useful features. Thanks for adding.
This publicly archived list offers a means to provide input to the OASIS Virtual I/O Device (VIRTIO) TC.
In order to verify user consent to the Feedback License terms and to minimize spam in the list archive, subscription is required before posting.
Subscribe: [email protected]
Unsubscribe: [email protected]
List help: [email protected]
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/
Quoted 59 lines · Parav Pandit
Quoted 20 lines
From: Heng Qi <[email protected]>
Sent: Tuesday, January 31, 2023 12:57 AMQuoted 9 lines · Parav Pandit
Quoted 9 lines
From: [email protected]
<[email protected]>
Sent: Wednesday, January 4, 2023 2:14 AMIf the tunnel is used to encapsulate the packets, the hash calculated using the outer header of the receive packets is always fixed for the same flow packets, i.e. they will be steered to the same
receive queue.
Quoted 4 lines
Quoted 4 lines
+\item[VIRTIO_NET_F_HASH_TUNNEL(52)] Device supports inner
content.tex +2
-0+ header hash for GRE, VXLAN and GENEVE tunnel-encapsulated packets. +Quoted 8 lines
Quoted 3 lines
A device may not support all 3 at the same time.
Please remove mentioning tunneling protocols description from here. Just say device support inner header hash ...Sorry for the late reply due to vacation.
Good idea, Michael suggested doing the same. But we also discussed this issue:
Early, we used a feature bit to force devices to support GRE and VXLAN..
There is no need to force the device.
LM will work as it will use both he values - the feature bit and the supported/negotiated hash type in the config space.Quoted 5 lines
Quoted 1 line
An additional bit map somewhere else should say supported hash over
different tunneling types.
Quoted 1 line
Yes, we use \field{supported_hash_types} to declare supported hash types.
Nice. So yes, just remove the description from the feature bits. [..]
Quoted 4 lines
Quoted 2 lines
With the inclusion of tunnel outer header, it doesn't need to redefine the
hashing for inner packets which is exactly same without the tunnel.
Quoted 1 line
hash tunnel capability only indicates that hashing is done on the inner packets
Quoted 8 lines
as_before.Quoted 1 line
This seems like a trade-off, and I can get rid of this if the simple statement "computing the hash for the inner header of the tunnel packet is the same as without tunnel"
is also clear to the reader.
What do you think? Cc Jason and Michael.Not sure I follow your comment.
Inclusion of outer header has zero change in inner header hash.The data structure etc is just fine in the patch you proposed.
So, there is no need to redefine all over it again.
Only description need to be simplified as below.
Please just change the text wording to indicate that when tunnel feature is negotiated, and if the hashing is performed by the device on the inner header,hash_report_tunnelwill contain the valid outer tunnel type information.
Quoted 6 lines
Also, this patch is adding two functionalities.
1. Inner header hash calculation of existing already defined hash types
2. outer header hash for new type for GRE,VXLAN,GENEVE.
#1 should be in 1st patch.
#2 should be in 2nd patch.
This is better to review.
Oh, that's an interesting take. Do you feel we need more separate hash type bits then?
Quoted 1 line
Overall useful features. Thanks for adding.
Quoted 59 lines · Parav Pandit
Quoted 20 lines
From: Heng Qi <[email protected]>
Sent: Tuesday, January 31, 2023 12:57 AMQuoted 9 lines · Parav Pandit
Quoted 9 lines
From: [email protected]
<[email protected]>
Sent: Wednesday, January 4, 2023 2:14 AMIf the tunnel is used to encapsulate the packets, the hash calculated using the outer header of the receive packets is always fixed for the same flow packets, i.e. they will be steered to the same
receive queue.
Quoted 4 lines
Quoted 4 lines
+\item[VIRTIO_NET_F_HASH_TUNNEL(52)] Device supports inner
content.tex +2
-0+ header hash for GRE, VXLAN and GENEVE tunnel-encapsulated packets. +Quoted 8 lines
Quoted 3 lines
A device may not support all 3 at the same time.
Please remove mentioning tunneling protocols description from here. Just say device support inner header hash ...Sorry for the late reply due to vacation.
Good idea, Michael suggested doing the same. But we also discussed this issue:
Early, we used a feature bit to force devices to support GRE and VXLAN..
There is no need to force the device.
LM will work as it will use both he values - the feature bit and the supported/negotiated hash type in the config space.Quoted 5 lines
Quoted 1 line
An additional bit map somewhere else should say supported hash over
different tunneling types.
Quoted 1 line
Yes, we use \field{supported_hash_types} to declare supported hash types.
Nice. So yes, just remove the description from the feature bits. [..]
Quoted 4 lines
Quoted 2 lines
With the inclusion of tunnel outer header, it doesn't need to redefine the
hashing for inner packets which is exactly same without the tunnel.
Quoted 1 line
hash tunnel capability only indicates that hashing is done on the inner packets
Quoted 8 lines
as_before.Quoted 1 line
This seems like a trade-off, and I can get rid of this if the simple statement "computing the hash for the inner header of the tunnel packet is the same as without tunnel"
is also clear to the reader.
What do you think? Cc Jason and Michael.Not sure I follow your comment.
Inclusion of outer header has zero change in inner header hash.The data structure etc is just fine in the patch you proposed.
So, there is no need to redefine all over it again.
Only description need to be simplified as below.
Please just change the text wording to indicate that when tunnel feature is negotiated, and if the hashing is performed by the device on the inner header,hash_report_tunnelwill contain the valid outer tunnel type information.
Quoted 6 lines
Also, this patch is adding two functionalities.
1. Inner header hash calculation of existing already defined hash types
2. outer header hash for new type for GRE,VXLAN,GENEVE.
#1 should be in 1st patch.
#2 should be in 2nd patch.
This is better to review.
Parav, you come to this discussion pretty late. Asking to split up the patch when it's v1/v2 is ok. Asking after others have already reviewed v6 is not you are making review easier for yourself but re-review harder for others who already have a mind map of the patch.
In this case unless we really want to enable these separately (and frankly I don't see a good reason to) then splitting it up makes review more confusing.
Quoted 1 line
Overall useful features. Thanks for adding.
Quoted 12 lines
From: [email protected] <[email protected] open.org> On Behalf Of Michael S. Tsirkin
Sent: Wednesday, February 1, 2023 1:57 AMQuoted 6 lines
Also, this patch is adding two functionalities.
1. Inner header hash calculation of existing already defined hash types 2. outer header hash for new type for GRE,VXLAN,GENEVE. #1 should be in 1st patch.
#2 should be in 2nd patch.
This is better to review.
Quoted 8 lines
Parav, you come to this discussion pretty late. Asking to split up the patch when it's v1/v2 is ok. Asking after others have already reviewed v6 is not you are making review easier for yourself but re-review harder for others who already have a mind map of the patch.
In this case unless we really want to enable these separately (and frankly I don't see a good reason to) then splitting it up makes review more confusing.
No. There is no need to enable it separately.
It was hard to parse new inner type decoding addition which has close to zero relation to outer headers.
As you say, it has some history, I don't have strong opinion to split.
But going forward in subsequent work, it is better to see logical changes in multiple patches.
Quoted 14 lines · Parav Pandit
Quoted 14 lines
From: [email protected] <[email protected] open.org> On Behalf Of Michael S. Tsirkin
Sent: Wednesday, February 1, 2023 1:57 AMQuoted 6 lines
Also, this patch is adding two functionalities.
1. Inner header hash calculation of existing already defined hash types 2. outer header hash for new type for GRE,VXLAN,GENEVE. #1 should be in 1st patch.
#2 should be in 2nd patch.
This is better to review.Parav, you come to this discussion pretty late. Asking to split up the patch when it's v1/v2 is ok. Asking after others have already reviewed v6 is not you are making review easier for yourself but re-review harder for others who already have a mind map of the patch.
Quoted 8 lines
Quoted 3 lines
In this case unless we really want to enable these separately (and frankly I don't see a good reason to) then splitting it up makes review more confusing.
No. There is no need to enable it separately.
It was hard to parse new inner type decoding addition which has close to zero relation to outer headers.As you say, it has some history, I don't have strong opinion to split.
But going forward in subsequent work, it is better to see logical changes in multiple patches.
It seems that we don't need to emphasize the outer header hash, which is the same behavior as usual when the inner header hash is not raised.
Thanks.
This publicly archived list offers a means to provide input to the OASIS Virtual I/O Device (VIRTIO) TC.
In order to verify user consent to the Feedback License terms and to minimize spam in the list archive, subscription is required before posting.
Subscribe: [email protected]
Unsubscribe: [email protected]
List help: [email protected]
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/
Quoted 24 lines · Parav Pandit
Quoted 24 lines
From: Heng Qi <[email protected]>
Sent: Tuesday, January 31, 2023 12:57 AMQuoted 9 lines · Parav Pandit
Quoted 9 lines
From: [email protected]
<[email protected]>
Sent: Wednesday, January 4, 2023 2:14 AMIf the tunnel is used to encapsulate the packets, the hash calculated using the outer header of the receive packets is always fixed for the same flow packets, i.e. they will be steered to the same
receive queue.
Quoted 7 lines
content.tex \subsection{Feature bits}\label{sec:Device Types / Network Device / Feature bits · line 3089
Quoted 4 lines
+\item[VIRTIO_NET_F_HASH_TUNNEL(52)] Device supports inner
content.tex +2
-0+ header hash for GRE, VXLAN and GENEVE tunnel-encapsulated packets. +A device may not support all 3 at the same time.
Please remove mentioning tunneling protocols description from here. Just say device support inner header hash ...
Quoted 7 lines
Quoted 4 lines
Sorry for the late reply due to vacation.
Good idea, Michael suggested doing the same. But we also discussed this issue:
Early, we used a feature bit to force devices to support GRE and VXLAN..
There is no need to force the device.
LM will work as it will use both he values - the feature bit and the supported/negotiated hash type in the config space.
Yes, that's exactly what we do now.
Quoted 7 lines
Quoted 6 lines
Quoted 2 lines
An additional bit map somewhere else should say supported hash over
different tunneling types.
Quoted 1 line
Yes, we use \field{supported_hash_types} to declare supported hash types.
Nice. So yes, just remove the description from the feature bits.
The next version will see this modification.
Quoted 13 lines
[..]
Quoted 4 lines
Quoted 2 lines
With the inclusion of tunnel outer header, it doesn't need to redefine the
hashing for inner packets which is exactly same without the tunnel.
Quoted 1 line
hash tunnel capability only indicates that hashing is done on the inner packets
Quoted 8 lines
as_before.Quoted 1 line
This seems like a trade-off, and I can get rid of this if the simple statement "computing the hash for the inner header of the tunnel packet is the same as without tunnel"
is also clear to the reader.
What do you think? Cc Jason and Michael.
Quoted 8 lines
Not sure I follow your comment.
Inclusion of outer header has zero change in inner header hash.The data structure etc is just fine in the patch you proposed.
So, there is no need to redefine all over it again.
Only description need to be simplified as below.
Please just change the text wording to indicate that when tunnel feature is negotiated, and if the hashing is performed by the device on the inner header,hash_report_tunnelwill contain the valid outer tunnel type information.
OK, I'll take your advice and make appropriate changes.
Thanks.
Quoted 8 lines
Also, this patch is adding two functionalities.
1. Inner header hash calculation of existing already defined hash types
2. outer header hash for new type for GRE,VXLAN,GENEVE.
#1 should be in 1st patch.
#2 should be in 2nd patch.
This is better to review.Overall useful features. Thanks for adding.
This publicly archived list offers a means to provide input to the OASIS Virtual I/O Device (VIRTIO) TC.
In order to verify user consent to the Feedback License terms and to minimize spam in the list archive, subscription is required before posting.
Subscribe: [email protected]
Unsubscribe: [email protected]
List help: [email protected]
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/