Skip to contentSkip to conversation

virtio-net: support inner header hash

PATCH Superseded Heng Qi 5 participants 54 messages Jan 4, 2023 – Feb 11, 2023

v7 of 21 Superseded Latest is v21 →

lore
Share & tools
lore

Heng Qi

author
#0 lore raw

PATCH v7 1/1 virtio-net: support inner header hash

If the tunnel is used to encapsulate the packets, the hash calculated using the outer header of the receive packets is always fixed for the same flow packets, i.e. they will be steered to the same receive queue.

We add a feature bit VIRTIO_NET_F_HASH_TUNNEL and related bitmasks in \field{hash_types}, which instructs the device to calculate the hash using the inner headers of tunnel-encapsulated packets. Besides, values in \field{hash_report_tunnel} are added to report tunnel types.

---
v6:
1. Modify the wording of some sentences for clarity. @Michael S. Tsirkin
2. Fix some syntax issues. @Michael S. Tsirkin

v5:
1. Fix some syntax and capitalization issues. @Michael S. Tsirkin
2. Use encapsulated/encaptulation uniformly. @Michael S. Tsirkin
3. Move the links to introduction section. @Michael S. Tsirkin
4. Clarify some sentences. @Michael S. Tsirkin

v4:
1. Clarify some paragraphs. @Cornelia Huck
2. Fix the u8 type. @Cornelia Huck

v3:
1. Rename VIRTIO_NET_F_HASH_GRE_VXLAN_GENEVE_INNER to VIRTIO_NET_F_HASH_TUNNEL. @Jason Wang
2. Make things clearer. @Jason Wang @Michael S. Tsirkin
3. Keep the possibility to use inner hash for automatic receive steering. @Jason Wang
4. Add the "Tunnel packet" paragraph to avoid repeating the GRE etc. many times. @Michael S. Tsirkin

v2:
1. Add a feature bit for GRE/VXLAN/GENEVE inner hash. @Jason Wang
2. Chang \field{hash_tunnel} to \field{hash_report_tunnel}. @Jason Wang, @Michael S. Tsirkin

v1:
1. Remove the patch for the bitmask fix. @Michael S. Tsirkin
2. Clarify some paragraphs. @Jason Wang
3. Add \field{hash_tunnel} and VIRTIO_NET_HASH_REPORT_GRE. @Yuri Benditovich

 content.tex      | 191 +++++++++++++++++++++++++++++++++++++++++++++--
 introduction.tex |  19 +++++
 2 files changed, 203 insertions(+), 7 deletions(-)
content.tex +184 -7
index e863709..7845f6c 100644
@@ -3084,6 +3084,9 @@ \subsection{Feature bits}\label{sec:Device Types / Network Device / Feature bits
 \item[VIRTIO_NET_F_CTRL_MAC_ADDR(23)] Set MAC address through control
     channel.
 
+\item[VIRTIO_NET_F_HASH_TUNNEL(52)] Device supports inner
+    header hash for GRE, VXLAN and GENEVE tunnel-encapsulated packets.
+
 \item[VIRTIO_NET_F_NOTF_COAL(53)] Device supports notifications coalescing.
 
 \item[VIRTIO_NET_F_GUEST_USO4 (54)] Driver can receive USOv4 packets.
@@ -3095,7 +3098,8 @@ \subsection{Feature bits}\label{sec:Device Types / Network Device / Feature bits
  to several segments when each of these smaller packets has UDP header.
 
 \item[VIRTIO_NET_F_HASH_REPORT(57)] Device can report per-packet hash
-    value and a type of calculated hash.
+    value, a type of calculated hash, and, if VIRTIO_NET_F_HASH_TUNNEL
+    is negotiated, an encapsulation packet type.
 
 \item[VIRTIO_NET_F_GUEST_HDRLEN(59)] Driver can provide the exact \field{hdr_len}
     value. Device benefits from knowing the exact header length.
@@ -3140,6 +3144,7 @@ \subsubsection{Feature bit requirements}\label{sec:Device Types / Network Device
 \item[VIRTIO_NET_F_NOTF_COAL] Requires VIRTIO_NET_F_CTRL_VQ.
 \item[VIRTIO_NET_F_RSC_EXT] Requires VIRTIO_NET_F_HOST_TSO4 or VIRTIO_NET_F_HOST_TSO6.
 \item[VIRTIO_NET_F_RSS] Requires VIRTIO_NET_F_CTRL_VQ.
+\item[VIRTIO_NET_F_HASH_TUNNEL] Requires VIRTIO_NET_F_CTRL_VQ.
 \end{description}
 
 \subsubsection{Legacy Interface: Feature bits}\label{sec:Device Types / Network Device / Feature bits / Legacy Interface: Feature bits}
@@ -3386,7 +3391,8 @@ \subsection{Device Operation}\label{sec:Device Types / Network Device / Device O
         le16 num_buffers;
         le32 hash_value;        (Only if VIRTIO_NET_F_HASH_REPORT negotiated)
         le16 hash_report;       (Only if VIRTIO_NET_F_HASH_REPORT negotiated)
-        le16 padding_reserved;  (Only if VIRTIO_NET_F_HASH_REPORT negotiated)
+        u8 hash_report_tunnel;  (Only if VIRTIO_NET_F_HASH_REPORT negotiated, only valid of VIRTIO_NET_F_HASH_TUNNEL negotiated)
+        u8 padding_reserved;    (Only if VIRTIO_NET_F_HASH_REPORT negotiated)
 };
 \end{lstlisting}
 
@@ -3837,7 +3843,8 @@ \subsubsection{Processing of Incoming Packets}\label{sec:Device Types / Network
 A device attempts to calculate a per-packet hash in the following cases:
 \begin{itemize}
 \item The feature VIRTIO_NET_F_RSS was negotiated. The device uses the hash to determine the receive virtqueue to place incoming packets.
-\item The feature VIRTIO_NET_F_HASH_REPORT was negotiated. The device reports the hash value and the hash type with the packet.
+\item The feature VIRTIO_NET_F_HASH_REPORT was negotiated. The device reports the hash value and the hash type. If additionally
+VIRTIO_NET_F_HASH_TUNNEL was negotiated, the device reports the encapsulation type as well.
 \end{itemize}
 
 If the feature VIRTIO_NET_F_RSS was negotiated:
@@ -3863,8 +3870,36 @@ \subsubsection{Processing of Incoming Packets}\label{sec:Device Types / Network
  \ref{sec:Device Types / Network Device / Device Operation / Control Virtqueue / Automatic receive steering in multiqueue mode / Hash calculation}.
 \end{itemize}
 
+\subparagraph{Tunnel/Encapsulated packet}
+\label{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / Tunnel/Encapsulated packet}
+A tunnel packet is encapsulated from the original packet based on the tunneling
+protocol (only a single level of encapsulation is currently supported). The
+encapsulated packet contains an outer header and an inner header, and the device
+calculates the hash over either the inner header or the outer header.
+
+When the feature VIRTIO_NET_F_HASH_TUNNEL is negotiated and the corresponding
+encapsulation type is set in \field{hash_types}, the hash for a specific type of
+encapsulated packet is calculated over the inner as opposed to outer header.
+Supported encapsulation types are listed in \ref{sec:Device Types / Network Device /
+Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets /
+Supported/enabled hash types}.
+
+If both VIRTIO_NET_F_HASH_REPORT and VIRTIO_NET_F_HASH_TUNNEL are negotiated,
+the device can support inner hash calculation for \hyperref[intro:GRE]{[GRE]},
+\hyperref[intro:VXLAN]{[VXLAN]} and \hyperref[intro:GENEVE]{[GENEVE]}
+encapsulated packets, and the corresponding encapsulation type in \field{hash_types}
+is VIRTIO_NET_HASH_TYPE_{GRE, VXLAN, GENEVE}_INNER respectively. The value in
+\field{hash_report_tunnel} is VIRTIO_NET_HASH_REPORT_{NONE, GRE, VXLAN, GENEVE} respectively.
+
+If VIRTIO_NET_F_HASH_REPORT is negotiated but VIRTIO_NET_F_HASH_TUNNEL is not
+negotiated, the device calculates the hash over the outer header, and
+\field{hash_report} reports the hash type. \field{hash_report_tunnel}
+is no longer valid.
+
 \subparagraph{Supported/enabled hash types}
 \label{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / Supported/enabled hash types}
+This paragraph relies on definitions from \hyperref[intro:IP]{[IP]},
+\hyperref[intro:UDP]{[UDP]} and \hyperref[intro:TCP]{[TCP]}.
 Hash types applicable for IPv4 packets:
 \begin{lstlisting}
 #define VIRTIO_NET_HASH_TYPE_IPv4              (1 << 0)
@@ -3884,6 +3919,22 @@ \subsubsection{Processing of Incoming Packets}\label{sec:Device Types / Network
 #define VIRTIO_NET_HASH_TYPE_UDP_EX            (1 << 8)
 \end{lstlisting}
 
+If the feature VIRTIO_NET_F_HASH_TUNNEL is negotiated, the encapsulation
+hash type below indicates that the hash is calculated over the inner
+header of the encapsulated packet:
+Hash type applicable for inner payload of the gre-encapsulated packet
+\begin{lstlisting}
+#define VIRTIO_NET_HASH_TYPE_GRE_INNER         (1 << 9)
+\end{lstlisting}
+Hash type applicable for inner payload of the vxlan-encapsulated packet
+\begin{lstlisting}
+#define VIRTIO_NET_HASH_TYPE_VXLAN_INNER       (1 << 10)
+\end{lstlisting}
+Hash type applicable for inner payload of the geneve-encapsulated packet
+\begin{lstlisting}
+#define VIRTIO_NET_HASH_TYPE_GENEVE_INNER      (1 << 11)
+\end{lstlisting}
+
 \subparagraph{IPv4 packets}
 \label{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / IPv4 packets}
 The device calculates the hash on IPv4 packets according to 'Enabled hash types' bitmask as follows:
@@ -3975,15 +4026,125 @@ \subsubsection{Processing of Incoming Packets}\label{sec:Device Types / Network
 (see \ref{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / IPv6 packets without extension header}).
 \end{itemize}
 
+\subparagraph{Inner payload of an encapsulated packet}
+\label{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / Inner payload of the encapsulated packet}
+If the feature VIRTIO_NET_F_HASH_TUNNEL is negotiated and the corresponding
+encapsulation hash type is set in \field{hash_types}, the device calculates the
+inner header hash on an encapsulated packet (See \ref{sec:Device Types
+/ Network Device / Device Operation / Processing of Incoming Packets /
+Hash calculation for incoming packets / Tunnel/Encapsulated packet}) as follows:
+
+The device calculates the hash on the inner IPv4 packet of an encapsulated packet
+according to 'Enabled hash types' bitmask as follows:
+\begin{itemize}
+  \item If VIRTIO_NET_HASH_TYPE_TCPv4 is set and the encapsulated packet has an inner
+	TCPv4 header, the hash is calculated over the following fields:
+        \begin{itemsize}
+          \item inner Source IP address
+          \item inner Destination IP address
+          \item inner Source TCP port
+          \item inner Destination TCP port
+        \end{itemsize}
+  \item Else if VIRTIO_NET_HASH_TYPE_UDPv4 is set and the encapsulated packet has an
+	inner UDPv4 header, the hash is calculated over the following fields:
+        \begin{itemsize}
+          \item inner Source IP address
+          \item inner Destination IP address
+          \item inner Source UDP port
+          \item inner Destination UDP port
+        \end{itemize}
+  \item Else if VIRTIO_NET_HASH_TYPE_IPv4 is set, the hash is calculated over the
+	following fields:
+        \begin{itemsize}
+          \item inner Source IP address
+          \item inner Destination IP address
+        \end{itemsize}
+  \item Else the device does not calculate the hash
+\end{itemize}
+
+The device calculates the hash on the inner IPv6 packet without an extension header
+of an encapsulated packet according to 'Enabled hash types' bitmask as follows:
+\begin{itemize}
+  \item If VIRTIO_NET_HASH_TYPE_TCPv6 is set and the encapsulated packet has an inner
+	TCPv6 header, the hash is calculated over the following fields:
+        \begin{itemsize}
+          \item inner Source IPv6 address
+          \item inner Destination IPv6 address
+          \item inner Source TCP port
+          \item inner Destination TCP port
+        \end{itemsize}
+  \item Else if VIRTIO_NET_HASH_TYPE_UDPv6 is set and the encapsulated packet has an
+	inner UDPv6 header, the hash is calculated over the following fields:
+        \begin{itemsize}
+          \item inner Source IPv6 address
+          \item inner Destination IPv6 address
+          \item inner Source UDP port
+          \item inner Destination UDP port
+        \end{itemize}
+  \item Else if VIRTIO_NET_HASH_TYPE_IPv6 is set, the hash is calculated over the
+	following fields:
+        \begin{itemsize}
+          \item inner Source IPv6 address
+          \item inner Destination IPv6 address
+        \end{itemsize}
+  \item Else the device does not calculate the hash
+\end{itemize}
+
+The device calculates the hash on the inner IPv6 packet with an extension header
+of an encapsulated packet according to 'Enabled hash types' bitmask as follows:
+\begin{itemsize}
+  \item If VIRTIO_NET_HASH_TYPE_TCP_EX is set and the encapsulated packet has an inner
+	TCPv6 header, the hash is calculated over the following fields:
+        \begin{itemize}
+          \item Home address from the home address option in the inner IPv6 destination
+              options header. If the inner extension header is not present, use the
+              inner Source IPv6 address.
+          \item IPv6 address that is contained in the Routing-Header-Type-2 from the
+              associated inner extension header. If the inner extension header is not
+              present, use the inner Destination IPv6 address.
+          \item inner Source TCP port
+          \item inner Destination TCP port
+        \end{itemize}
+  \item Else if VIRTIO_NET_HASH_TYPE_UDP_EX is set and the encapsulated packet has an inner
+	UDPv6 header, the hash is calculated over the following fields:
+        \begin{itemsize}
+          \item Home address from the home address option in the inner IPv6 destination
+              options header. If the inner extension header is not present, use the
+              inner Source IPv6 address.
+          \item IPv6 address that is contained in the Routing-Header-Type-2 from the
+              associated inner extension header. If the inner extension header is not
+              present, use the inner Destination IPv6 address.
+          \item inner Source UDP port
+          \item inner Destination UDP port
+        \end{itemize}
+  \item Else if VIRTIO_NET_HASH_TYPE_IP_EX is set, the hash is calculated over the
+	following fields:
+        \begin{itemsize}
+          \item Home address from the home address option in the inner IPv6 destination
+              options header. If the inner extension header is not present, use the
+              inner Source IPv6 address.
+          \item IPv6 address that is contained in the Routing-Header-Type-2 from the
+              associated inner extension header. If the inner extension header is not
+              present, use the inner Destination IPv6 address.
+        \end{itemize}
+  \item Else skip the inner IPv6 extension header and calculate the inner header hash as
+	defined for an encapsulated packet whose inner payload is an IPv6 packet without
+	an extension header.
+\end{itemsize}
+
 \paragraph{Hash reporting for incoming packets}
 \label{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash reporting for incoming packets}
 
-If VIRTIO_NET_F_HASH_REPORT was negotiated and
- the device has calculated the hash for the packet, the device fills \field{hash_report} with the report type of calculated hash
-and \field{hash_value} with the value of calculated hash.
+If VIRTIO_NET_F_HASH_REPORT was negotiated and the device has calculated the
+hash for the packet, the device fills \field{hash_report} with the report type
+of calculated hash, and \field{hash_value} with the value of calculated hash.
+Also, if VIRTIO_NET_F_HASH_TUNNEL was negotiated, the device needs to fill
+\field{hash_report_tunnel} with the report type of the encapsulated packet, and
+it is set to VIRTIO_NET_HASH_REPORT_TUNNEL_NONE for the unencapsulated packet.
 
 If VIRTIO_NET_F_HASH_REPORT was negotiated but due to any reason the
-hash was not calculated, the device sets \field{hash_report} to VIRTIO_NET_HASH_REPORT_NONE.
+hash was not calculated, the device sets \field{hash_report} to VIRTIO_NET_HASH_REPORT_NONE,
+and sets \field{hash_report_tunnel} to VIRTIO_NET_HASH_REPORT_TUNNEL_NONE.
 
 Possible values that the device can report in \field{hash_report} are defined below.
 They correspond to supported hash types defined in
@@ -4005,6 +4166,22 @@ \subsubsection{Processing of Incoming Packets}\label{sec:Device Types / Network
 #define VIRTIO_NET_HASH_REPORT_UDPv6_EX        9
 \end{lstlisting}
 
+If \field{hash_report} differs from VIRTIO_NET_HASH_REPORT_NONE,
+\field{hash_report_tunnel} can report the type of the encapsulated
+packet to the driver over the inner header hash calculation.
+Possible values that the device can report in \field{hash_report_tunnel}
+are defined below:
+
+\begin{lstlisting}
+#define VIRTIO_NET_HASH_REPORT_TUNNEL_NONE     0
+#define VIRTIO_NET_HASH_REPORT_GRE             1
+#define VIRTIO_NET_HASH_REPORT_VXLAN           2
+#define VIRTIO_NET_HASH_REPORT_GENEVE          3
+\end{lstlisting}
+
+They correspond to supported hash types defined in
+\ref{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / Supported/enabled hash types}.
+
 \subsubsection{Control Virtqueue}\label{sec:Device Types / Network Device / Device Operation / Control Virtqueue}
 
 The driver uses the control virtqueue (if VIRTIO_NET_F_CTRL_VQ is
introduction.tex +19 -0
index 287c5fc..ff01a9b 100644
@@ -98,6 +98,25 @@ \section{Normative References}\label{sec:Normative References}
 	\phantomsection\label{intro:SEC1}\textbf{[SEC1]} &
     Standards for Efficient Cryptography Group(SECG), ``SEC1: Elliptic Cureve Cryptography'', Version 1.0, September 2000.
 	\newline\url{https://www.secg.org/sec1-v2.pdf}\\
+	\phantomsection\label{intro:GRE}\textbf{[GRE]} &
+	Generic Routing Encapsulation
+	\newline\url{https://datatracker.ietf.org/doc/rfc2784/}\\
+	\phantomsection\label{intro:VXLAN}\textbf{[VXLAN]} &
+	Virtual eXtensible Local Area Network
+	\newline\url{https://datatracker.ietf.org/doc/rfc7348/}\\
+	\phantomsection\label{intro:GENEVE}\textbf{[GENEVE]} &
+	Generic Network Virtualization Encapsulation
+	\newline\url{https://datatracker.ietf.org/doc/rfc8926/}\\
+	\phantomsection\label{intro:IP}\textbf{[IP]} &
+	INTERNET PROTOCOL
+	\newline\url{https://www.rfc-editor.org/rfc/rfc791}\\
+	\phantomsection\label{intro:UDP}\textbf{[UDP]} &
+	User Datagram Protocol
+	\newline\url{https://www.rfc-editor.org/rfc/rfc768}\\
+	\phantomsection\label{intro:TCP}\textbf{[TCP]} &
+	TRANSMISSION CONTROL PROTOCOL
+	\newline\url{https://www.rfc-editor.org/rfc/rfc793}\\
+
 
 \end{longtable}
 
Signature · 1 line
2.19.1.6.gb485710b
author
#1 lore raw

Hi Michael, do you have any other comments?

Thanks.

Quoted 343 lines · Heng Qi

If the tunnel is used to encapsulate the packets, the hash calculated using the outer header of the receive packets is always fixed for the same flow packets, i.e. they will be steered to the same receive queue.

We add a feature bit VIRTIO_NET_F_HASH_TUNNEL and related bitmasks in \field{hash_types}, which instructs the device to calculate the hash using the inner headers of tunnel-encapsulated packets. Besides, values in \field{hash_report_tunnel} are added to report tunnel types.

---
v6:
1. Modify the wording of some sentences for clarity. @Michael S. Tsirkin
2. Fix some syntax issues. @Michael S. Tsirkin

v5:
1. Fix some syntax and capitalization issues. @Michael S. Tsirkin
2. Use encapsulated/encaptulation uniformly. @Michael S. Tsirkin
3. Move the links to introduction section. @Michael S. Tsirkin
4. Clarify some sentences. @Michael S. Tsirkin

v4:
1. Clarify some paragraphs. @Cornelia Huck
2. Fix the u8 type. @Cornelia Huck

v3:
1. Rename VIRTIO_NET_F_HASH_GRE_VXLAN_GENEVE_INNER to VIRTIO_NET_F_HASH_TUNNEL. @Jason Wang
2. Make things clearer. @Jason Wang @Michael S. Tsirkin
3. Keep the possibility to use inner hash for automatic receive steering. @Jason Wang
4. Add the "Tunnel packet" paragraph to avoid repeating the GRE etc. many times. @Michael S. Tsirkin

v2:
1. Add a feature bit for GRE/VXLAN/GENEVE inner hash. @Jason Wang
2. Chang \field{hash_tunnel} to \field{hash_report_tunnel}. @Jason Wang, @Michael S. Tsirkin

v1:
1. Remove the patch for the bitmask fix. @Michael S. Tsirkin
2. Clarify some paragraphs. @Jason Wang
3. Add \field{hash_tunnel} and VIRTIO_NET_HASH_REPORT_GRE. @Yuri Benditovich

  content.tex      | 191 +++++++++++++++++++++++++++++++++++++++++++++--
  introduction.tex |  19 +++++
  2 files changed, 203 insertions(+), 7 deletions(-)
content.tex +184 -7
index e863709..7845f6c 100644
@@ -3084,6 +3084,9 @@ \subsection{Feature bits}\label{sec:Device Types / Network Device / Feature bits
  \item[VIRTIO_NET_F_CTRL_MAC_ADDR(23)] Set MAC address through control
      channel.
 
+\item[VIRTIO_NET_F_HASH_TUNNEL(52)] Device supports inner
+    header hash for GRE, VXLAN and GENEVE tunnel-encapsulated packets.
+
  \item[VIRTIO_NET_F_NOTF_COAL(53)] Device supports notifications coalescing.
 
  \item[VIRTIO_NET_F_GUEST_USO4 (54)] Driver can receive USOv4 packets.
@@ -3095,7 +3098,8 @@ \subsection{Feature bits}\label{sec:Device Types / Network Device / Feature bits
   to several segments when each of these smaller packets has UDP header.
 
  \item[VIRTIO_NET_F_HASH_REPORT(57)] Device can report per-packet hash
-    value and a type of calculated hash.
+    value, a type of calculated hash, and, if VIRTIO_NET_F_HASH_TUNNEL
+    is negotiated, an encapsulation packet type.
 
  \item[VIRTIO_NET_F_GUEST_HDRLEN(59)] Driver can provide the exact \field{hdr_len}
      value. Device benefits from knowing the exact header length.
@@ -3140,6 +3144,7 @@ \subsubsection{Feature bit requirements}\label{sec:Device Types / Network Device
  \item[VIRTIO_NET_F_NOTF_COAL] Requires VIRTIO_NET_F_CTRL_VQ.
  \item[VIRTIO_NET_F_RSC_EXT] Requires VIRTIO_NET_F_HOST_TSO4 or VIRTIO_NET_F_HOST_TSO6.
  \item[VIRTIO_NET_F_RSS] Requires VIRTIO_NET_F_CTRL_VQ.
+\item[VIRTIO_NET_F_HASH_TUNNEL] Requires VIRTIO_NET_F_CTRL_VQ.
  \end{description}
 
  \subsubsection{Legacy Interface: Feature bits}\label{sec:Device Types / Network Device / Feature bits / Legacy Interface: Feature bits}
@@ -3386,7 +3391,8 @@ \subsection{Device Operation}\label{sec:Device Types / Network Device / Device O
          le16 num_buffers;
          le32 hash_value;        (Only if VIRTIO_NET_F_HASH_REPORT negotiated)
          le16 hash_report;       (Only if VIRTIO_NET_F_HASH_REPORT negotiated)
-        le16 padding_reserved;  (Only if VIRTIO_NET_F_HASH_REPORT negotiated)
+        u8 hash_report_tunnel;  (Only if VIRTIO_NET_F_HASH_REPORT negotiated, only valid of VIRTIO_NET_F_HASH_TUNNEL negotiated)
+        u8 padding_reserved;    (Only if VIRTIO_NET_F_HASH_REPORT negotiated)
  };
  \end{lstlisting}
 
@@ -3837,7 +3843,8 @@ \subsubsection{Processing of Incoming Packets}\label{sec:Device Types / Network
  A device attempts to calculate a per-packet hash in the following cases:
  \begin{itemize}
  \item The feature VIRTIO_NET_F_RSS was negotiated. The device uses the hash to determine the receive virtqueue to place incoming packets.
-\item The feature VIRTIO_NET_F_HASH_REPORT was negotiated. The device reports the hash value and the hash type with the packet.
+\item The feature VIRTIO_NET_F_HASH_REPORT was negotiated. The device reports the hash value and the hash type. If additionally
+VIRTIO_NET_F_HASH_TUNNEL was negotiated, the device reports the encapsulation type as well.
  \end{itemize}
 
  If the feature VIRTIO_NET_F_RSS was negotiated:
@@ -3863,8 +3870,36 @@ \subsubsection{Processing of Incoming Packets}\label{sec:Device Types / Network
   \ref{sec:Device Types / Network Device / Device Operation / Control Virtqueue / Automatic receive steering in multiqueue mode / Hash calculation}.
  \end{itemize}
 
+\subparagraph{Tunnel/Encapsulated packet}
+\label{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / Tunnel/Encapsulated packet}
+A tunnel packet is encapsulated from the original packet based on the tunneling
+protocol (only a single level of encapsulation is currently supported). The
+encapsulated packet contains an outer header and an inner header, and the device
+calculates the hash over either the inner header or the outer header.
+
+When the feature VIRTIO_NET_F_HASH_TUNNEL is negotiated and the corresponding
+encapsulation type is set in \field{hash_types}, the hash for a specific type of
+encapsulated packet is calculated over the inner as opposed to outer header.
+Supported encapsulation types are listed in \ref{sec:Device Types / Network Device /
+Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets /
+Supported/enabled hash types}.
+
+If both VIRTIO_NET_F_HASH_REPORT and VIRTIO_NET_F_HASH_TUNNEL are negotiated,
+the device can support inner hash calculation for \hyperref[intro:GRE]{[GRE]},
+\hyperref[intro:VXLAN]{[VXLAN]} and \hyperref[intro:GENEVE]{[GENEVE]}
+encapsulated packets, and the corresponding encapsulation type in \field{hash_types}
+is VIRTIO_NET_HASH_TYPE_{GRE, VXLAN, GENEVE}_INNER respectively. The value in
+\field{hash_report_tunnel} is VIRTIO_NET_HASH_REPORT_{NONE, GRE, VXLAN, GENEVE} respectively.
+
+If VIRTIO_NET_F_HASH_REPORT is negotiated but VIRTIO_NET_F_HASH_TUNNEL is not
+negotiated, the device calculates the hash over the outer header, and
+\field{hash_report} reports the hash type. \field{hash_report_tunnel}
+is no longer valid.
+
  \subparagraph{Supported/enabled hash types}
  \label{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / Supported/enabled hash types}
+This paragraph relies on definitions from \hyperref[intro:IP]{[IP]},
+\hyperref[intro:UDP]{[UDP]} and \hyperref[intro:TCP]{[TCP]}.
  Hash types applicable for IPv4 packets:
  \begin{lstlisting}
  #define VIRTIO_NET_HASH_TYPE_IPv4              (1 << 0)
@@ -3884,6 +3919,22 @@ \subsubsection{Processing of Incoming Packets}\label{sec:Device Types / Network
  #define VIRTIO_NET_HASH_TYPE_UDP_EX            (1 << 8)
  \end{lstlisting}
 
+If the feature VIRTIO_NET_F_HASH_TUNNEL is negotiated, the encapsulation
+hash type below indicates that the hash is calculated over the inner
+header of the encapsulated packet:
+Hash type applicable for inner payload of the gre-encapsulated packet
+\begin{lstlisting}
+#define VIRTIO_NET_HASH_TYPE_GRE_INNER         (1 << 9)
+\end{lstlisting}
+Hash type applicable for inner payload of the vxlan-encapsulated packet
+\begin{lstlisting}
+#define VIRTIO_NET_HASH_TYPE_VXLAN_INNER       (1 << 10)
+\end{lstlisting}
+Hash type applicable for inner payload of the geneve-encapsulated packet
+\begin{lstlisting}
+#define VIRTIO_NET_HASH_TYPE_GENEVE_INNER      (1 << 11)
+\end{lstlisting}
+
  \subparagraph{IPv4 packets}
  \label{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / IPv4 packets}
  The device calculates the hash on IPv4 packets according to 'Enabled hash types' bitmask as follows:
@@ -3975,15 +4026,125 @@ \subsubsection{Processing of Incoming Packets}\label{sec:Device Types / Network
  (see \ref{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / IPv6 packets without extension header}).
  \end{itemize}
 
+\subparagraph{Inner payload of an encapsulated packet}
+\label{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / Inner payload of the encapsulated packet}
+If the feature VIRTIO_NET_F_HASH_TUNNEL is negotiated and the corresponding
+encapsulation hash type is set in \field{hash_types}, the device calculates the
+inner header hash on an encapsulated packet (See \ref{sec:Device Types
+/ Network Device / Device Operation / Processing of Incoming Packets /
+Hash calculation for incoming packets / Tunnel/Encapsulated packet}) as follows:
+
+The device calculates the hash on the inner IPv4 packet of an encapsulated packet
+according to 'Enabled hash types' bitmask as follows:
+\begin{itemize}
+  \item If VIRTIO_NET_HASH_TYPE_TCPv4 is set and the encapsulated packet has an inner
+	TCPv4 header, the hash is calculated over the following fields:
+        \begin{itemsize}
+          \item inner Source IP address
+          \item inner Destination IP address
+          \item inner Source TCP port
+          \item inner Destination TCP port
+        \end{itemsize}
+  \item Else if VIRTIO_NET_HASH_TYPE_UDPv4 is set and the encapsulated packet has an
+	inner UDPv4 header, the hash is calculated over the following fields:
+        \begin{itemsize}
+          \item inner Source IP address
+          \item inner Destination IP address
+          \item inner Source UDP port
+          \item inner Destination UDP port
+        \end{itemize}
+  \item Else if VIRTIO_NET_HASH_TYPE_IPv4 is set, the hash is calculated over the
+	following fields:
+        \begin{itemsize}
+          \item inner Source IP address
+          \item inner Destination IP address
+        \end{itemsize}
+  \item Else the device does not calculate the hash
+\end{itemize}
+
+The device calculates the hash on the inner IPv6 packet without an extension header
+of an encapsulated packet according to 'Enabled hash types' bitmask as follows:
+\begin{itemize}
+  \item If VIRTIO_NET_HASH_TYPE_TCPv6 is set and the encapsulated packet has an inner
+	TCPv6 header, the hash is calculated over the following fields:
+        \begin{itemsize}
+          \item inner Source IPv6 address
+          \item inner Destination IPv6 address
+          \item inner Source TCP port
+          \item inner Destination TCP port
+        \end{itemsize}
+  \item Else if VIRTIO_NET_HASH_TYPE_UDPv6 is set and the encapsulated packet has an
+	inner UDPv6 header, the hash is calculated over the following fields:
+        \begin{itemsize}
+          \item inner Source IPv6 address
+          \item inner Destination IPv6 address
+          \item inner Source UDP port
+          \item inner Destination UDP port
+        \end{itemize}
+  \item Else if VIRTIO_NET_HASH_TYPE_IPv6 is set, the hash is calculated over the
+	following fields:
+        \begin{itemsize}
+          \item inner Source IPv6 address
+          \item inner Destination IPv6 address
+        \end{itemsize}
+  \item Else the device does not calculate the hash
+\end{itemize}
+
+The device calculates the hash on the inner IPv6 packet with an extension header
+of an encapsulated packet according to 'Enabled hash types' bitmask as follows:
+\begin{itemsize}
+  \item If VIRTIO_NET_HASH_TYPE_TCP_EX is set and the encapsulated packet has an inner
+	TCPv6 header, the hash is calculated over the following fields:
+        \begin{itemize}
+          \item Home address from the home address option in the inner IPv6 destination
+              options header. If the inner extension header is not present, use the
+              inner Source IPv6 address.
+          \item IPv6 address that is contained in the Routing-Header-Type-2 from the
+              associated inner extension header. If the inner extension header is not
+              present, use the inner Destination IPv6 address.
+          \item inner Source TCP port
+          \item inner Destination TCP port
+        \end{itemize}
+  \item Else if VIRTIO_NET_HASH_TYPE_UDP_EX is set and the encapsulated packet has an inner
+	UDPv6 header, the hash is calculated over the following fields:
+        \begin{itemsize}
+          \item Home address from the home address option in the inner IPv6 destination
+              options header. If the inner extension header is not present, use the
+              inner Source IPv6 address.
+          \item IPv6 address that is contained in the Routing-Header-Type-2 from the
+              associated inner extension header. If the inner extension header is not
+              present, use the inner Destination IPv6 address.
+          \item inner Source UDP port
+          \item inner Destination UDP port
+        \end{itemize}
+  \item Else if VIRTIO_NET_HASH_TYPE_IP_EX is set, the hash is calculated over the
+	following fields:
+        \begin{itemsize}
+          \item Home address from the home address option in the inner IPv6 destination
+              options header. If the inner extension header is not present, use the
+              inner Source IPv6 address.
+          \item IPv6 address that is contained in the Routing-Header-Type-2 from the
+              associated inner extension header. If the inner extension header is not
+              present, use the inner Destination IPv6 address.
+        \end{itemize}
+  \item Else skip the inner IPv6 extension header and calculate the inner header hash as
+	defined for an encapsulated packet whose inner payload is an IPv6 packet without
+	an extension header.
+\end{itemsize}
+
  \paragraph{Hash reporting for incoming packets}
  \label{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash reporting for incoming packets}
 
-If VIRTIO_NET_F_HASH_REPORT was negotiated and
- the device has calculated the hash for the packet, the device fills \field{hash_report} with the report type of calculated hash
-and \field{hash_value} with the value of calculated hash.
+If VIRTIO_NET_F_HASH_REPORT was negotiated and the device has calculated the
+hash for the packet, the device fills \field{hash_report} with the report type
+of calculated hash, and \field{hash_value} with the value of calculated hash.
+Also, if VIRTIO_NET_F_HASH_TUNNEL was negotiated, the device needs to fill
+\field{hash_report_tunnel} with the report type of the encapsulated packet, and
+it is set to VIRTIO_NET_HASH_REPORT_TUNNEL_NONE for the unencapsulated packet.
 
  If VIRTIO_NET_F_HASH_REPORT was negotiated but due to any reason the
-hash was not calculated, the device sets \field{hash_report} to VIRTIO_NET_HASH_REPORT_NONE.
+hash was not calculated, the device sets \field{hash_report} to VIRTIO_NET_HASH_REPORT_NONE,
+and sets \field{hash_report_tunnel} to VIRTIO_NET_HASH_REPORT_TUNNEL_NONE.
 
  Possible values that the device can report in \field{hash_report} are defined below.
  They correspond to supported hash types defined in
@@ -4005,6 +4166,22 @@ \subsubsection{Processing of Incoming Packets}\label{sec:Device Types / Network
  #define VIRTIO_NET_HASH_REPORT_UDPv6_EX        9
  \end{lstlisting}
 
+If \field{hash_report} differs from VIRTIO_NET_HASH_REPORT_NONE,
+\field{hash_report_tunnel} can report the type of the encapsulated
+packet to the driver over the inner header hash calculation.
+Possible values that the device can report in \field{hash_report_tunnel}
+are defined below:
+
+\begin{lstlisting}
+#define VIRTIO_NET_HASH_REPORT_TUNNEL_NONE     0
+#define VIRTIO_NET_HASH_REPORT_GRE             1
+#define VIRTIO_NET_HASH_REPORT_VXLAN           2
+#define VIRTIO_NET_HASH_REPORT_GENEVE          3
+\end{lstlisting}
+
+They correspond to supported hash types defined in
+\ref{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / Supported/enabled hash types}.
+
  \subsubsection{Control Virtqueue}\label{sec:Device Types / Network Device / Device Operation / Control Virtqueue}
 
  The driver uses the control virtqueue (if VIRTIO_NET_F_CTRL_VQ is
introduction.tex +19 -0
index 287c5fc..ff01a9b 100644
@@ -98,6 +98,25 @@ \section{Normative References}\label{sec:Normative References}
  	\phantomsection\label{intro:SEC1}\textbf{[SEC1]} &
      Standards for Efficient Cryptography Group(SECG), ``SEC1: Elliptic Cureve Cryptography'', Version 1.0, September 2000.
  	\newline\url{https://www.secg.org/sec1-v2.pdf}\\
+	\phantomsection\label{intro:GRE}\textbf{[GRE]} &
+	Generic Routing Encapsulation
+	\newline\url{https://datatracker.ietf.org/doc/rfc2784/}\\
+	\phantomsection\label{intro:VXLAN}\textbf{[VXLAN]} &
+	Virtual eXtensible Local Area Network
+	\newline\url{https://datatracker.ietf.org/doc/rfc7348/}\\
+	\phantomsection\label{intro:GENEVE}\textbf{[GENEVE]} &
+	Generic Network Virtualization Encapsulation
+	\newline\url{https://datatracker.ietf.org/doc/rfc8926/}\\
+	\phantomsection\label{intro:IP}\textbf{[IP]} &
+	INTERNET PROTOCOL
+	\newline\url{https://www.rfc-editor.org/rfc/rfc791}\\
+	\phantomsection\label{intro:UDP}\textbf{[UDP]} &
+	User Datagram Protocol
+	\newline\url{https://www.rfc-editor.org/rfc/rfc768}\\
+	\phantomsection\label{intro:TCP}\textbf{[TCP]} &
+	TRANSMISSION CONTROL PROTOCOL
+	\newline\url{https://www.rfc-editor.org/rfc/rfc793}\\
+
 
  \end{longtable}
 
#2 lore raw
Quoted 4 lines · Heng Qi

Hi Michael, do you have any other comments?

Thanks.

Sit tight pls, you only posted this 5 hours ago.

Signature · 1 line
MST
#3 lore raw
In reply to Heng Qi
Quoted 9 lines · Heng Qi

If the tunnel is used to encapsulate the packets, the hash calculated using the outer header of the receive packets is always fixed for the same flow packets, i.e. they will be steered to the same receive queue.

We add a feature bit VIRTIO_NET_F_HASH_TUNNEL and related bitmasks in \field{hash_types}, which instructs the device to calculate the hash using the inner headers of tunnel-encapsulated packets. Besides, values in \field{hash_report_tunnel} are added to report tunnel types.

Quoted 5 lines

ok close to being ready. a couple of minor comments.

Quoted 39 lines

---
v6:
1. Modify the wording of some sentences for clarity. @Michael S. Tsirkin
2. Fix some syntax issues. @Michael S. Tsirkin

v5:
1. Fix some syntax and capitalization issues. @Michael S. Tsirkin
2. Use encapsulated/encaptulation uniformly. @Michael S. Tsirkin
3. Move the links to introduction section. @Michael S. Tsirkin
4. Clarify some sentences. @Michael S. Tsirkin

v4:
1. Clarify some paragraphs. @Cornelia Huck
2. Fix the u8 type. @Cornelia Huck

v3:
1. Rename VIRTIO_NET_F_HASH_GRE_VXLAN_GENEVE_INNER to VIRTIO_NET_F_HASH_TUNNEL. @Jason Wang
2. Make things clearer. @Jason Wang @Michael S. Tsirkin
3. Keep the possibility to use inner hash for automatic receive steering. @Jason Wang
4. Add the "Tunnel packet" paragraph to avoid repeating the GRE etc. many times. @Michael S. Tsirkin

v2:
1. Add a feature bit for GRE/VXLAN/GENEVE inner hash. @Jason Wang
2. Chang \field{hash_tunnel} to \field{hash_report_tunnel}. @Jason Wang, @Michael S. Tsirkin

v1:
1. Remove the patch for the bitmask fix. @Michael S. Tsirkin
2. Clarify some paragraphs. @Jason Wang
3. Add \field{hash_tunnel} and VIRTIO_NET_HASH_REPORT_GRE. @Yuri Benditovich

 content.tex      | 191 +++++++++++++++++++++++++++++++++++++++++++++--
 introduction.tex |  19 +++++
 2 files changed, 203 insertions(+), 7 deletions(-)
content.tex +0 -0
index e863709..7845f6c 100644

content.tex \subsection{Feature bits}\label{sec:Device Types / Network Device / Feature bits · line 3088

Quoted 5 lines
content.tex +2 -0
@@ -3084,6 +3084,9 @@ \subsection{Feature bits}\label{sec:Device Types / Network Device / Feature bits
 \item[VIRTIO_NET_F_CTRL_MAC_ADDR(23)] Set MAC address through control
     channel.
 
+\item[VIRTIO_NET_F_HASH_TUNNEL(52)] Device supports inner
+    header hash for GRE, VXLAN and GENEVE tunnel-encapsulated packets.

I would probably drop the list of tunnel types here.

Quoted 212 lines
content.tex +158 -3
+
 \item[VIRTIO_NET_F_NOTF_COAL(53)] Device supports notifications coalescing.
 
 \item[VIRTIO_NET_F_GUEST_USO4 (54)] Driver can receive USOv4 packets.
@@ -3095,7 +3098,8 @@ \subsection{Feature bits}\label{sec:Device Types / Network Device / Feature bits
  to several segments when each of these smaller packets has UDP header.
 
 \item[VIRTIO_NET_F_HASH_REPORT(57)] Device can report per-packet hash
-    value and a type of calculated hash.
+    value, a type of calculated hash, and, if VIRTIO_NET_F_HASH_TUNNEL
+    is negotiated, an encapsulation packet type.
 
 \item[VIRTIO_NET_F_GUEST_HDRLEN(59)] Driver can provide the exact \field{hdr_len}
     value. Device benefits from knowing the exact header length.
@@ -3140,6 +3144,7 @@ \subsubsection{Feature bit requirements}\label{sec:Device Types / Network Device
 \item[VIRTIO_NET_F_NOTF_COAL] Requires VIRTIO_NET_F_CTRL_VQ.
 \item[VIRTIO_NET_F_RSC_EXT] Requires VIRTIO_NET_F_HOST_TSO4 or VIRTIO_NET_F_HOST_TSO6.
 \item[VIRTIO_NET_F_RSS] Requires VIRTIO_NET_F_CTRL_VQ.
+\item[VIRTIO_NET_F_HASH_TUNNEL] Requires VIRTIO_NET_F_CTRL_VQ.
 \end{description}
 
 \subsubsection{Legacy Interface: Feature bits}\label{sec:Device Types / Network Device / Feature bits / Legacy Interface: Feature bits}
@@ -3386,7 +3391,8 @@ \subsection{Device Operation}\label{sec:Device Types / Network Device / Device O
         le16 num_buffers;
         le32 hash_value;        (Only if VIRTIO_NET_F_HASH_REPORT negotiated)
         le16 hash_report;       (Only if VIRTIO_NET_F_HASH_REPORT negotiated)
-        le16 padding_reserved;  (Only if VIRTIO_NET_F_HASH_REPORT negotiated)
+        u8 hash_report_tunnel;  (Only if VIRTIO_NET_F_HASH_REPORT negotiated, only valid of VIRTIO_NET_F_HASH_TUNNEL negotiated)
+        u8 padding_reserved;    (Only if VIRTIO_NET_F_HASH_REPORT negotiated)
 };
 \end{lstlisting}
 
@@ -3837,7 +3843,8 @@ \subsubsection{Processing of Incoming Packets}\label{sec:Device Types / Network
 A device attempts to calculate a per-packet hash in the following cases:
 \begin{itemize}
 \item The feature VIRTIO_NET_F_RSS was negotiated. The device uses the hash to determine the receive virtqueue to place incoming packets.
-\item The feature VIRTIO_NET_F_HASH_REPORT was negotiated. The device reports the hash value and the hash type with the packet.
+\item The feature VIRTIO_NET_F_HASH_REPORT was negotiated. The device reports the hash value and the hash type. If additionally
+VIRTIO_NET_F_HASH_TUNNEL was negotiated, the device reports the encapsulation type as well.
 \end{itemize}
 
 If the feature VIRTIO_NET_F_RSS was negotiated:
@@ -3863,8 +3870,36 @@ \subsubsection{Processing of Incoming Packets}\label{sec:Device Types / Network
  \ref{sec:Device Types / Network Device / Device Operation / Control Virtqueue / Automatic receive steering in multiqueue mode / Hash calculation}.
 \end{itemize}
 
+\subparagraph{Tunnel/Encapsulated packet}
+\label{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / Tunnel/Encapsulated packet}
+A tunnel packet is encapsulated from the original packet based on the tunneling
+protocol (only a single level of encapsulation is currently supported). The
+encapsulated packet contains an outer header and an inner header, and the device
+calculates the hash over either the inner header or the outer header.
+
+When the feature VIRTIO_NET_F_HASH_TUNNEL is negotiated and the corresponding
+encapsulation type is set in \field{hash_types}, the hash for a specific type of
+encapsulated packet is calculated over the inner as opposed to outer header.
+Supported encapsulation types are listed in \ref{sec:Device Types / Network Device /
+Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets /
+Supported/enabled hash types}.
+
+If both VIRTIO_NET_F_HASH_REPORT and VIRTIO_NET_F_HASH_TUNNEL are negotiated,
+the device can support inner hash calculation for \hyperref[intro:GRE]{[GRE]},
+\hyperref[intro:VXLAN]{[VXLAN]} and \hyperref[intro:GENEVE]{[GENEVE]}
+encapsulated packets, and the corresponding encapsulation type in \field{hash_types}
+is VIRTIO_NET_HASH_TYPE_{GRE, VXLAN, GENEVE}_INNER respectively. The value in
+\field{hash_report_tunnel} is VIRTIO_NET_HASH_REPORT_{NONE, GRE, VXLAN, GENEVE} respectively.
+
+If VIRTIO_NET_F_HASH_REPORT is negotiated but VIRTIO_NET_F_HASH_TUNNEL is not
+negotiated, the device calculates the hash over the outer header, and
+\field{hash_report} reports the hash type. \field{hash_report_tunnel}
+is no longer valid.
+
 \subparagraph{Supported/enabled hash types}
 \label{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / Supported/enabled hash types}
+This paragraph relies on definitions from \hyperref[intro:IP]{[IP]},
+\hyperref[intro:UDP]{[UDP]} and \hyperref[intro:TCP]{[TCP]}.
 Hash types applicable for IPv4 packets:
 \begin{lstlisting}
 #define VIRTIO_NET_HASH_TYPE_IPv4              (1 << 0)
@@ -3884,6 +3919,22 @@ \subsubsection{Processing of Incoming Packets}\label{sec:Device Types / Network
 #define VIRTIO_NET_HASH_TYPE_UDP_EX            (1 << 8)
 \end{lstlisting}
 
+If the feature VIRTIO_NET_F_HASH_TUNNEL is negotiated, the encapsulation
+hash type below indicates that the hash is calculated over the inner
+header of the encapsulated packet:
+Hash type applicable for inner payload of the gre-encapsulated packet
+\begin{lstlisting}
+#define VIRTIO_NET_HASH_TYPE_GRE_INNER         (1 << 9)
+\end{lstlisting}
+Hash type applicable for inner payload of the vxlan-encapsulated packet
+\begin{lstlisting}
+#define VIRTIO_NET_HASH_TYPE_VXLAN_INNER       (1 << 10)
+\end{lstlisting}
+Hash type applicable for inner payload of the geneve-encapsulated packet
+\begin{lstlisting}
+#define VIRTIO_NET_HASH_TYPE_GENEVE_INNER      (1 << 11)
+\end{lstlisting}
+
 \subparagraph{IPv4 packets}
 \label{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / IPv4 packets}
 The device calculates the hash on IPv4 packets according to 'Enabled hash types' bitmask as follows:
@@ -3975,15 +4026,125 @@ \subsubsection{Processing of Incoming Packets}\label{sec:Device Types / Network
 (see \ref{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / IPv6 packets without extension header}).
 \end{itemize}
 
+\subparagraph{Inner payload of an encapsulated packet}
+\label{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / Inner payload of the encapsulated packet}
+If the feature VIRTIO_NET_F_HASH_TUNNEL is negotiated and the corresponding
+encapsulation hash type is set in \field{hash_types}, the device calculates the
+inner header hash on an encapsulated packet (See \ref{sec:Device Types
+/ Network Device / Device Operation / Processing of Incoming Packets /
+Hash calculation for incoming packets / Tunnel/Encapsulated packet}) as follows:
+
+The device calculates the hash on the inner IPv4 packet of an encapsulated packet
+according to 'Enabled hash types' bitmask as follows:
+\begin{itemize}
+  \item If VIRTIO_NET_HASH_TYPE_TCPv4 is set and the encapsulated packet has an inner
+	TCPv4 header, the hash is calculated over the following fields:
+        \begin{itemsize}
+          \item inner Source IP address
+          \item inner Destination IP address
+          \item inner Source TCP port
+          \item inner Destination TCP port
+        \end{itemsize}
+  \item Else if VIRTIO_NET_HASH_TYPE_UDPv4 is set and the encapsulated packet has an
+	inner UDPv4 header, the hash is calculated over the following fields:
+        \begin{itemsize}
+          \item inner Source IP address
+          \item inner Destination IP address
+          \item inner Source UDP port
+          \item inner Destination UDP port
+        \end{itemize}
+  \item Else if VIRTIO_NET_HASH_TYPE_IPv4 is set, the hash is calculated over the
+	following fields:
+        \begin{itemsize}
+          \item inner Source IP address
+          \item inner Destination IP address
+        \end{itemsize}
+  \item Else the device does not calculate the hash
+\end{itemize}
+
+The device calculates the hash on the inner IPv6 packet without an extension header
+of an encapsulated packet according to 'Enabled hash types' bitmask as follows:
+\begin{itemize}
+  \item If VIRTIO_NET_HASH_TYPE_TCPv6 is set and the encapsulated packet has an inner
+	TCPv6 header, the hash is calculated over the following fields:
+        \begin{itemsize}
+          \item inner Source IPv6 address
+          \item inner Destination IPv6 address
+          \item inner Source TCP port
+          \item inner Destination TCP port
+        \end{itemsize}
+  \item Else if VIRTIO_NET_HASH_TYPE_UDPv6 is set and the encapsulated packet has an
+	inner UDPv6 header, the hash is calculated over the following fields:
+        \begin{itemsize}
+          \item inner Source IPv6 address
+          \item inner Destination IPv6 address
+          \item inner Source UDP port
+          \item inner Destination UDP port
+        \end{itemize}
+  \item Else if VIRTIO_NET_HASH_TYPE_IPv6 is set, the hash is calculated over the
+	following fields:
+        \begin{itemsize}
+          \item inner Source IPv6 address
+          \item inner Destination IPv6 address
+        \end{itemsize}
+  \item Else the device does not calculate the hash
+\end{itemize}
+
+The device calculates the hash on the inner IPv6 packet with an extension header
+of an encapsulated packet according to 'Enabled hash types' bitmask as follows:
+\begin{itemsize}
+  \item If VIRTIO_NET_HASH_TYPE_TCP_EX is set and the encapsulated packet has an inner
+	TCPv6 header, the hash is calculated over the following fields:
+        \begin{itemize}
+          \item Home address from the home address option in the inner IPv6 destination
+              options header. If the inner extension header is not present, use the
+              inner Source IPv6 address.
+          \item IPv6 address that is contained in the Routing-Header-Type-2 from the
+              associated inner extension header. If the inner extension header is not
+              present, use the inner Destination IPv6 address.
+          \item inner Source TCP port
+          \item inner Destination TCP port
+        \end{itemize}
+  \item Else if VIRTIO_NET_HASH_TYPE_UDP_EX is set and the encapsulated packet has an inner
+	UDPv6 header, the hash is calculated over the following fields:
+        \begin{itemsize}
+          \item Home address from the home address option in the inner IPv6 destination
+              options header. If the inner extension header is not present, use the
+              inner Source IPv6 address.
+          \item IPv6 address that is contained in the Routing-Header-Type-2 from the
+              associated inner extension header. If the inner extension header is not
+              present, use the inner Destination IPv6 address.
+          \item inner Source UDP port
+          \item inner Destination UDP port
+        \end{itemize}
+  \item Else if VIRTIO_NET_HASH_TYPE_IP_EX is set, the hash is calculated over the
+	following fields:
+        \begin{itemsize}
+          \item Home address from the home address option in the inner IPv6 destination
+              options header. If the inner extension header is not present, use the
+              inner Source IPv6 address.
+          \item IPv6 address that is contained in the Routing-Header-Type-2 from the
+              associated inner extension header. If the inner extension header is not
+              present, use the inner Destination IPv6 address.
+        \end{itemize}
+  \item Else skip the inner IPv6 extension header and calculate the inner header hash as
+	defined for an encapsulated packet whose inner payload is an IPv6 packet without
+	an extension header.
+\end{itemsize}
+

content.tex \subsubsection{Processing of Incoming Packets}\label{sec:Device Types / Network · line 4141

Quoted 10 lines
content.tex +4 -3
 \paragraph{Hash reporting for incoming packets}
 \label{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash reporting for incoming packets}
 
-If VIRTIO_NET_F_HASH_REPORT was negotiated and
- the device has calculated the hash for the packet, the device fills \field{hash_report} with the report type of calculated hash
-and \field{hash_value} with the value of calculated hash.
+If VIRTIO_NET_F_HASH_REPORT was negotiated and the device has calculated the
+hash for the packet, the device fills \field{hash_report} with the report type
+of calculated hash, and \field{hash_value} with the value of calculated hash.
+Also, if VIRTIO_NET_F_HASH_TUNNEL was negotiated, the device needs to fill

needs to fill -> fills

for consistency

Quoted 65 lines
content.tex +20 -1
+\field{hash_report_tunnel} with the report type of the encapsulated packet, and
+it is set to VIRTIO_NET_HASH_REPORT_TUNNEL_NONE for the unencapsulated packet.
 
 If VIRTIO_NET_F_HASH_REPORT was negotiated but due to any reason the
-hash was not calculated, the device sets \field{hash_report} to VIRTIO_NET_HASH_REPORT_NONE.
+hash was not calculated, the device sets \field{hash_report} to VIRTIO_NET_HASH_REPORT_NONE,
+and sets \field{hash_report_tunnel} to VIRTIO_NET_HASH_REPORT_TUNNEL_NONE.
 
 Possible values that the device can report in \field{hash_report} are defined below.
 They correspond to supported hash types defined in
@@ -4005,6 +4166,22 @@ \subsubsection{Processing of Incoming Packets}\label{sec:Device Types / Network
 #define VIRTIO_NET_HASH_REPORT_UDPv6_EX        9
 \end{lstlisting}
 
+If \field{hash_report} differs from VIRTIO_NET_HASH_REPORT_NONE,
+\field{hash_report_tunnel} can report the type of the encapsulated
+packet to the driver over the inner header hash calculation.
+Possible values that the device can report in \field{hash_report_tunnel}
+are defined below:
+
+\begin{lstlisting}
+#define VIRTIO_NET_HASH_REPORT_TUNNEL_NONE     0
+#define VIRTIO_NET_HASH_REPORT_GRE             1
+#define VIRTIO_NET_HASH_REPORT_VXLAN           2
+#define VIRTIO_NET_HASH_REPORT_GENEVE          3
+\end{lstlisting}
+
+They correspond to supported hash types defined in
+\ref{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / Supported/enabled hash types}.
+
 \subsubsection{Control Virtqueue}\label{sec:Device Types / Network Device / Device Operation / Control Virtqueue}
 
 The driver uses the control virtqueue (if VIRTIO_NET_F_CTRL_VQ is
introduction.tex +19 -0
index 287c5fc..ff01a9b 100644
@@ -98,6 +98,25 @@ \section{Normative References}\label{sec:Normative References}
 	\phantomsection\label{intro:SEC1}\textbf{[SEC1]} &
     Standards for Efficient Cryptography Group(SECG), ``SEC1: Elliptic Cureve Cryptography'', Version 1.0, September 2000.
 	\newline\url{https://www.secg.org/sec1-v2.pdf}\\
+	\phantomsection\label{intro:GRE}\textbf{[GRE]} &
+	Generic Routing Encapsulation
+	\newline\url{https://datatracker.ietf.org/doc/rfc2784/}\\
+	\phantomsection\label{intro:VXLAN}\textbf{[VXLAN]} &
+	Virtual eXtensible Local Area Network
+	\newline\url{https://datatracker.ietf.org/doc/rfc7348/}\\
+	\phantomsection\label{intro:GENEVE}\textbf{[GENEVE]} &
+	Generic Network Virtualization Encapsulation
+	\newline\url{https://datatracker.ietf.org/doc/rfc8926/}\\
+	\phantomsection\label{intro:IP}\textbf{[IP]} &
+	INTERNET PROTOCOL
+	\newline\url{https://www.rfc-editor.org/rfc/rfc791}\\
+	\phantomsection\label{intro:UDP}\textbf{[UDP]} &
+	User Datagram Protocol
+	\newline\url{https://www.rfc-editor.org/rfc/rfc768}\\
+	\phantomsection\label{intro:TCP}\textbf{[TCP]} &
+	TRANSMISSION CONTROL PROTOCOL
+	\newline\url{https://www.rfc-editor.org/rfc/rfc793}\\
+
 
 \end{longtable}
 
Signature · 1 line
2.19.1.6.gb485710b
author
#4 lore raw
Quoted 61 lines · Michael S. Tsirkin
Quoted 9 lines · Heng Qi

If the tunnel is used to encapsulate the packets, the hash calculated using the outer header of the receive packets is always fixed for the same flow packets, i.e. they will be steered to the same receive queue.

We add a feature bit VIRTIO_NET_F_HASH_TUNNEL and related bitmasks in \field{hash_types}, which instructs the device to calculate the hash using the inner headers of tunnel-encapsulated packets. Besides, values in \field{hash_report_tunnel} are added to report tunnel types.

Quoted 5 lines

ok close to being ready. a couple of minor comments.

Quoted 42 lines

---
v6:
1. Modify the wording of some sentences for clarity. @Michael S. Tsirkin
2. Fix some syntax issues. @Michael S. Tsirkin

v5:
1. Fix some syntax and capitalization issues. @Michael S. Tsirkin
2. Use encapsulated/encaptulation uniformly. @Michael S. Tsirkin
3. Move the links to introduction section. @Michael S. Tsirkin
4. Clarify some sentences. @Michael S. Tsirkin

v4:
1. Clarify some paragraphs. @Cornelia Huck
2. Fix the u8 type. @Cornelia Huck

v3:
1. Rename VIRTIO_NET_F_HASH_GRE_VXLAN_GENEVE_INNER to VIRTIO_NET_F_HASH_TUNNEL. @Jason Wang
2. Make things clearer. @Jason Wang @Michael S. Tsirkin
3. Keep the possibility to use inner hash for automatic receive steering. @Jason Wang
4. Add the "Tunnel packet" paragraph to avoid repeating the GRE etc. many times. @Michael S. Tsirkin

v2:
1. Add a feature bit for GRE/VXLAN/GENEVE inner hash. @Jason Wang
2. Chang \field{hash_tunnel} to \field{hash_report_tunnel}. @Jason Wang, @Michael S. Tsirkin

v1:
1. Remove the patch for the bitmask fix. @Michael S. Tsirkin
2. Clarify some paragraphs. @Jason Wang
3. Add \field{hash_tunnel} and VIRTIO_NET_HASH_REPORT_GRE. @Yuri Benditovich

 content.tex      | 191 +++++++++++++++++++++++++++++++++++++++++++++--
 introduction.tex |  19 +++++
 2 files changed, 203 insertions(+), 7 deletions(-)
content.tex +0 -0
index e863709..7845f6c 100644
@@ -3084,6 +3084,9 @@ \subsection{Feature bits}\label{sec:Device Types / Network Device / Feature bits
 \item[VIRTIO_NET_F_CTRL_MAC_ADDR(23)] Set MAC address through control
     channel.
 
Quoted 4 lines

content.tex \subsection{Feature bits}\label{sec:Device Types / Network Device / Feature bits · line 3088

Quoted 2 lines
content.tex +2 -0
+\item[VIRTIO_NET_F_HASH_TUNNEL(52)] Device supports inner
+    header hash for GRE, VXLAN and GENEVE tunnel-encapsulated packets.

I would probably drop the list of tunnel types here.

Do you mean to use "Device supports inner header hash for tunnel-encapsulated packets." instead? Why? We do want to use this feature bit to indicate that the device supports inner hashing of GRE, VXLAN and GENEVE encapsulated packets. As in the v3 discussion
https://lists.oasis-open.org/archives/virtio-dev/202212/msg00024.html , we discussed using VIRTIO_NET_F_HASH_TUNNEL to replace VIRTIO_NET_F_HASH_GRE_VXLAN_GENEVE_INNER and plan to use VIRTIO_NET_F_HASH_TUNNEL_XYZ for future extensions.

Quoted 226 lines
Quoted 213 lines
content.tex +158 -3
+
 \item[VIRTIO_NET_F_NOTF_COAL(53)] Device supports notifications coalescing.
 
 \item[VIRTIO_NET_F_GUEST_USO4 (54)] Driver can receive USOv4 packets.
@@ -3095,7 +3098,8 @@ \subsection{Feature bits}\label{sec:Device Types / Network Device / Feature bits
  to several segments when each of these smaller packets has UDP header.
 
 \item[VIRTIO_NET_F_HASH_REPORT(57)] Device can report per-packet hash
-    value and a type of calculated hash.
+    value, a type of calculated hash, and, if VIRTIO_NET_F_HASH_TUNNEL
+    is negotiated, an encapsulation packet type.
 
 \item[VIRTIO_NET_F_GUEST_HDRLEN(59)] Driver can provide the exact \field{hdr_len}
     value. Device benefits from knowing the exact header length.
@@ -3140,6 +3144,7 @@ \subsubsection{Feature bit requirements}\label{sec:Device Types / Network Device
 \item[VIRTIO_NET_F_NOTF_COAL] Requires VIRTIO_NET_F_CTRL_VQ.
 \item[VIRTIO_NET_F_RSC_EXT] Requires VIRTIO_NET_F_HOST_TSO4 or VIRTIO_NET_F_HOST_TSO6.
 \item[VIRTIO_NET_F_RSS] Requires VIRTIO_NET_F_CTRL_VQ.
+\item[VIRTIO_NET_F_HASH_TUNNEL] Requires VIRTIO_NET_F_CTRL_VQ.
 \end{description}
 
 \subsubsection{Legacy Interface: Feature bits}\label{sec:Device Types / Network Device / Feature bits / Legacy Interface: Feature bits}
@@ -3386,7 +3391,8 @@ \subsection{Device Operation}\label{sec:Device Types / Network Device / Device O
         le16 num_buffers;
         le32 hash_value;        (Only if VIRTIO_NET_F_HASH_REPORT negotiated)
         le16 hash_report;       (Only if VIRTIO_NET_F_HASH_REPORT negotiated)
-        le16 padding_reserved;  (Only if VIRTIO_NET_F_HASH_REPORT negotiated)
+        u8 hash_report_tunnel;  (Only if VIRTIO_NET_F_HASH_REPORT negotiated, only valid of VIRTIO_NET_F_HASH_TUNNEL negotiated)
+        u8 padding_reserved;    (Only if VIRTIO_NET_F_HASH_REPORT negotiated)
 };
 \end{lstlisting}
 
@@ -3837,7 +3843,8 @@ \subsubsection{Processing of Incoming Packets}\label{sec:Device Types / Network
 A device attempts to calculate a per-packet hash in the following cases:
 \begin{itemize}
 \item The feature VIRTIO_NET_F_RSS was negotiated. The device uses the hash to determine the receive virtqueue to place incoming packets.
-\item The feature VIRTIO_NET_F_HASH_REPORT was negotiated. The device reports the hash value and the hash type with the packet.
+\item The feature VIRTIO_NET_F_HASH_REPORT was negotiated. The device reports the hash value and the hash type. If additionally
+VIRTIO_NET_F_HASH_TUNNEL was negotiated, the device reports the encapsulation type as well.
 \end{itemize}
 
 If the feature VIRTIO_NET_F_RSS was negotiated:
@@ -3863,8 +3870,36 @@ \subsubsection{Processing of Incoming Packets}\label{sec:Device Types / Network
  \ref{sec:Device Types / Network Device / Device Operation / Control Virtqueue / Automatic receive steering in multiqueue mode / Hash calculation}.
 \end{itemize}
 
+\subparagraph{Tunnel/Encapsulated packet}
+\label{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / Tunnel/Encapsulated packet}
+A tunnel packet is encapsulated from the original packet based on the tunneling
+protocol (only a single level of encapsulation is currently supported). The
+encapsulated packet contains an outer header and an inner header, and the device
+calculates the hash over either the inner header or the outer header.
+
+When the feature VIRTIO_NET_F_HASH_TUNNEL is negotiated and the corresponding
+encapsulation type is set in \field{hash_types}, the hash for a specific type of
+encapsulated packet is calculated over the inner as opposed to outer header.
+Supported encapsulation types are listed in \ref{sec:Device Types / Network Device /
+Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets /
+Supported/enabled hash types}.
+
+If both VIRTIO_NET_F_HASH_REPORT and VIRTIO_NET_F_HASH_TUNNEL are negotiated,
+the device can support inner hash calculation for \hyperref[intro:GRE]{[GRE]},
+\hyperref[intro:VXLAN]{[VXLAN]} and \hyperref[intro:GENEVE]{[GENEVE]}
+encapsulated packets, and the corresponding encapsulation type in \field{hash_types}
+is VIRTIO_NET_HASH_TYPE_{GRE, VXLAN, GENEVE}_INNER respectively. The value in
+\field{hash_report_tunnel} is VIRTIO_NET_HASH_REPORT_{NONE, GRE, VXLAN, GENEVE} respectively.
+
+If VIRTIO_NET_F_HASH_REPORT is negotiated but VIRTIO_NET_F_HASH_TUNNEL is not
+negotiated, the device calculates the hash over the outer header, and
+\field{hash_report} reports the hash type. \field{hash_report_tunnel}
+is no longer valid.
+
 \subparagraph{Supported/enabled hash types}
 \label{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / Supported/enabled hash types}
+This paragraph relies on definitions from \hyperref[intro:IP]{[IP]},
+\hyperref[intro:UDP]{[UDP]} and \hyperref[intro:TCP]{[TCP]}.
 Hash types applicable for IPv4 packets:
 \begin{lstlisting}
 #define VIRTIO_NET_HASH_TYPE_IPv4              (1 << 0)
@@ -3884,6 +3919,22 @@ \subsubsection{Processing of Incoming Packets}\label{sec:Device Types / Network
 #define VIRTIO_NET_HASH_TYPE_UDP_EX            (1 << 8)
 \end{lstlisting}
 
+If the feature VIRTIO_NET_F_HASH_TUNNEL is negotiated, the encapsulation
+hash type below indicates that the hash is calculated over the inner
+header of the encapsulated packet:
+Hash type applicable for inner payload of the gre-encapsulated packet
+\begin{lstlisting}
+#define VIRTIO_NET_HASH_TYPE_GRE_INNER         (1 << 9)
+\end{lstlisting}
+Hash type applicable for inner payload of the vxlan-encapsulated packet
+\begin{lstlisting}
+#define VIRTIO_NET_HASH_TYPE_VXLAN_INNER       (1 << 10)
+\end{lstlisting}
+Hash type applicable for inner payload of the geneve-encapsulated packet
+\begin{lstlisting}
+#define VIRTIO_NET_HASH_TYPE_GENEVE_INNER      (1 << 11)
+\end{lstlisting}
+
 \subparagraph{IPv4 packets}
 \label{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / IPv4 packets}
 The device calculates the hash on IPv4 packets according to 'Enabled hash types' bitmask as follows:
@@ -3975,15 +4026,125 @@ \subsubsection{Processing of Incoming Packets}\label{sec:Device Types / Network
 (see \ref{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / IPv6 packets without extension header}).
 \end{itemize}
 
+\subparagraph{Inner payload of an encapsulated packet}
+\label{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / Inner payload of the encapsulated packet}
+If the feature VIRTIO_NET_F_HASH_TUNNEL is negotiated and the corresponding
+encapsulation hash type is set in \field{hash_types}, the device calculates the
+inner header hash on an encapsulated packet (See \ref{sec:Device Types
+/ Network Device / Device Operation / Processing of Incoming Packets /
+Hash calculation for incoming packets / Tunnel/Encapsulated packet}) as follows:
+
+The device calculates the hash on the inner IPv4 packet of an encapsulated packet
+according to 'Enabled hash types' bitmask as follows:
+\begin{itemize}
+  \item If VIRTIO_NET_HASH_TYPE_TCPv4 is set and the encapsulated packet has an inner
+	TCPv4 header, the hash is calculated over the following fields:
+        \begin{itemsize}
+          \item inner Source IP address
+          \item inner Destination IP address
+          \item inner Source TCP port
+          \item inner Destination TCP port
+        \end{itemsize}
+  \item Else if VIRTIO_NET_HASH_TYPE_UDPv4 is set and the encapsulated packet has an
+	inner UDPv4 header, the hash is calculated over the following fields:
+        \begin{itemsize}
+          \item inner Source IP address
+          \item inner Destination IP address
+          \item inner Source UDP port
+          \item inner Destination UDP port
+        \end{itemize}
+  \item Else if VIRTIO_NET_HASH_TYPE_IPv4 is set, the hash is calculated over the
+	following fields:
+        \begin{itemsize}
+          \item inner Source IP address
+          \item inner Destination IP address
+        \end{itemsize}
+  \item Else the device does not calculate the hash
+\end{itemize}
+
+The device calculates the hash on the inner IPv6 packet without an extension header
+of an encapsulated packet according to 'Enabled hash types' bitmask as follows:
+\begin{itemize}
+  \item If VIRTIO_NET_HASH_TYPE_TCPv6 is set and the encapsulated packet has an inner
+	TCPv6 header, the hash is calculated over the following fields:
+        \begin{itemsize}
+          \item inner Source IPv6 address
+          \item inner Destination IPv6 address
+          \item inner Source TCP port
+          \item inner Destination TCP port
+        \end{itemsize}
+  \item Else if VIRTIO_NET_HASH_TYPE_UDPv6 is set and the encapsulated packet has an
+	inner UDPv6 header, the hash is calculated over the following fields:
+        \begin{itemsize}
+          \item inner Source IPv6 address
+          \item inner Destination IPv6 address
+          \item inner Source UDP port
+          \item inner Destination UDP port
+        \end{itemize}
+  \item Else if VIRTIO_NET_HASH_TYPE_IPv6 is set, the hash is calculated over the
+	following fields:
+        \begin{itemsize}
+          \item inner Source IPv6 address
+          \item inner Destination IPv6 address
+        \end{itemsize}
+  \item Else the device does not calculate the hash
+\end{itemize}
+
+The device calculates the hash on the inner IPv6 packet with an extension header
+of an encapsulated packet according to 'Enabled hash types' bitmask as follows:
+\begin{itemsize}
+  \item If VIRTIO_NET_HASH_TYPE_TCP_EX is set and the encapsulated packet has an inner
+	TCPv6 header, the hash is calculated over the following fields:
+        \begin{itemize}
+          \item Home address from the home address option in the inner IPv6 destination
+              options header. If the inner extension header is not present, use the
+              inner Source IPv6 address.
+          \item IPv6 address that is contained in the Routing-Header-Type-2 from the
+              associated inner extension header. If the inner extension header is not
+              present, use the inner Destination IPv6 address.
+          \item inner Source TCP port
+          \item inner Destination TCP port
+        \end{itemize}
+  \item Else if VIRTIO_NET_HASH_TYPE_UDP_EX is set and the encapsulated packet has an inner
+	UDPv6 header, the hash is calculated over the following fields:
+        \begin{itemsize}
+          \item Home address from the home address option in the inner IPv6 destination
+              options header. If the inner extension header is not present, use the
+              inner Source IPv6 address.
+          \item IPv6 address that is contained in the Routing-Header-Type-2 from the
+              associated inner extension header. If the inner extension header is not
+              present, use the inner Destination IPv6 address.
+          \item inner Source UDP port
+          \item inner Destination UDP port
+        \end{itemize}
+  \item Else if VIRTIO_NET_HASH_TYPE_IP_EX is set, the hash is calculated over the
+	following fields:
+        \begin{itemsize}
+          \item Home address from the home address option in the inner IPv6 destination
+              options header. If the inner extension header is not present, use the
+              inner Source IPv6 address.
+          \item IPv6 address that is contained in the Routing-Header-Type-2 from the
+              associated inner extension header. If the inner extension header is not
+              present, use the inner Destination IPv6 address.
+        \end{itemize}
+  \item Else skip the inner IPv6 extension header and calculate the inner header hash as
+	defined for an encapsulated packet whose inner payload is an IPv6 packet without
+	an extension header.
+\end{itemsize}
+

content.tex \subsubsection{Processing of Incoming Packets}\label{sec:Device Types / Network · line 4141

Quoted 10 lines
content.tex +4 -3
 \paragraph{Hash reporting for incoming packets}
 \label{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash reporting for incoming packets}
 
-If VIRTIO_NET_F_HASH_REPORT was negotiated and
- the device has calculated the hash for the packet, the device fills \field{hash_report} with the report type of calculated hash
-and \field{hash_value} with the value of calculated hash.
+If VIRTIO_NET_F_HASH_REPORT was negotiated and the device has calculated the
+hash for the packet, the device fills \field{hash_report} with the report type
+of calculated hash, and \field{hash_value} with the value of calculated hash.
+Also, if VIRTIO_NET_F_HASH_TUNNEL was negotiated, the device needs to fill

needs to fill -> fills

Quoted 1 line

for consistency

I agree.

Thanks.

Quoted 83 lines
Quoted 66 lines
content.tex +20 -1
+\field{hash_report_tunnel} with the report type of the encapsulated packet, and
+it is set to VIRTIO_NET_HASH_REPORT_TUNNEL_NONE for the unencapsulated packet.
 
 If VIRTIO_NET_F_HASH_REPORT was negotiated but due to any reason the
-hash was not calculated, the device sets \field{hash_report} to VIRTIO_NET_HASH_REPORT_NONE.
+hash was not calculated, the device sets \field{hash_report} to VIRTIO_NET_HASH_REPORT_NONE,
+and sets \field{hash_report_tunnel} to VIRTIO_NET_HASH_REPORT_TUNNEL_NONE.
 
 Possible values that the device can report in \field{hash_report} are defined below.
 They correspond to supported hash types defined in
@@ -4005,6 +4166,22 @@ \subsubsection{Processing of Incoming Packets}\label{sec:Device Types / Network
 #define VIRTIO_NET_HASH_REPORT_UDPv6_EX        9
 \end{lstlisting}
 
+If \field{hash_report} differs from VIRTIO_NET_HASH_REPORT_NONE,
+\field{hash_report_tunnel} can report the type of the encapsulated
+packet to the driver over the inner header hash calculation.
+Possible values that the device can report in \field{hash_report_tunnel}
+are defined below:
+
+\begin{lstlisting}
+#define VIRTIO_NET_HASH_REPORT_TUNNEL_NONE     0
+#define VIRTIO_NET_HASH_REPORT_GRE             1
+#define VIRTIO_NET_HASH_REPORT_VXLAN           2
+#define VIRTIO_NET_HASH_REPORT_GENEVE          3
+\end{lstlisting}
+
+They correspond to supported hash types defined in
+\ref{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / Supported/enabled hash types}.
+
 \subsubsection{Control Virtqueue}\label{sec:Device Types / Network Device / Device Operation / Control Virtqueue}
 
 The driver uses the control virtqueue (if VIRTIO_NET_F_CTRL_VQ is
introduction.tex +19 -0
index 287c5fc..ff01a9b 100644
@@ -98,6 +98,25 @@ \section{Normative References}\label{sec:Normative References}
 	\phantomsection\label{intro:SEC1}\textbf{[SEC1]} &
     Standards for Efficient Cryptography Group(SECG), ``SEC1: Elliptic Cureve Cryptography'', Version 1.0, September 2000.
 	\newline\url{https://www.secg.org/sec1-v2.pdf}\\
+	\phantomsection\label{intro:GRE}\textbf{[GRE]} &
+	Generic Routing Encapsulation
+	\newline\url{https://datatracker.ietf.org/doc/rfc2784/}\\
+	\phantomsection\label{intro:VXLAN}\textbf{[VXLAN]} &
+	Virtual eXtensible Local Area Network
+	\newline\url{https://datatracker.ietf.org/doc/rfc7348/}\\
+	\phantomsection\label{intro:GENEVE}\textbf{[GENEVE]} &
+	Generic Network Virtualization Encapsulation
+	\newline\url{https://datatracker.ietf.org/doc/rfc8926/}\\
+	\phantomsection\label{intro:IP}\textbf{[IP]} &
+	INTERNET PROTOCOL
+	\newline\url{https://www.rfc-editor.org/rfc/rfc791}\\
+	\phantomsection\label{intro:UDP}\textbf{[UDP]} &
+	User Datagram Protocol
+	\newline\url{https://www.rfc-editor.org/rfc/rfc768}\\
+	\phantomsection\label{intro:TCP}\textbf{[TCP]} &
+	TRANSMISSION CONTROL PROTOCOL
+	\newline\url{https://www.rfc-editor.org/rfc/rfc793}\\
+
 
 \end{longtable}
 
Signature · 1 line
2.19.1.6.gb485710b

This publicly archived list offers a means to provide input to the OASIS Virtual I/O Device (VIRTIO) TC.

In order to verify user consent to the Feedback License terms and to minimize spam in the list archive, subscription is required before posting.

Subscribe: [email protected]
Unsubscribe: [email protected]
List help: [email protected]
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/

#5 lore raw
Quoted 67 lines · Heng Qi
Quoted 61 lines · Michael S. Tsirkin
Quoted 9 lines · Heng Qi

If the tunnel is used to encapsulate the packets, the hash calculated using the outer header of the receive packets is always fixed for the same flow packets, i.e. they will be steered to the same receive queue.

We add a feature bit VIRTIO_NET_F_HASH_TUNNEL and related bitmasks in \field{hash_types}, which instructs the device to calculate the hash using the inner headers of tunnel-encapsulated packets. Besides, values in \field{hash_report_tunnel} are added to report tunnel types.

Quoted 5 lines

ok close to being ready. a couple of minor comments.

Quoted 42 lines

---
v6:
1. Modify the wording of some sentences for clarity. @Michael S. Tsirkin
2. Fix some syntax issues. @Michael S. Tsirkin

v5:
1. Fix some syntax and capitalization issues. @Michael S. Tsirkin
2. Use encapsulated/encaptulation uniformly. @Michael S. Tsirkin
3. Move the links to introduction section. @Michael S. Tsirkin
4. Clarify some sentences. @Michael S. Tsirkin

v4:
1. Clarify some paragraphs. @Cornelia Huck
2. Fix the u8 type. @Cornelia Huck

v3:
1. Rename VIRTIO_NET_F_HASH_GRE_VXLAN_GENEVE_INNER to VIRTIO_NET_F_HASH_TUNNEL. @Jason Wang
2. Make things clearer. @Jason Wang @Michael S. Tsirkin
3. Keep the possibility to use inner hash for automatic receive steering. @Jason Wang
4. Add the "Tunnel packet" paragraph to avoid repeating the GRE etc. many times. @Michael S. Tsirkin

v2:
1. Add a feature bit for GRE/VXLAN/GENEVE inner hash. @Jason Wang
2. Chang \field{hash_tunnel} to \field{hash_report_tunnel}. @Jason Wang, @Michael S. Tsirkin

v1:
1. Remove the patch for the bitmask fix. @Michael S. Tsirkin
2. Clarify some paragraphs. @Jason Wang
3. Add \field{hash_tunnel} and VIRTIO_NET_HASH_REPORT_GRE. @Yuri Benditovich

 content.tex      | 191 +++++++++++++++++++++++++++++++++++++++++++++--
 introduction.tex |  19 +++++
 2 files changed, 203 insertions(+), 7 deletions(-)
content.tex +0 -0
index e863709..7845f6c 100644
@@ -3084,6 +3084,9 @@ \subsection{Feature bits}\label{sec:Device Types / Network Device / Feature bits
 \item[VIRTIO_NET_F_CTRL_MAC_ADDR(23)] Set MAC address through control
     channel.
 
Quoted 4 lines

content.tex \subsection{Feature bits}\label{sec:Device Types / Network Device / Feature bits · line 3088

Quoted 2 lines
content.tex +2 -0
+\item[VIRTIO_NET_F_HASH_TUNNEL(52)] Device supports inner
+    header hash for GRE, VXLAN and GENEVE tunnel-encapsulated packets.

I would probably drop the list of tunnel types here.

Quoted 8 lines

Do you mean to use "Device supports inner header hash for tunnel-encapsulated packets." instead? Why? We do want to use this feature bit to indicate that the device supports inner hashing of GRE, VXLAN and GENEVE encapsulated packets. As in the v3 discussion
https://lists.oasis-open.org/archives/virtio-dev/202212/msg00024.html , we discussed using VIRTIO_NET_F_HASH_TUNNEL to replace VIRTIO_NET_F_HASH_GRE_VXLAN_GENEVE_INNER and plan to use VIRTIO_NET_F_HASH_TUNNEL_XYZ for future extensions.

So imagine we add a new tunnel type. Let's say there's VXLAN v2. why would we need a new feature bit? I think a new hash type will be sufficient. No?

Signature · 1 line
MST
author
#6 lore raw
Quoted 77 lines · Michael S. Tsirkin
Quoted 67 lines · Heng Qi
Quoted 61 lines · Michael S. Tsirkin
Quoted 9 lines · Heng Qi

If the tunnel is used to encapsulate the packets, the hash calculated using the outer header of the receive packets is always fixed for the same flow packets, i.e. they will be steered to the same receive queue.

We add a feature bit VIRTIO_NET_F_HASH_TUNNEL and related bitmasks in \field{hash_types}, which instructs the device to calculate the hash using the inner headers of tunnel-encapsulated packets. Besides, values in \field{hash_report_tunnel} are added to report tunnel types.

Quoted 5 lines

ok close to being ready. a couple of minor comments.

Quoted 42 lines

---
v6:
1. Modify the wording of some sentences for clarity. @Michael S. Tsirkin
2. Fix some syntax issues. @Michael S. Tsirkin

v5:
1. Fix some syntax and capitalization issues. @Michael S. Tsirkin
2. Use encapsulated/encaptulation uniformly. @Michael S. Tsirkin
3. Move the links to introduction section. @Michael S. Tsirkin
4. Clarify some sentences. @Michael S. Tsirkin

v4:
1. Clarify some paragraphs. @Cornelia Huck
2. Fix the u8 type. @Cornelia Huck

v3:
1. Rename VIRTIO_NET_F_HASH_GRE_VXLAN_GENEVE_INNER to VIRTIO_NET_F_HASH_TUNNEL. @Jason Wang
2. Make things clearer. @Jason Wang @Michael S. Tsirkin
3. Keep the possibility to use inner hash for automatic receive steering. @Jason Wang
4. Add the "Tunnel packet" paragraph to avoid repeating the GRE etc. many times. @Michael S. Tsirkin

v2:
1. Add a feature bit for GRE/VXLAN/GENEVE inner hash. @Jason Wang
2. Chang \field{hash_tunnel} to \field{hash_report_tunnel}. @Jason Wang, @Michael S. Tsirkin

v1:
1. Remove the patch for the bitmask fix. @Michael S. Tsirkin
2. Clarify some paragraphs. @Jason Wang
3. Add \field{hash_tunnel} and VIRTIO_NET_HASH_REPORT_GRE. @Yuri Benditovich

 content.tex      | 191 +++++++++++++++++++++++++++++++++++++++++++++--
 introduction.tex |  19 +++++
 2 files changed, 203 insertions(+), 7 deletions(-)
content.tex +0 -0
index e863709..7845f6c 100644
@@ -3084,6 +3084,9 @@ \subsection{Feature bits}\label{sec:Device Types / Network Device / Feature bits
 \item[VIRTIO_NET_F_CTRL_MAC_ADDR(23)] Set MAC address through control
     channel.
 
Quoted 4 lines

content.tex \subsection{Feature bits}\label{sec:Device Types / Network Device / Feature bits · line 3088

Quoted 2 lines
content.tex +2 -0
+\item[VIRTIO_NET_F_HASH_TUNNEL(52)] Device supports inner
+    header hash for GRE, VXLAN and GENEVE tunnel-encapsulated packets.

I would probably drop the list of tunnel types here.

Quoted 8 lines

Do you mean to use "Device supports inner header hash for tunnel-encapsulated packets." instead? Why? We do want to use this feature bit to indicate that the device supports inner hashing of GRE, VXLAN and GENEVE encapsulated packets. As in the v3 discussion
https://lists.oasis-open.org/archives/virtio-dev/202212/msg00024.html , we discussed using VIRTIO_NET_F_HASH_TUNNEL to replace VIRTIO_NET_F_HASH_GRE_VXLAN_GENEVE_INNER and plan to use VIRTIO_NET_F_HASH_TUNNEL_XYZ for future extensions.

Quoted 3 lines

So imagine we add a new tunnel type. Let's say there's VXLAN v2. why would we need a new feature bit? I think a new hash type will be sufficient. No?

If the description for VIRTIO_NET_F_HASH_TUNNEL is as follows:
"[VIRTIO_NET_F_HASH_TUNNEL(52)] Device supports inner header hash for tunnel-encapsulated packets.". Then the following may happen
1. For VXLANv2, if both device src and device dst have negotiated this feature, it is assumed that device src supports VXLAN and VXLANv2, but device dst may only support VXLAN, not VXLANv2.
2. For other encapsulation protocols such as ip in ip, after device src and device dst have negotiated this feature, it is assumed that device src supports GRE, VXLAN, GENEVE and ip in ip, but it is not clear that device dst also supports ip in ip. Especially when migrating, this can lead to inconsistencies in live migrations.
So, I think it's better to keep the original description:
"[VIRTIO_NET_F_HASH_TUNNEL(52)] Device supports inner header hash for GRE, VXLAN and GENEVE tunnel-encapsulated packets."

Thanks.

Quoted 8 lines
Signature · 7 lines
-- 
MST


---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

This publicly archived list offers a means to provide input to the OASIS Virtual I/O Device (VIRTIO) TC.

In order to verify user consent to the Feedback License terms and to minimize spam in the list archive, subscription is required before posting.

Subscribe: [email protected]
Unsubscribe: [email protected]
List help: [email protected]
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/

#7 lore raw
Quoted 83 lines · Heng Qi
Quoted 77 lines · Michael S. Tsirkin
Quoted 67 lines · Heng Qi
Quoted 61 lines · Michael S. Tsirkin
Quoted 9 lines · Heng Qi

If the tunnel is used to encapsulate the packets, the hash calculated using the outer header of the receive packets is always fixed for the same flow packets, i.e. they will be steered to the same receive queue.

We add a feature bit VIRTIO_NET_F_HASH_TUNNEL and related bitmasks in \field{hash_types}, which instructs the device to calculate the hash using the inner headers of tunnel-encapsulated packets. Besides, values in \field{hash_report_tunnel} are added to report tunnel types.

Quoted 5 lines

ok close to being ready. a couple of minor comments.

Quoted 42 lines

---
v6:
1. Modify the wording of some sentences for clarity. @Michael S. Tsirkin
2. Fix some syntax issues. @Michael S. Tsirkin

v5:
1. Fix some syntax and capitalization issues. @Michael S. Tsirkin
2. Use encapsulated/encaptulation uniformly. @Michael S. Tsirkin
3. Move the links to introduction section. @Michael S. Tsirkin
4. Clarify some sentences. @Michael S. Tsirkin

v4:
1. Clarify some paragraphs. @Cornelia Huck
2. Fix the u8 type. @Cornelia Huck

v3:
1. Rename VIRTIO_NET_F_HASH_GRE_VXLAN_GENEVE_INNER to VIRTIO_NET_F_HASH_TUNNEL. @Jason Wang
2. Make things clearer. @Jason Wang @Michael S. Tsirkin
3. Keep the possibility to use inner hash for automatic receive steering. @Jason Wang
4. Add the "Tunnel packet" paragraph to avoid repeating the GRE etc. many times. @Michael S. Tsirkin

v2:
1. Add a feature bit for GRE/VXLAN/GENEVE inner hash. @Jason Wang
2. Chang \field{hash_tunnel} to \field{hash_report_tunnel}. @Jason Wang, @Michael S. Tsirkin

v1:
1. Remove the patch for the bitmask fix. @Michael S. Tsirkin
2. Clarify some paragraphs. @Jason Wang
3. Add \field{hash_tunnel} and VIRTIO_NET_HASH_REPORT_GRE. @Yuri Benditovich

 content.tex      | 191 +++++++++++++++++++++++++++++++++++++++++++++--
 introduction.tex |  19 +++++
 2 files changed, 203 insertions(+), 7 deletions(-)
content.tex +0 -0
index e863709..7845f6c 100644
@@ -3084,6 +3084,9 @@ \subsection{Feature bits}\label{sec:Device Types / Network Device / Feature bits
 \item[VIRTIO_NET_F_CTRL_MAC_ADDR(23)] Set MAC address through control
     channel.
 
Quoted 4 lines

content.tex \subsection{Feature bits}\label{sec:Device Types / Network Device / Feature bits · line 3088

Quoted 2 lines
content.tex +2 -0
+\item[VIRTIO_NET_F_HASH_TUNNEL(52)] Device supports inner
+    header hash for GRE, VXLAN and GENEVE tunnel-encapsulated packets.

I would probably drop the list of tunnel types here.

Quoted 8 lines

Do you mean to use "Device supports inner header hash for tunnel-encapsulated packets." instead? Why? We do want to use this feature bit to indicate that the device supports inner hashing of GRE, VXLAN and GENEVE encapsulated packets. As in the v3 discussion
https://lists.oasis-open.org/archives/virtio-dev/202212/msg00024.html , we discussed using VIRTIO_NET_F_HASH_TUNNEL to replace VIRTIO_NET_F_HASH_GRE_VXLAN_GENEVE_INNER and plan to use VIRTIO_NET_F_HASH_TUNNEL_XYZ for future extensions.

Quoted 3 lines

So imagine we add a new tunnel type. Let's say there's VXLAN v2. why would we need a new feature bit? I think a new hash type will be sufficient. No?

Quoted 9 lines

If the description for VIRTIO_NET_F_HASH_TUNNEL is as follows:
"[VIRTIO_NET_F_HASH_TUNNEL(52)] Device supports inner header hash for tunnel-encapsulated packets.". Then the following may happen
1. For VXLANv2, if both device src and device dst have negotiated this feature, it is assumed that device src supports VXLAN and VXLANv2, but device dst may only support VXLAN, not VXLANv2.
2. For other encapsulation protocols such as ip in ip, after device src and device dst have negotiated this feature, it is assumed that device src supports GRE, VXLAN, GENEVE and ip in ip, but it is not clear that device dst also supports ip in ip. Especially when migrating, this can lead to inconsistencies in live migrations.

Yes, this looks like the only way that I can think of to keep migration compatibility in an easy way.

Thanks

Quoted 14 lines

So, I think it's better to keep the original description:
"[VIRTIO_NET_F_HASH_TUNNEL(52)] Device supports inner header hash for GRE, VXLAN and GENEVE tunnel-encapsulated packets."

Thanks.

Quoted 8 lines

--
MST

---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]

#8 lore raw
Quoted 84 lines · Jason Wang
Quoted 83 lines · Heng Qi
Quoted 80 lines · Michael S. Tsirkin
Quoted 67 lines · Heng Qi
Quoted 61 lines · Michael S. Tsirkin
Quoted 9 lines · Heng Qi

If the tunnel is used to encapsulate the packets, the hash calculated using the outer header of the receive packets is always fixed for the same flow packets, i.e. they will be steered to the same receive queue.

We add a feature bit VIRTIO_NET_F_HASH_TUNNEL and related bitmasks in \field{hash_types}, which instructs the device to calculate the hash using the inner headers of tunnel-encapsulated packets. Besides, values in \field{hash_report_tunnel} are added to report tunnel types.

Quoted 5 lines

ok close to being ready. a couple of minor comments.

Quoted 42 lines

---
v6:
1. Modify the wording of some sentences for clarity. @Michael S. Tsirkin
2. Fix some syntax issues. @Michael S. Tsirkin

v5:
1. Fix some syntax and capitalization issues. @Michael S. Tsirkin
2. Use encapsulated/encaptulation uniformly. @Michael S. Tsirkin
3. Move the links to introduction section. @Michael S. Tsirkin
4. Clarify some sentences. @Michael S. Tsirkin

v4:
1. Clarify some paragraphs. @Cornelia Huck
2. Fix the u8 type. @Cornelia Huck

v3:
1. Rename VIRTIO_NET_F_HASH_GRE_VXLAN_GENEVE_INNER to VIRTIO_NET_F_HASH_TUNNEL. @Jason Wang
2. Make things clearer. @Jason Wang @Michael S. Tsirkin
3. Keep the possibility to use inner hash for automatic receive steering. @Jason Wang
4. Add the "Tunnel packet" paragraph to avoid repeating the GRE etc. many times. @Michael S. Tsirkin

v2:
1. Add a feature bit for GRE/VXLAN/GENEVE inner hash. @Jason Wang
2. Chang \field{hash_tunnel} to \field{hash_report_tunnel}. @Jason Wang, @Michael S. Tsirkin

v1:
1. Remove the patch for the bitmask fix. @Michael S. Tsirkin
2. Clarify some paragraphs. @Jason Wang
3. Add \field{hash_tunnel} and VIRTIO_NET_HASH_REPORT_GRE. @Yuri Benditovich

 content.tex      | 191 +++++++++++++++++++++++++++++++++++++++++++++--
 introduction.tex |  19 +++++
 2 files changed, 203 insertions(+), 7 deletions(-)
content.tex +0 -0
index e863709..7845f6c 100644
@@ -3084,6 +3084,9 @@ \subsection{Feature bits}\label{sec:Device Types / Network Device / Feature bits
 \item[VIRTIO_NET_F_CTRL_MAC_ADDR(23)] Set MAC address through control
     channel.
 
Quoted 4 lines

content.tex \subsection{Feature bits}\label{sec:Device Types / Network Device / Feature bits · line 3088

Quoted 2 lines
content.tex +2 -0
+\item[VIRTIO_NET_F_HASH_TUNNEL(52)] Device supports inner
+    header hash for GRE, VXLAN and GENEVE tunnel-encapsulated packets.

I would probably drop the list of tunnel types here.

Quoted 8 lines

Do you mean to use "Device supports inner header hash for tunnel-encapsulated packets." instead? Why? We do want to use this feature bit to indicate that the device supports inner hashing of GRE, VXLAN and GENEVE encapsulated packets. As in the v3 discussion
https://lists.oasis-open.org/archives/virtio-dev/202212/msg00024.html , we discussed using VIRTIO_NET_F_HASH_TUNNEL to replace VIRTIO_NET_F_HASH_GRE_VXLAN_GENEVE_INNER and plan to use VIRTIO_NET_F_HASH_TUNNEL_XYZ for future extensions.

So imagine we add a new tunnel type. Let's say there's VXLAN v2. why would we need a new feature bit? I think a new hash type will be sufficient. No?

Quoted 9 lines
Quoted 9 lines

If the description for VIRTIO_NET_F_HASH_TUNNEL is as follows:
"[VIRTIO_NET_F_HASH_TUNNEL(52)] Device supports inner header hash for tunnel-encapsulated packets.". Then the following may happen
1. For VXLANv2, if both device src and device dst have negotiated this feature, it is assumed that device src supports VXLAN and VXLANv2, but device dst may only support VXLAN, not VXLANv2.
2. For other encapsulation protocols such as ip in ip, after device src and device dst have negotiated this feature, it is assumed that device src supports GRE, VXLAN, GENEVE and ip in ip, but it is not clear that device dst also supports ip in ip. Especially when migrating, this can lead to inconsistencies in live migrations.

it does not matter. e.g. if device supports GRE but not VXLAN then feature bit it set but hash types are different.
and this is not different from any other offload we have.

Generally the only reason we even use a new feature bit for this tunneling is because the header structure is a bit different so this kind of makes sense.

Quoted 5 lines

Yes, this looks like the only way that I can think of to keep migration compatibility in an easy way.

Thanks

Jason migration compatibility must also check hash types. There's no chance with the current hash offload scheme to only rely on feature bits for migration compatibility.

Quoted 14 lines
Quoted 14 lines

So, I think it's better to keep the original description:
"[VIRTIO_NET_F_HASH_TUNNEL(52)] Device supports inner header hash for GRE, VXLAN and GENEVE tunnel-encapsulated packets."

Thanks.

Quoted 8 lines

--
MST

---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]

#9 lore raw
Quoted 96 lines · Michael S. Tsirkin
Quoted 84 lines · Jason Wang
Quoted 83 lines · Heng Qi
Quoted 80 lines · Michael S. Tsirkin
Quoted 67 lines · Heng Qi
Quoted 61 lines · Michael S. Tsirkin
Quoted 9 lines · Heng Qi

If the tunnel is used to encapsulate the packets, the hash calculated using the outer header of the receive packets is always fixed for the same flow packets, i.e. they will be steered to the same receive queue.

We add a feature bit VIRTIO_NET_F_HASH_TUNNEL and related bitmasks in \field{hash_types}, which instructs the device to calculate the hash using the inner headers of tunnel-encapsulated packets. Besides, values in \field{hash_report_tunnel} are added to report tunnel types.

Quoted 5 lines

ok close to being ready. a couple of minor comments.

Quoted 42 lines

---
v6:
1. Modify the wording of some sentences for clarity. @Michael S. Tsirkin
2. Fix some syntax issues. @Michael S. Tsirkin

v5:
1. Fix some syntax and capitalization issues. @Michael S. Tsirkin
2. Use encapsulated/encaptulation uniformly. @Michael S. Tsirkin
3. Move the links to introduction section. @Michael S. Tsirkin
4. Clarify some sentences. @Michael S. Tsirkin

v4:
1. Clarify some paragraphs. @Cornelia Huck
2. Fix the u8 type. @Cornelia Huck

v3:
1. Rename VIRTIO_NET_F_HASH_GRE_VXLAN_GENEVE_INNER to VIRTIO_NET_F_HASH_TUNNEL. @Jason Wang
2. Make things clearer. @Jason Wang @Michael S. Tsirkin
3. Keep the possibility to use inner hash for automatic receive steering. @Jason Wang
4. Add the "Tunnel packet" paragraph to avoid repeating the GRE etc. many times. @Michael S. Tsirkin

v2:
1. Add a feature bit for GRE/VXLAN/GENEVE inner hash. @Jason Wang
2. Chang \field{hash_tunnel} to \field{hash_report_tunnel}. @Jason Wang, @Michael S. Tsirkin

v1:
1. Remove the patch for the bitmask fix. @Michael S. Tsirkin
2. Clarify some paragraphs. @Jason Wang
3. Add \field{hash_tunnel} and VIRTIO_NET_HASH_REPORT_GRE. @Yuri Benditovich

 content.tex      | 191 +++++++++++++++++++++++++++++++++++++++++++++--
 introduction.tex |  19 +++++
 2 files changed, 203 insertions(+), 7 deletions(-)
content.tex +0 -0
index e863709..7845f6c 100644
@@ -3084,6 +3084,9 @@ \subsection{Feature bits}\label{sec:Device Types / Network Device / Feature bits
 \item[VIRTIO_NET_F_CTRL_MAC_ADDR(23)] Set MAC address through control
     channel.
 
Quoted 4 lines

content.tex \subsection{Feature bits}\label{sec:Device Types / Network Device / Feature bits · line 3088

Quoted 2 lines
content.tex +2 -0
+\item[VIRTIO_NET_F_HASH_TUNNEL(52)] Device supports inner
+    header hash for GRE, VXLAN and GENEVE tunnel-encapsulated packets.

I would probably drop the list of tunnel types here.

Quoted 8 lines

Do you mean to use "Device supports inner header hash for tunnel-encapsulated packets." instead? Why? We do want to use this feature bit to indicate that the device supports inner hashing of GRE, VXLAN and GENEVE encapsulated packets. As in the v3 discussion
https://lists.oasis-open.org/archives/virtio-dev/202212/msg00024.html , we discussed using VIRTIO_NET_F_HASH_TUNNEL to replace VIRTIO_NET_F_HASH_GRE_VXLAN_GENEVE_INNER and plan to use VIRTIO_NET_F_HASH_TUNNEL_XYZ for future extensions.

So imagine we add a new tunnel type. Let's say there's VXLAN v2. why would we need a new feature bit? I think a new hash type will be sufficient. No?

Quoted 9 lines
Quoted 9 lines

If the description for VIRTIO_NET_F_HASH_TUNNEL is as follows:
"[VIRTIO_NET_F_HASH_TUNNEL(52)] Device supports inner header hash for tunnel-encapsulated packets.". Then the following may happen
1. For VXLANv2, if both device src and device dst have negotiated this feature, it is assumed that device src supports VXLAN and VXLANv2, but device dst may only support VXLAN, not VXLANv2.
2. For other encapsulation protocols such as ip in ip, after device src and device dst have negotiated this feature, it is assumed that device src supports GRE, VXLAN, GENEVE and ip in ip, but it is not clear that device dst also supports ip in ip. Especially when migrating, this can lead to inconsistencies in live migrations.

Quoted 2 lines

it does not matter. e.g. if device supports GRE but not VXLAN then feature bit it set but hash types are different.

But the hash will be used for RSS steering. Consider the steering on ipip work on src but not on dst it would break application logic.

Quoted 1 line

and this is not different from any other offload we have.

But we have different feature bits for TCPv4/v6.

Quoted 6 lines

Generally the only reason we even use a new feature bit for this tunneling is because the header structure is a bit different so this kind of makes sense.

Quoted 1 line
Quoted 8 lines
Quoted 4 lines

Yes, this looks like the only way that I can think of to keep migration compatibility in an easy way.

Thanks

Jason migration compatibility must also check hash types. There's no chance with the current hash offload scheme to only rely on feature bits for migration compatibility.

This will give extra pressure on the management stack, e.g it requires the device to have an out of spec way for introspection.

Thanks

Quoted 17 lines
Quoted 16 lines
Quoted 16 lines

So, I think it's better to keep the original description:
"[VIRTIO_NET_F_HASH_TUNNEL(52)] Device supports inner header hash for GRE, VXLAN and GENEVE tunnel-encapsulated packets."

Thanks.

Quoted 8 lines

--
MST

---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]

#10 lore raw
Quoted 97 lines · Jason Wang
Quoted 96 lines · Michael S. Tsirkin
Quoted 93 lines · Jason Wang
Quoted 92 lines · Heng Qi
Quoted 77 lines · Michael S. Tsirkin
Quoted 67 lines · Heng Qi
Quoted 61 lines · Michael S. Tsirkin
Quoted 9 lines · Heng Qi

If the tunnel is used to encapsulate the packets, the hash calculated using the outer header of the receive packets is always fixed for the same flow packets, i.e. they will be steered to the same receive queue.

We add a feature bit VIRTIO_NET_F_HASH_TUNNEL and related bitmasks in \field{hash_types}, which instructs the device to calculate the hash using the inner headers of tunnel-encapsulated packets. Besides, values in \field{hash_report_tunnel} are added to report tunnel types.

Quoted 5 lines

ok close to being ready. a couple of minor comments.

Quoted 42 lines

---
v6:
1. Modify the wording of some sentences for clarity. @Michael S. Tsirkin
2. Fix some syntax issues. @Michael S. Tsirkin

v5:
1. Fix some syntax and capitalization issues. @Michael S. Tsirkin
2. Use encapsulated/encaptulation uniformly. @Michael S. Tsirkin
3. Move the links to introduction section. @Michael S. Tsirkin
4. Clarify some sentences. @Michael S. Tsirkin

v4:
1. Clarify some paragraphs. @Cornelia Huck
2. Fix the u8 type. @Cornelia Huck

v3:
1. Rename VIRTIO_NET_F_HASH_GRE_VXLAN_GENEVE_INNER to VIRTIO_NET_F_HASH_TUNNEL. @Jason Wang
2. Make things clearer. @Jason Wang @Michael S. Tsirkin
3. Keep the possibility to use inner hash for automatic receive steering. @Jason Wang
4. Add the "Tunnel packet" paragraph to avoid repeating the GRE etc. many times. @Michael S. Tsirkin

v2:
1. Add a feature bit for GRE/VXLAN/GENEVE inner hash. @Jason Wang
2. Chang \field{hash_tunnel} to \field{hash_report_tunnel}. @Jason Wang, @Michael S. Tsirkin

v1:
1. Remove the patch for the bitmask fix. @Michael S. Tsirkin
2. Clarify some paragraphs. @Jason Wang
3. Add \field{hash_tunnel} and VIRTIO_NET_HASH_REPORT_GRE. @Yuri Benditovich

 content.tex      | 191 +++++++++++++++++++++++++++++++++++++++++++++--
 introduction.tex |  19 +++++
 2 files changed, 203 insertions(+), 7 deletions(-)
content.tex +0 -0
index e863709..7845f6c 100644
@@ -3084,6 +3084,9 @@ \subsection{Feature bits}\label{sec:Device Types / Network Device / Feature bits
 \item[VIRTIO_NET_F_CTRL_MAC_ADDR(23)] Set MAC address through control
     channel.
 
Quoted 4 lines

content.tex \subsection{Feature bits}\label{sec:Device Types / Network Device / Feature bits · line 3088

Quoted 2 lines
content.tex +2 -0
+\item[VIRTIO_NET_F_HASH_TUNNEL(52)] Device supports inner
+    header hash for GRE, VXLAN and GENEVE tunnel-encapsulated packets.

I would probably drop the list of tunnel types here.

Quoted 8 lines

Do you mean to use "Device supports inner header hash for tunnel-encapsulated packets." instead? Why? We do want to use this feature bit to indicate that the device supports inner hashing of GRE, VXLAN and GENEVE encapsulated packets. As in the v3 discussion
https://lists.oasis-open.org/archives/virtio-dev/202212/msg00024.html , we discussed using VIRTIO_NET_F_HASH_TUNNEL to replace VIRTIO_NET_F_HASH_GRE_VXLAN_GENEVE_INNER and plan to use VIRTIO_NET_F_HASH_TUNNEL_XYZ for future extensions.

Quoted 3 lines

So imagine we add a new tunnel type. Let's say there's VXLAN v2. why would we need a new feature bit? I think a new hash type will be sufficient. No?

If the description for VIRTIO_NET_F_HASH_TUNNEL is as follows:
"[VIRTIO_NET_F_HASH_TUNNEL(52)] Device supports inner header hash for tunnel-encapsulated packets.". Then the following may happen
1. For VXLANv2, if both device src and device dst have negotiated this feature, it is assumed that device src supports VXLAN and VXLANv2, but device dst may only support VXLAN, not VXLANv2.
2. For other encapsulation protocols such as ip in ip, after device src and device dst have negotiated this feature, it is assumed that device src supports GRE, VXLAN, GENEVE and ip in ip, but it is not clear that device dst also supports ip in ip. Especially when migrating, this can lead to inconsistencies in live migrations.

Quoted 5 lines
Quoted 2 lines

it does not matter. e.g. if device supports GRE but not VXLAN then feature bit it set but hash types are different.

But the hash will be used for RSS steering. Consider the steering on ipip work on src but not on dst it would break application logic.

Exactly. And for this reason just making sure feature bits match is not sufficient for migration. It follows that hash types
must also match, and therefore there is no need to invent
new feature bits for migration if all we are doing is adding new hash types.

Quoted 3 lines
Quoted 1 line

and this is not different from any other offload we have.

But we have different feature bits for TCPv4/v6.

For segmentation but not for hash. Basically because we don't have segmentation types analogous to hash types.

Quoted 11 lines
Quoted 11 lines

Generally the only reason we even use a new feature bit for this tunneling is because the header structure is a bit different so this kind of makes sense.

Quoted 5 lines

Yes, this looks like the only way that I can think of to keep migration compatibility in an easy way.

Thanks

Quoted 8 lines
Quoted 3 lines

Jason migration compatibility must also check hash types. There's no chance with the current hash offload scheme to only rely on feature bits for migration compatibility.

This will give extra pressure on the management stack, e.g it requires the device to have an out of spec way for introspection.

Thanks

As I tried to explain this is already the case. Feature bits do not describe device capabilities fully, some of them are in config space.

Quoted 17 lines
Quoted 17 lines
Quoted 16 lines
Quoted 16 lines

So, I think it's better to keep the original description:
"[VIRTIO_NET_F_HASH_TUNNEL(52)] Device supports inner header hash for GRE, VXLAN and GENEVE tunnel-encapsulated packets."

Thanks.

Quoted 8 lines

--
MST

---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]

#11 lore raw
Quoted 7 lines · Michael S. Tsirkin
Quoted 4 lines

This will give extra pressure on the management stack, e.g it requires the device to have an out of spec way for introspection.

Thanks

As I tried to explain this is already the case. Feature bits do not describe device capabilities fully, some of them are in config space.

To be precise, this does not necessarily require introspection, but it does require management control over config space
such as supported hash types just like it has control over feature bits. E.g. QEMU currently seems to hard-code these to

#define VIRTIO_NET_RSS_SUPPORTED_HASHES (VIRTIO_NET_RSS_HASH_TYPE_IPv4 | \
                                         VIRTIO_NET_RSS_HASH_TYPE_TCPv4 | \
                                         VIRTIO_NET_RSS_HASH_TYPE_UDPv4 | \
                                         VIRTIO_NET_RSS_HASH_TYPE_IPv6 | \
                                         VIRTIO_NET_RSS_HASH_TYPE_TCPv6 | \
                                         VIRTIO_NET_RSS_HASH_TYPE_UDPv6 | \
                                         VIRTIO_NET_RSS_HASH_TYPE_IP_EX | \
                                         VIRTIO_NET_RSS_HASH_TYPE_TCP_EX | \
                                         VIRTIO_NET_RSS_HASH_TYPE_UDP_EX)

but there's no reason not to give management control over these.

Signature · 1 line
MST
author
#12 lore raw
Quoted 23 lines · Michael S. Tsirkin
Quoted 7 lines · Michael S. Tsirkin
Quoted 4 lines

This will give extra pressure on the management stack, e.g it requires the device to have an out of spec way for introspection.

Thanks

As I tried to explain this is already the case. Feature bits do not describe device capabilities fully, some of them are in config space.

To be precise, this does not necessarily require introspection, but it does require management control over config space
such as supported hash types just like it has control over feature bits. E.g. QEMU currently seems to hard-code these to

#define VIRTIO_NET_RSS_SUPPORTED_HASHES (VIRTIO_NET_RSS_HASH_TYPE_IPv4 | \
                                         VIRTIO_NET_RSS_HASH_TYPE_TCPv4 | \
                                         VIRTIO_NET_RSS_HASH_TYPE_UDPv4 | \
                                         VIRTIO_NET_RSS_HASH_TYPE_IPv6 | \
                                         VIRTIO_NET_RSS_HASH_TYPE_TCPv6 | \
                                         VIRTIO_NET_RSS_HASH_TYPE_UDPv6 | \
                                         VIRTIO_NET_RSS_HASH_TYPE_IP_EX | \
                                         VIRTIO_NET_RSS_HASH_TYPE_TCP_EX | \
                                         VIRTIO_NET_RSS_HASH_TYPE_UDP_EX)
Quoted 1 line

but there's no reason not to give management control over these.

Yes, QEMU has requirements for live migration: the PCI config space will be checked in get_pci_config_device(), and if src and dst are inconsistent, it will prompt that the live migration failed.
In fact, this is also done within our group. Live migration requires that the two VMs have the same rss configuration, otherwise the migration will fail.

Therefore, it seems that we can regularize the description of VIRTIO_NET_F_HASH_TUNNEL into "[VIRTIO_NET_F_HASH_TUNNEL(52)] Device supports inner header hash for tunnel-encapsulated packets.", and use different hash_types to help the migration determine whether it can succeed.

Thanks.

Quoted 3 lines
Signature · 2 lines
-- 
MST
author
#13 lore raw
Quoted 22 lines · Heng Qi
Quoted 21 lines · Michael S. Tsirkin
Quoted 6 lines · Michael S. Tsirkin
Quoted 4 lines

This will give extra pressure on the management stack, e.g it requires the device to have an out of spec way for introspection.

Thanks

As I tried to explain this is already the case. Feature bits do not describe device capabilities fully, some of them are in config space.

To be precise, this does not necessarily require introspection, but it does require management control over config space
such as supported hash types just like it has control over feature bits. E.g. QEMU currently seems to hard-code these to

#define VIRTIO_NET_RSS_SUPPORTED_HASHES (VIRTIO_NET_RSS_HASH_TYPE_IPv4 | \
                                          VIRTIO_NET_RSS_HASH_TYPE_TCPv4 | \
                                          VIRTIO_NET_RSS_HASH_TYPE_UDPv4 | \
                                          VIRTIO_NET_RSS_HASH_TYPE_IPv6 | \
                                          VIRTIO_NET_RSS_HASH_TYPE_TCPv6 | \
                                          VIRTIO_NET_RSS_HASH_TYPE_UDPv6 | \
                                          VIRTIO_NET_RSS_HASH_TYPE_IP_EX | \
                                          VIRTIO_NET_RSS_HASH_TYPE_TCP_EX | \
                                          VIRTIO_NET_RSS_HASH_TYPE_UDP_EX)
Quoted 4 lines
Quoted 1 line

but there's no reason not to give management control over these.

Yes, QEMU has requirements for live migration: the PCI config space will be checked in get_pci_config_device(), and if src and dst are inconsistent, it will prompt that the live migration failed.

To be clearer, I mean \filed{supported_hash_types} in structure virtio_net_config.

Thanks.

Quoted 26 lines

In fact, this is also done within our group. Live migration requires that the two VMs have the same rss configuration, otherwise the migration will fail.

Therefore, it seems that we can regularize the description of VIRTIO_NET_F_HASH_TUNNEL into "[VIRTIO_NET_F_HASH_TUNNEL(52)] Device supports inner header hash for tunnel-encapsulated packets.", and use different hash_types to help the migration determine whether it can succeed.

Thanks.

Quoted 2 lines
Signature · 1 line
MST

This publicly archived list offers a means to provide input to the OASIS Virtual I/O Device (VIRTIO) TC.

In order to verify user consent to the Feedback License terms and to minimize spam in the list archive, subscription is required before posting.

Subscribe: [email protected]
Unsubscribe: [email protected]
List help: [email protected]
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/

#14 lore raw
Quoted 4 lines · Heng Qi

在 2023/1/10 下午3:26, Heng Qi 写道:

Quoted 8 lines
Quoted 8 lines · Michael S. Tsirkin
Quoted 7 lines · Michael S. Tsirkin
Quoted 6 lines
Quoted 4 lines

This will give extra pressure on the management stack, e.g it requires the device to have an out of spec way for introspection.

Thanks

As I tried to explain this is already the case. Feature bits do not describe device capabilities fully, some of them are in config space.

Yes.

Quoted 14 lines
Quoted 14 lines
Quoted 14 lines

To be precise, this does not necessarily require introspection, but it does require management control over config space
such as supported hash types just like it has control over feature bits. E.g. QEMU currently seems to hard-code these to

#define VIRTIO_NET_RSS_SUPPORTED_HASHES (VIRTIO_NET_RSS_HASH_TYPE_IPv4 | \
                                          VIRTIO_NET_RSS_HASH_TYPE_TCPv4 | \
                                          VIRTIO_NET_RSS_HASH_TYPE_UDPv4 | \
                                          VIRTIO_NET_RSS_HASH_TYPE_IPv6 | \
                                          VIRTIO_NET_RSS_HASH_TYPE_TCPv6 | \
                                          VIRTIO_NET_RSS_HASH_TYPE_UDPv6 | \
                                          VIRTIO_NET_RSS_HASH_TYPE_IP_EX | \
                                          VIRTIO_NET_RSS_HASH_TYPE_TCP_EX | \
                                          VIRTIO_NET_RSS_HASH_TYPE_UDP_EX)
Quoted 1 line
Quoted 1 line
Quoted 1 line

but there's no reason not to give management control over these.

Note that the management expects the migration compatibility to work with machine types. So it needs a way to disable some tunnel hash types to make it work for old machine types.

Quoted 3 lines
Quoted 3 lines

Yes, QEMU has requirements for live migration: the PCI config space will be checked in get_pci_config_device(), and if src and dst are inconsistent, it will prompt that the live migration failed.

It might be too late since it can't work for the second run (unlike subsection).

Quoted 3 lines

To be clearer, I mean \filed{supported_hash_types} in structure virtio_net_config.

Yes.

Thanks

Quoted 30 lines

Thanks.

Quoted 26 lines

In fact, this is also done within our group. Live migration requires that the two VMs have the same rss configuration, otherwise the migration will fail.

Therefore, it seems that we can regularize the description of VIRTIO_NET_F_HASH_TUNNEL into "[VIRTIO_NET_F_HASH_TUNNEL(52)] Device supports inner header hash for tunnel-encapsulated packets.", and use different hash_types to help the migration determine whether it can succeed.

Thanks.

Quoted 2 lines

--
MST

This publicly archived list offers a means to provide input to the OASIS Virtual I/O Device (VIRTIO) TC.

In order to verify user consent to the Feedback License terms and to minimize spam in the list archive, subscription is required before posting.

Subscribe: [email protected]
Unsubscribe: [email protected]
List help: [email protected]
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/

#15 lore raw
Quoted 30 lines · Jason Wang
Quoted 4 lines · Heng Qi

在 2023/1/10 下午3:26, Heng Qi 写道:

Quoted 8 lines
Quoted 8 lines · Michael S. Tsirkin
Quoted 7 lines · Michael S. Tsirkin
Quoted 6 lines
Quoted 4 lines

This will give extra pressure on the management stack, e.g it requires the device to have an out of spec way for introspection.

Thanks

As I tried to explain this is already the case. Feature bits do not describe device capabilities fully, some of them are in config space.

Yes.

Quoted 14 lines
Quoted 14 lines
Quoted 14 lines

To be precise, this does not necessarily require introspection, but it does require management control over config space
such as supported hash types just like it has control over feature bits. E.g. QEMU currently seems to hard-code these to

#define VIRTIO_NET_RSS_SUPPORTED_HASHES (VIRTIO_NET_RSS_HASH_TYPE_IPv4 | \
                                          VIRTIO_NET_RSS_HASH_TYPE_TCPv4 | \
                                          VIRTIO_NET_RSS_HASH_TYPE_UDPv4 | \
                                          VIRTIO_NET_RSS_HASH_TYPE_IPv6 | \
                                          VIRTIO_NET_RSS_HASH_TYPE_TCPv6 | \
                                          VIRTIO_NET_RSS_HASH_TYPE_UDPv6 | \
                                          VIRTIO_NET_RSS_HASH_TYPE_IP_EX | \
                                          VIRTIO_NET_RSS_HASH_TYPE_TCP_EX | \
                                          VIRTIO_NET_RSS_HASH_TYPE_UDP_EX)
Quoted 5 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line

but there's no reason not to give management control over these.

Note that the management expects the migration compatibility to work with machine types. So it needs a way to disable some tunnel hash types to make it work for old machine types.

yes.
This means qemu will need to create properties for these things and control through machine type compatibility machinery. For those not hacking qemu - "machine type" is
a string roughly describing a version of guest/host interface used.

Quoted 5 lines
Quoted 3 lines
Quoted 3 lines

Yes, QEMU has requirements for live migration: the PCI config space will be checked in get_pci_config_device(), and if src and dst are inconsistent, it will prompt that the live migration failed.

It might be too late since it can't work for the second run (unlike subsection).

This is really a low level detail of qemu. I'm not sure how important this is for the spec.

Quoted 38 lines
Quoted 3 lines

To be clearer, I mean \filed{supported_hash_types} in structure virtio_net_config.

Yes.

Thanks

Quoted 30 lines

Thanks.

Quoted 26 lines

In fact, this is also done within our group. Live migration requires that the two VMs have the same rss configuration, otherwise the migration will fail.

Therefore, it seems that we can regularize the description of VIRTIO_NET_F_HASH_TUNNEL into "[VIRTIO_NET_F_HASH_TUNNEL(52)] Device supports inner header hash for tunnel-encapsulated packets.", and use different hash_types to help the migration determine whether it can succeed.

Thanks.

Quoted 2 lines

--
MST

This publicly archived list offers a means to provide input to the OASIS Virtual I/O Device (VIRTIO) TC.

In order to verify user consent to the Feedback License terms and to minimize spam in the list archive, subscription is required before posting.

Subscribe: [email protected]
Unsubscribe: [email protected]
List help: [email protected]
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/

#16 lore raw
In reply to Heng Qi
Quoted 291 lines · Heng Qi

If the tunnel is used to encapsulate the packets, the hash calculated using the outer header of the receive packets is always fixed for the same flow packets, i.e. they will be steered to the same receive queue.

We add a feature bit VIRTIO_NET_F_HASH_TUNNEL and related bitmasks in \field{hash_types}, which instructs the device to calculate the hash using the inner headers of tunnel-encapsulated packets. Besides, values in \field{hash_report_tunnel} are added to report tunnel types.

---
v6:
1. Modify the wording of some sentences for clarity. @Michael S. Tsirkin
2. Fix some syntax issues. @Michael S. Tsirkin

v5:
1. Fix some syntax and capitalization issues. @Michael S. Tsirkin
2. Use encapsulated/encaptulation uniformly. @Michael S. Tsirkin
3. Move the links to introduction section. @Michael S. Tsirkin
4. Clarify some sentences. @Michael S. Tsirkin

v4:
1. Clarify some paragraphs. @Cornelia Huck
2. Fix the u8 type. @Cornelia Huck

v3:
1. Rename VIRTIO_NET_F_HASH_GRE_VXLAN_GENEVE_INNER to VIRTIO_NET_F_HASH_TUNNEL. @Jason Wang
2. Make things clearer. @Jason Wang @Michael S. Tsirkin
3. Keep the possibility to use inner hash for automatic receive steering. @Jason Wang
4. Add the "Tunnel packet" paragraph to avoid repeating the GRE etc. many times. @Michael S. Tsirkin

v2:
1. Add a feature bit for GRE/VXLAN/GENEVE inner hash. @Jason Wang
2. Chang \field{hash_tunnel} to \field{hash_report_tunnel}. @Jason Wang, @Michael S. Tsirkin

v1:
1. Remove the patch for the bitmask fix. @Michael S. Tsirkin
2. Clarify some paragraphs. @Jason Wang
3. Add \field{hash_tunnel} and VIRTIO_NET_HASH_REPORT_GRE. @Yuri Benditovich

 content.tex      | 191 +++++++++++++++++++++++++++++++++++++++++++++--
 introduction.tex |  19 +++++
 2 files changed, 203 insertions(+), 7 deletions(-)
content.tex +168 -7
index e863709..7845f6c 100644
@@ -3084,6 +3084,9 @@ \subsection{Feature bits}\label{sec:Device Types / Network Device / Feature bits
 \item[VIRTIO_NET_F_CTRL_MAC_ADDR(23)] Set MAC address through control
     channel.
 
+\item[VIRTIO_NET_F_HASH_TUNNEL(52)] Device supports inner
+    header hash for GRE, VXLAN and GENEVE tunnel-encapsulated packets.
+
 \item[VIRTIO_NET_F_NOTF_COAL(53)] Device supports notifications coalescing.
 
 \item[VIRTIO_NET_F_GUEST_USO4 (54)] Driver can receive USOv4 packets.
@@ -3095,7 +3098,8 @@ \subsection{Feature bits}\label{sec:Device Types / Network Device / Feature bits
  to several segments when each of these smaller packets has UDP header.
 
 \item[VIRTIO_NET_F_HASH_REPORT(57)] Device can report per-packet hash
-    value and a type of calculated hash.
+    value, a type of calculated hash, and, if VIRTIO_NET_F_HASH_TUNNEL
+    is negotiated, an encapsulation packet type.
 
 \item[VIRTIO_NET_F_GUEST_HDRLEN(59)] Driver can provide the exact \field{hdr_len}
     value. Device benefits from knowing the exact header length.
@@ -3140,6 +3144,7 @@ \subsubsection{Feature bit requirements}\label{sec:Device Types / Network Device
 \item[VIRTIO_NET_F_NOTF_COAL] Requires VIRTIO_NET_F_CTRL_VQ.
 \item[VIRTIO_NET_F_RSC_EXT] Requires VIRTIO_NET_F_HOST_TSO4 or VIRTIO_NET_F_HOST_TSO6.
 \item[VIRTIO_NET_F_RSS] Requires VIRTIO_NET_F_CTRL_VQ.
+\item[VIRTIO_NET_F_HASH_TUNNEL] Requires VIRTIO_NET_F_CTRL_VQ.
 \end{description}
 
 \subsubsection{Legacy Interface: Feature bits}\label{sec:Device Types / Network Device / Feature bits / Legacy Interface: Feature bits}
@@ -3386,7 +3391,8 @@ \subsection{Device Operation}\label{sec:Device Types / Network Device / Device O
         le16 num_buffers;
         le32 hash_value;        (Only if VIRTIO_NET_F_HASH_REPORT negotiated)
         le16 hash_report;       (Only if VIRTIO_NET_F_HASH_REPORT negotiated)
-        le16 padding_reserved;  (Only if VIRTIO_NET_F_HASH_REPORT negotiated)
+        u8 hash_report_tunnel;  (Only if VIRTIO_NET_F_HASH_REPORT negotiated, only valid of VIRTIO_NET_F_HASH_TUNNEL negotiated)
+        u8 padding_reserved;    (Only if VIRTIO_NET_F_HASH_REPORT negotiated)
 };
 \end{lstlisting}
 
@@ -3837,7 +3843,8 @@ \subsubsection{Processing of Incoming Packets}\label{sec:Device Types / Network
 A device attempts to calculate a per-packet hash in the following cases:
 \begin{itemize}
 \item The feature VIRTIO_NET_F_RSS was negotiated. The device uses the hash to determine the receive virtqueue to place incoming packets.
-\item The feature VIRTIO_NET_F_HASH_REPORT was negotiated. The device reports the hash value and the hash type with the packet.
+\item The feature VIRTIO_NET_F_HASH_REPORT was negotiated. The device reports the hash value and the hash type. If additionally
+VIRTIO_NET_F_HASH_TUNNEL was negotiated, the device reports the encapsulation type as well.
 \end{itemize}
 
 If the feature VIRTIO_NET_F_RSS was negotiated:
@@ -3863,8 +3870,36 @@ \subsubsection{Processing of Incoming Packets}\label{sec:Device Types / Network
  \ref{sec:Device Types / Network Device / Device Operation / Control Virtqueue / Automatic receive steering in multiqueue mode / Hash calculation}.
 \end{itemize}
 
+\subparagraph{Tunnel/Encapsulated packet}
+\label{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / Tunnel/Encapsulated packet}
+A tunnel packet is encapsulated from the original packet based on the tunneling
+protocol (only a single level of encapsulation is currently supported). The
+encapsulated packet contains an outer header and an inner header, and the device
+calculates the hash over either the inner header or the outer header.
+
+When the feature VIRTIO_NET_F_HASH_TUNNEL is negotiated and the corresponding
+encapsulation type is set in \field{hash_types}, the hash for a specific type of
+encapsulated packet is calculated over the inner as opposed to outer header.
+Supported encapsulation types are listed in \ref{sec:Device Types / Network Device /
+Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets /
+Supported/enabled hash types}.
+
+If both VIRTIO_NET_F_HASH_REPORT and VIRTIO_NET_F_HASH_TUNNEL are negotiated,
+the device can support inner hash calculation for \hyperref[intro:GRE]{[GRE]},
+\hyperref[intro:VXLAN]{[VXLAN]} and \hyperref[intro:GENEVE]{[GENEVE]}
+encapsulated packets, and the corresponding encapsulation type in \field{hash_types}
+is VIRTIO_NET_HASH_TYPE_{GRE, VXLAN, GENEVE}_INNER respectively. The value in
+\field{hash_report_tunnel} is VIRTIO_NET_HASH_REPORT_{NONE, GRE, VXLAN, GENEVE} respectively.
+
+If VIRTIO_NET_F_HASH_REPORT is negotiated but VIRTIO_NET_F_HASH_TUNNEL is not
+negotiated, the device calculates the hash over the outer header, and
+\field{hash_report} reports the hash type. \field{hash_report_tunnel}
+is no longer valid.
+
 \subparagraph{Supported/enabled hash types}
 \label{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / Supported/enabled hash types}
+This paragraph relies on definitions from \hyperref[intro:IP]{[IP]},
+\hyperref[intro:UDP]{[UDP]} and \hyperref[intro:TCP]{[TCP]}.
 Hash types applicable for IPv4 packets:
 \begin{lstlisting}
 #define VIRTIO_NET_HASH_TYPE_IPv4              (1 << 0)
@@ -3884,6 +3919,22 @@ \subsubsection{Processing of Incoming Packets}\label{sec:Device Types / Network
 #define VIRTIO_NET_HASH_TYPE_UDP_EX            (1 << 8)
 \end{lstlisting}
 
+If the feature VIRTIO_NET_F_HASH_TUNNEL is negotiated, the encapsulation
+hash type below indicates that the hash is calculated over the inner
+header of the encapsulated packet:
+Hash type applicable for inner payload of the gre-encapsulated packet
+\begin{lstlisting}
+#define VIRTIO_NET_HASH_TYPE_GRE_INNER         (1 << 9)
+\end{lstlisting}
+Hash type applicable for inner payload of the vxlan-encapsulated packet
+\begin{lstlisting}
+#define VIRTIO_NET_HASH_TYPE_VXLAN_INNER       (1 << 10)
+\end{lstlisting}
+Hash type applicable for inner payload of the geneve-encapsulated packet
+\begin{lstlisting}
+#define VIRTIO_NET_HASH_TYPE_GENEVE_INNER      (1 << 11)
+\end{lstlisting}
+
 \subparagraph{IPv4 packets}
 \label{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / IPv4 packets}
 The device calculates the hash on IPv4 packets according to 'Enabled hash types' bitmask as follows:
@@ -3975,15 +4026,125 @@ \subsubsection{Processing of Incoming Packets}\label{sec:Device Types / Network
 (see \ref{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / IPv6 packets without extension header}).
 \end{itemize}
 
+\subparagraph{Inner payload of an encapsulated packet}
+\label{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / Inner payload of the encapsulated packet}
+If the feature VIRTIO_NET_F_HASH_TUNNEL is negotiated and the corresponding
+encapsulation hash type is set in \field{hash_types}, the device calculates the
+inner header hash on an encapsulated packet (See \ref{sec:Device Types
+/ Network Device / Device Operation / Processing of Incoming Packets /
+Hash calculation for incoming packets / Tunnel/Encapsulated packet}) as follows:
+
+The device calculates the hash on the inner IPv4 packet of an encapsulated packet
+according to 'Enabled hash types' bitmask as follows:
+\begin{itemize}
+  \item If VIRTIO_NET_HASH_TYPE_TCPv4 is set and the encapsulated packet has an inner
+	TCPv4 header, the hash is calculated over the following fields:
+        \begin{itemsize}
+          \item inner Source IP address
+          \item inner Destination IP address
+          \item inner Source TCP port
+          \item inner Destination TCP port
+        \end{itemsize}
+  \item Else if VIRTIO_NET_HASH_TYPE_UDPv4 is set and the encapsulated packet has an
+	inner UDPv4 header, the hash is calculated over the following fields:
+        \begin{itemsize}
+          \item inner Source IP address
+          \item inner Destination IP address
+          \item inner Source UDP port
+          \item inner Destination UDP port
+        \end{itemize}
+  \item Else if VIRTIO_NET_HASH_TYPE_IPv4 is set, the hash is calculated over the
+	following fields:
+        \begin{itemsize}
+          \item inner Source IP address
+          \item inner Destination IP address
+        \end{itemsize}
+  \item Else the device does not calculate the hash
+\end{itemize}
+
+The device calculates the hash on the inner IPv6 packet without an extension header
+of an encapsulated packet according to 'Enabled hash types' bitmask as follows:
+\begin{itemize}
+  \item If VIRTIO_NET_HASH_TYPE_TCPv6 is set and the encapsulated packet has an inner
+	TCPv6 header, the hash is calculated over the following fields:
+        \begin{itemsize}
+          \item inner Source IPv6 address
+          \item inner Destination IPv6 address
+          \item inner Source TCP port
+          \item inner Destination TCP port
+        \end{itemsize}
+  \item Else if VIRTIO_NET_HASH_TYPE_UDPv6 is set and the encapsulated packet has an
+	inner UDPv6 header, the hash is calculated over the following fields:
+        \begin{itemsize}
+          \item inner Source IPv6 address
+          \item inner Destination IPv6 address
+          \item inner Source UDP port
+          \item inner Destination UDP port
+        \end{itemize}
+  \item Else if VIRTIO_NET_HASH_TYPE_IPv6 is set, the hash is calculated over the
+	following fields:
+        \begin{itemsize}
+          \item inner Source IPv6 address
+          \item inner Destination IPv6 address
+        \end{itemsize}
+  \item Else the device does not calculate the hash
+\end{itemize}
+
+The device calculates the hash on the inner IPv6 packet with an extension header
+of an encapsulated packet according to 'Enabled hash types' bitmask as follows:
+\begin{itemsize}
+  \item If VIRTIO_NET_HASH_TYPE_TCP_EX is set and the encapsulated packet has an inner
+	TCPv6 header, the hash is calculated over the following fields:
+        \begin{itemize}
+          \item Home address from the home address option in the inner IPv6 destination
+              options header. If the inner extension header is not present, use the
+              inner Source IPv6 address.
+          \item IPv6 address that is contained in the Routing-Header-Type-2 from the
+              associated inner extension header. If the inner extension header is not
+              present, use the inner Destination IPv6 address.
+          \item inner Source TCP port
+          \item inner Destination TCP port
+        \end{itemize}
+  \item Else if VIRTIO_NET_HASH_TYPE_UDP_EX is set and the encapsulated packet has an inner
+	UDPv6 header, the hash is calculated over the following fields:
+        \begin{itemsize}
+          \item Home address from the home address option in the inner IPv6 destination
+              options header. If the inner extension header is not present, use the
+              inner Source IPv6 address.
+          \item IPv6 address that is contained in the Routing-Header-Type-2 from the
+              associated inner extension header. If the inner extension header is not
+              present, use the inner Destination IPv6 address.
+          \item inner Source UDP port
+          \item inner Destination UDP port
+        \end{itemize}
+  \item Else if VIRTIO_NET_HASH_TYPE_IP_EX is set, the hash is calculated over the
+	following fields:
+        \begin{itemsize}
+          \item Home address from the home address option in the inner IPv6 destination
+              options header. If the inner extension header is not present, use the
+              inner Source IPv6 address.
+          \item IPv6 address that is contained in the Routing-Header-Type-2 from the
+              associated inner extension header. If the inner extension header is not
+              present, use the inner Destination IPv6 address.
+        \end{itemize}
+  \item Else skip the inner IPv6 extension header and calculate the inner header hash as
+	defined for an encapsulated packet whose inner payload is an IPv6 packet without
+	an extension header.
+\end{itemsize}
+
 \paragraph{Hash reporting for incoming packets}
 \label{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash reporting for incoming packets}
 
-If VIRTIO_NET_F_HASH_REPORT was negotiated and
- the device has calculated the hash for the packet, the device fills \field{hash_report} with the report type of calculated hash
-and \field{hash_value} with the value of calculated hash.
+If VIRTIO_NET_F_HASH_REPORT was negotiated and the device has calculated the
+hash for the packet, the device fills \field{hash_report} with the report type
+of calculated hash, and \field{hash_value} with the value of calculated hash.
+Also, if VIRTIO_NET_F_HASH_TUNNEL was negotiated, the device needs to fill
+\field{hash_report_tunnel} with the report type of the encapsulated packet, and
+it is set to VIRTIO_NET_HASH_REPORT_TUNNEL_NONE for the unencapsulated packet.
 
 If VIRTIO_NET_F_HASH_REPORT was negotiated but due to any reason the
-hash was not calculated, the device sets \field{hash_report} to VIRTIO_NET_HASH_REPORT_NONE.
+hash was not calculated, the device sets \field{hash_report} to VIRTIO_NET_HASH_REPORT_NONE,
+and sets \field{hash_report_tunnel} to VIRTIO_NET_HASH_REPORT_TUNNEL_NONE.
 
 Possible values that the device can report in \field{hash_report} are defined below.
 They correspond to supported hash types defined in

content.tex \subsubsection{Processing of Incoming Packets}\label{sec:Device Types / Network · line 4171

Quoted 6 lines
content.tex +3 -0
@@ -4005,6 +4166,22 @@ \subsubsection{Processing of Incoming Packets}\label{sec:Device Types / Network
 #define VIRTIO_NET_HASH_REPORT_UDPv6_EX        9
 \end{lstlisting}
 
+If \field{hash_report} differs from VIRTIO_NET_HASH_REPORT_NONE,
+\field{hash_report_tunnel} can report the type of the encapsulated
+packet to the driver over the inner header hash calculation.

this sentence is a bit confused btw. so this reports the type of the encapsulated packet. but what does "over the inner header hash calculation." mean? what happens over what?

Quoted 48 lines
content.tex +13 -0
+Possible values that the device can report in \field{hash_report_tunnel}
+are defined below:
+
+\begin{lstlisting}
+#define VIRTIO_NET_HASH_REPORT_TUNNEL_NONE     0
+#define VIRTIO_NET_HASH_REPORT_GRE             1
+#define VIRTIO_NET_HASH_REPORT_VXLAN           2
+#define VIRTIO_NET_HASH_REPORT_GENEVE          3
+\end{lstlisting}
+
+They correspond to supported hash types defined in
+\ref{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / Supported/enabled hash types}.
+
 \subsubsection{Control Virtqueue}\label{sec:Device Types / Network Device / Device Operation / Control Virtqueue}
 
 The driver uses the control virtqueue (if VIRTIO_NET_F_CTRL_VQ is
introduction.tex +19 -0
index 287c5fc..ff01a9b 100644
@@ -98,6 +98,25 @@ \section{Normative References}\label{sec:Normative References}
 	\phantomsection\label{intro:SEC1}\textbf{[SEC1]} &
     Standards for Efficient Cryptography Group(SECG), ``SEC1: Elliptic Cureve Cryptography'', Version 1.0, September 2000.
 	\newline\url{https://www.secg.org/sec1-v2.pdf}\\
+	\phantomsection\label{intro:GRE}\textbf{[GRE]} &
+	Generic Routing Encapsulation
+	\newline\url{https://datatracker.ietf.org/doc/rfc2784/}\\
+	\phantomsection\label{intro:VXLAN}\textbf{[VXLAN]} &
+	Virtual eXtensible Local Area Network
+	\newline\url{https://datatracker.ietf.org/doc/rfc7348/}\\
+	\phantomsection\label{intro:GENEVE}\textbf{[GENEVE]} &
+	Generic Network Virtualization Encapsulation
+	\newline\url{https://datatracker.ietf.org/doc/rfc8926/}\\
+	\phantomsection\label{intro:IP}\textbf{[IP]} &
+	INTERNET PROTOCOL
+	\newline\url{https://www.rfc-editor.org/rfc/rfc791}\\
+	\phantomsection\label{intro:UDP}\textbf{[UDP]} &
+	User Datagram Protocol
+	\newline\url{https://www.rfc-editor.org/rfc/rfc768}\\
+	\phantomsection\label{intro:TCP}\textbf{[TCP]} &
+	TRANSMISSION CONTROL PROTOCOL
+	\newline\url{https://www.rfc-editor.org/rfc/rfc793}\\
+
 
 \end{longtable}
 
Signature · 1 line
2.19.1.6.gb485710b
author
#17 lore raw
Quoted 295 lines · Michael S. Tsirkin
Quoted 294 lines · Heng Qi

If the tunnel is used to encapsulate the packets, the hash calculated using the outer header of the receive packets is always fixed for the same flow packets, i.e. they will be steered to the same receive queue.

We add a feature bit VIRTIO_NET_F_HASH_TUNNEL and related bitmasks in \field{hash_types}, which instructs the device to calculate the hash using the inner headers of tunnel-encapsulated packets. Besides, values in \field{hash_report_tunnel} are added to report tunnel types.

---
v6:
1. Modify the wording of some sentences for clarity. @Michael S. Tsirkin
2. Fix some syntax issues. @Michael S. Tsirkin

v5:
1. Fix some syntax and capitalization issues. @Michael S. Tsirkin
2. Use encapsulated/encaptulation uniformly. @Michael S. Tsirkin
3. Move the links to introduction section. @Michael S. Tsirkin
4. Clarify some sentences. @Michael S. Tsirkin

v4:
1. Clarify some paragraphs. @Cornelia Huck
2. Fix the u8 type. @Cornelia Huck

v3:
1. Rename VIRTIO_NET_F_HASH_GRE_VXLAN_GENEVE_INNER to VIRTIO_NET_F_HASH_TUNNEL. @Jason Wang
2. Make things clearer. @Jason Wang @Michael S. Tsirkin
3. Keep the possibility to use inner hash for automatic receive steering. @Jason Wang
4. Add the "Tunnel packet" paragraph to avoid repeating the GRE etc. many times. @Michael S. Tsirkin

v2:
1. Add a feature bit for GRE/VXLAN/GENEVE inner hash. @Jason Wang
2. Chang \field{hash_tunnel} to \field{hash_report_tunnel}. @Jason Wang, @Michael S. Tsirkin

v1:
1. Remove the patch for the bitmask fix. @Michael S. Tsirkin
2. Clarify some paragraphs. @Jason Wang
3. Add \field{hash_tunnel} and VIRTIO_NET_HASH_REPORT_GRE. @Yuri Benditovich

  content.tex      | 191 +++++++++++++++++++++++++++++++++++++++++++++--
  introduction.tex |  19 +++++
  2 files changed, 203 insertions(+), 7 deletions(-)
content.tex +168 -7
index e863709..7845f6c 100644
@@ -3084,6 +3084,9 @@ \subsection{Feature bits}\label{sec:Device Types / Network Device / Feature bits
  \item[VIRTIO_NET_F_CTRL_MAC_ADDR(23)] Set MAC address through control
      channel.
 
+\item[VIRTIO_NET_F_HASH_TUNNEL(52)] Device supports inner
+    header hash for GRE, VXLAN and GENEVE tunnel-encapsulated packets.
+
  \item[VIRTIO_NET_F_NOTF_COAL(53)] Device supports notifications coalescing.
 
  \item[VIRTIO_NET_F_GUEST_USO4 (54)] Driver can receive USOv4 packets.
@@ -3095,7 +3098,8 @@ \subsection{Feature bits}\label{sec:Device Types / Network Device / Feature bits
   to several segments when each of these smaller packets has UDP header.
 
  \item[VIRTIO_NET_F_HASH_REPORT(57)] Device can report per-packet hash
-    value and a type of calculated hash.
+    value, a type of calculated hash, and, if VIRTIO_NET_F_HASH_TUNNEL
+    is negotiated, an encapsulation packet type.
 
  \item[VIRTIO_NET_F_GUEST_HDRLEN(59)] Driver can provide the exact \field{hdr_len}
      value. Device benefits from knowing the exact header length.
@@ -3140,6 +3144,7 @@ \subsubsection{Feature bit requirements}\label{sec:Device Types / Network Device
  \item[VIRTIO_NET_F_NOTF_COAL] Requires VIRTIO_NET_F_CTRL_VQ.
  \item[VIRTIO_NET_F_RSC_EXT] Requires VIRTIO_NET_F_HOST_TSO4 or VIRTIO_NET_F_HOST_TSO6.
  \item[VIRTIO_NET_F_RSS] Requires VIRTIO_NET_F_CTRL_VQ.
+\item[VIRTIO_NET_F_HASH_TUNNEL] Requires VIRTIO_NET_F_CTRL_VQ.
  \end{description}
 
  \subsubsection{Legacy Interface: Feature bits}\label{sec:Device Types / Network Device / Feature bits / Legacy Interface: Feature bits}
@@ -3386,7 +3391,8 @@ \subsection{Device Operation}\label{sec:Device Types / Network Device / Device O
          le16 num_buffers;
          le32 hash_value;        (Only if VIRTIO_NET_F_HASH_REPORT negotiated)
          le16 hash_report;       (Only if VIRTIO_NET_F_HASH_REPORT negotiated)
-        le16 padding_reserved;  (Only if VIRTIO_NET_F_HASH_REPORT negotiated)
+        u8 hash_report_tunnel;  (Only if VIRTIO_NET_F_HASH_REPORT negotiated, only valid of VIRTIO_NET_F_HASH_TUNNEL negotiated)
+        u8 padding_reserved;    (Only if VIRTIO_NET_F_HASH_REPORT negotiated)
  };
  \end{lstlisting}
 
@@ -3837,7 +3843,8 @@ \subsubsection{Processing of Incoming Packets}\label{sec:Device Types / Network
  A device attempts to calculate a per-packet hash in the following cases:
  \begin{itemize}
  \item The feature VIRTIO_NET_F_RSS was negotiated. The device uses the hash to determine the receive virtqueue to place incoming packets.
-\item The feature VIRTIO_NET_F_HASH_REPORT was negotiated. The device reports the hash value and the hash type with the packet.
+\item The feature VIRTIO_NET_F_HASH_REPORT was negotiated. The device reports the hash value and the hash type. If additionally
+VIRTIO_NET_F_HASH_TUNNEL was negotiated, the device reports the encapsulation type as well.
  \end{itemize}
 
  If the feature VIRTIO_NET_F_RSS was negotiated:
@@ -3863,8 +3870,36 @@ \subsubsection{Processing of Incoming Packets}\label{sec:Device Types / Network
   \ref{sec:Device Types / Network Device / Device Operation / Control Virtqueue / Automatic receive steering in multiqueue mode / Hash calculation}.
  \end{itemize}
 
+\subparagraph{Tunnel/Encapsulated packet}
+\label{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / Tunnel/Encapsulated packet}
+A tunnel packet is encapsulated from the original packet based on the tunneling
+protocol (only a single level of encapsulation is currently supported). The
+encapsulated packet contains an outer header and an inner header, and the device
+calculates the hash over either the inner header or the outer header.
+
+When the feature VIRTIO_NET_F_HASH_TUNNEL is negotiated and the corresponding
+encapsulation type is set in \field{hash_types}, the hash for a specific type of
+encapsulated packet is calculated over the inner as opposed to outer header.
+Supported encapsulation types are listed in \ref{sec:Device Types / Network Device /
+Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets /
+Supported/enabled hash types}.
+
+If both VIRTIO_NET_F_HASH_REPORT and VIRTIO_NET_F_HASH_TUNNEL are negotiated,
+the device can support inner hash calculation for \hyperref[intro:GRE]{[GRE]},
+\hyperref[intro:VXLAN]{[VXLAN]} and \hyperref[intro:GENEVE]{[GENEVE]}
+encapsulated packets, and the corresponding encapsulation type in \field{hash_types}
+is VIRTIO_NET_HASH_TYPE_{GRE, VXLAN, GENEVE}_INNER respectively. The value in
+\field{hash_report_tunnel} is VIRTIO_NET_HASH_REPORT_{NONE, GRE, VXLAN, GENEVE} respectively.
+
+If VIRTIO_NET_F_HASH_REPORT is negotiated but VIRTIO_NET_F_HASH_TUNNEL is not
+negotiated, the device calculates the hash over the outer header, and
+\field{hash_report} reports the hash type. \field{hash_report_tunnel}
+is no longer valid.
+
  \subparagraph{Supported/enabled hash types}
  \label{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / Supported/enabled hash types}
+This paragraph relies on definitions from \hyperref[intro:IP]{[IP]},
+\hyperref[intro:UDP]{[UDP]} and \hyperref[intro:TCP]{[TCP]}.
  Hash types applicable for IPv4 packets:
  \begin{lstlisting}
  #define VIRTIO_NET_HASH_TYPE_IPv4              (1 << 0)
@@ -3884,6 +3919,22 @@ \subsubsection{Processing of Incoming Packets}\label{sec:Device Types / Network
  #define VIRTIO_NET_HASH_TYPE_UDP_EX            (1 << 8)
  \end{lstlisting}
 
+If the feature VIRTIO_NET_F_HASH_TUNNEL is negotiated, the encapsulation
+hash type below indicates that the hash is calculated over the inner
+header of the encapsulated packet:
+Hash type applicable for inner payload of the gre-encapsulated packet
+\begin{lstlisting}
+#define VIRTIO_NET_HASH_TYPE_GRE_INNER         (1 << 9)
+\end{lstlisting}
+Hash type applicable for inner payload of the vxlan-encapsulated packet
+\begin{lstlisting}
+#define VIRTIO_NET_HASH_TYPE_VXLAN_INNER       (1 << 10)
+\end{lstlisting}
+Hash type applicable for inner payload of the geneve-encapsulated packet
+\begin{lstlisting}
+#define VIRTIO_NET_HASH_TYPE_GENEVE_INNER      (1 << 11)
+\end{lstlisting}
+
  \subparagraph{IPv4 packets}
  \label{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / IPv4 packets}
  The device calculates the hash on IPv4 packets according to 'Enabled hash types' bitmask as follows:
@@ -3975,15 +4026,125 @@ \subsubsection{Processing of Incoming Packets}\label{sec:Device Types / Network
  (see \ref{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / IPv6 packets without extension header}).
  \end{itemize}
 
+\subparagraph{Inner payload of an encapsulated packet}
+\label{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / Inner payload of the encapsulated packet}
+If the feature VIRTIO_NET_F_HASH_TUNNEL is negotiated and the corresponding
+encapsulation hash type is set in \field{hash_types}, the device calculates the
+inner header hash on an encapsulated packet (See \ref{sec:Device Types
+/ Network Device / Device Operation / Processing of Incoming Packets /
+Hash calculation for incoming packets / Tunnel/Encapsulated packet}) as follows:
+
+The device calculates the hash on the inner IPv4 packet of an encapsulated packet
+according to 'Enabled hash types' bitmask as follows:
+\begin{itemize}
+  \item If VIRTIO_NET_HASH_TYPE_TCPv4 is set and the encapsulated packet has an inner
+	TCPv4 header, the hash is calculated over the following fields:
+        \begin{itemsize}
+          \item inner Source IP address
+          \item inner Destination IP address
+          \item inner Source TCP port
+          \item inner Destination TCP port
+        \end{itemsize}
+  \item Else if VIRTIO_NET_HASH_TYPE_UDPv4 is set and the encapsulated packet has an
+	inner UDPv4 header, the hash is calculated over the following fields:
+        \begin{itemsize}
+          \item inner Source IP address
+          \item inner Destination IP address
+          \item inner Source UDP port
+          \item inner Destination UDP port
+        \end{itemize}
+  \item Else if VIRTIO_NET_HASH_TYPE_IPv4 is set, the hash is calculated over the
+	following fields:
+        \begin{itemsize}
+          \item inner Source IP address
+          \item inner Destination IP address
+        \end{itemsize}
+  \item Else the device does not calculate the hash
+\end{itemize}
+
+The device calculates the hash on the inner IPv6 packet without an extension header
+of an encapsulated packet according to 'Enabled hash types' bitmask as follows:
+\begin{itemize}
+  \item If VIRTIO_NET_HASH_TYPE_TCPv6 is set and the encapsulated packet has an inner
+	TCPv6 header, the hash is calculated over the following fields:
+        \begin{itemsize}
+          \item inner Source IPv6 address
+          \item inner Destination IPv6 address
+          \item inner Source TCP port
+          \item inner Destination TCP port
+        \end{itemsize}
+  \item Else if VIRTIO_NET_HASH_TYPE_UDPv6 is set and the encapsulated packet has an
+	inner UDPv6 header, the hash is calculated over the following fields:
+        \begin{itemsize}
+          \item inner Source IPv6 address
+          \item inner Destination IPv6 address
+          \item inner Source UDP port
+          \item inner Destination UDP port
+        \end{itemize}
+  \item Else if VIRTIO_NET_HASH_TYPE_IPv6 is set, the hash is calculated over the
+	following fields:
+        \begin{itemsize}
+          \item inner Source IPv6 address
+          \item inner Destination IPv6 address
+        \end{itemsize}
+  \item Else the device does not calculate the hash
+\end{itemize}
+
+The device calculates the hash on the inner IPv6 packet with an extension header
+of an encapsulated packet according to 'Enabled hash types' bitmask as follows:
+\begin{itemsize}
+  \item If VIRTIO_NET_HASH_TYPE_TCP_EX is set and the encapsulated packet has an inner
+	TCPv6 header, the hash is calculated over the following fields:
+        \begin{itemize}
+          \item Home address from the home address option in the inner IPv6 destination
+              options header. If the inner extension header is not present, use the
+              inner Source IPv6 address.
+          \item IPv6 address that is contained in the Routing-Header-Type-2 from the
+              associated inner extension header. If the inner extension header is not
+              present, use the inner Destination IPv6 address.
+          \item inner Source TCP port
+          \item inner Destination TCP port
+        \end{itemize}
+  \item Else if VIRTIO_NET_HASH_TYPE_UDP_EX is set and the encapsulated packet has an inner
+	UDPv6 header, the hash is calculated over the following fields:
+        \begin{itemsize}
+          \item Home address from the home address option in the inner IPv6 destination
+              options header. If the inner extension header is not present, use the
+              inner Source IPv6 address.
+          \item IPv6 address that is contained in the Routing-Header-Type-2 from the
+              associated inner extension header. If the inner extension header is not
+              present, use the inner Destination IPv6 address.
+          \item inner Source UDP port
+          \item inner Destination UDP port
+        \end{itemize}
+  \item Else if VIRTIO_NET_HASH_TYPE_IP_EX is set, the hash is calculated over the
+	following fields:
+        \begin{itemsize}
+          \item Home address from the home address option in the inner IPv6 destination
+              options header. If the inner extension header is not present, use the
+              inner Source IPv6 address.
+          \item IPv6 address that is contained in the Routing-Header-Type-2 from the
+              associated inner extension header. If the inner extension header is not
+              present, use the inner Destination IPv6 address.
+        \end{itemize}
+  \item Else skip the inner IPv6 extension header and calculate the inner header hash as
+	defined for an encapsulated packet whose inner payload is an IPv6 packet without
+	an extension header.
+\end{itemsize}
+
  \paragraph{Hash reporting for incoming packets}
  \label{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash reporting for incoming packets}
 
-If VIRTIO_NET_F_HASH_REPORT was negotiated and
- the device has calculated the hash for the packet, the device fills \field{hash_report} with the report type of calculated hash
-and \field{hash_value} with the value of calculated hash.
+If VIRTIO_NET_F_HASH_REPORT was negotiated and the device has calculated the
+hash for the packet, the device fills \field{hash_report} with the report type
+of calculated hash, and \field{hash_value} with the value of calculated hash.
+Also, if VIRTIO_NET_F_HASH_TUNNEL was negotiated, the device needs to fill
+\field{hash_report_tunnel} with the report type of the encapsulated packet, and
+it is set to VIRTIO_NET_HASH_REPORT_TUNNEL_NONE for the unencapsulated packet.
 
  If VIRTIO_NET_F_HASH_REPORT was negotiated but due to any reason the
-hash was not calculated, the device sets \field{hash_report} to VIRTIO_NET_HASH_REPORT_NONE.
+hash was not calculated, the device sets \field{hash_report} to VIRTIO_NET_HASH_REPORT_NONE,
+and sets \field{hash_report_tunnel} to VIRTIO_NET_HASH_REPORT_TUNNEL_NONE.
 
  Possible values that the device can report in \field{hash_report} are defined below.
  They correspond to supported hash types defined in
@@ -4005,6 +4166,22 @@ \subsubsection{Processing of Incoming Packets}\label{sec:Device Types / Network
  #define VIRTIO_NET_HASH_REPORT_UDPv6_EX        9
  \end{lstlisting}
 
Quoted 7 lines

content.tex \subsubsection{Processing of Incoming Packets}\label{sec:Device Types / Network · line 4171

Quoted 3 lines
content.tex +3 -0
+If \field{hash_report} differs from VIRTIO_NET_HASH_REPORT_NONE,
+\field{hash_report_tunnel} can report the type of the encapsulated
+packet to the driver over the inner header hash calculation.

this sentence is a bit confused btw. so this reports the type of the encapsulated packet. but what does "over the inner header hash calculation." mean? what happens over what?

Just means "After the hash is computed, the encapsulation type can be reported to the driver". It means redundant here, I will remove "over the inner header hash calculation".

Thanks.

Quoted 50 lines
Quoted 50 lines
content.tex +13 -0
+Possible values that the device can report in \field{hash_report_tunnel}
+are defined below:
+
+\begin{lstlisting}
+#define VIRTIO_NET_HASH_REPORT_TUNNEL_NONE     0
+#define VIRTIO_NET_HASH_REPORT_GRE             1
+#define VIRTIO_NET_HASH_REPORT_VXLAN           2
+#define VIRTIO_NET_HASH_REPORT_GENEVE          3
+\end{lstlisting}
+
+They correspond to supported hash types defined in
+\ref{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / Supported/enabled hash types}.
+
  \subsubsection{Control Virtqueue}\label{sec:Device Types / Network Device / Device Operation / Control Virtqueue}
 
  The driver uses the control virtqueue (if VIRTIO_NET_F_CTRL_VQ is
introduction.tex +19 -0
index 287c5fc..ff01a9b 100644
@@ -98,6 +98,25 @@ \section{Normative References}\label{sec:Normative References}
  	\phantomsection\label{intro:SEC1}\textbf{[SEC1]} &
      Standards for Efficient Cryptography Group(SECG), ``SEC1: Elliptic Cureve Cryptography'', Version 1.0, September 2000.
  	\newline\url{https://www.secg.org/sec1-v2.pdf}\\
+	\phantomsection\label{intro:GRE}\textbf{[GRE]} &
+	Generic Routing Encapsulation
+	\newline\url{https://datatracker.ietf.org/doc/rfc2784/}\\
+	\phantomsection\label{intro:VXLAN}\textbf{[VXLAN]} &
+	Virtual eXtensible Local Area Network
+	\newline\url{https://datatracker.ietf.org/doc/rfc7348/}\\
+	\phantomsection\label{intro:GENEVE}\textbf{[GENEVE]} &
+	Generic Network Virtualization Encapsulation
+	\newline\url{https://datatracker.ietf.org/doc/rfc8926/}\\
+	\phantomsection\label{intro:IP}\textbf{[IP]} &
+	INTERNET PROTOCOL
+	\newline\url{https://www.rfc-editor.org/rfc/rfc791}\\
+	\phantomsection\label{intro:UDP}\textbf{[UDP]} &
+	User Datagram Protocol
+	\newline\url{https://www.rfc-editor.org/rfc/rfc768}\\
+	\phantomsection\label{intro:TCP}\textbf{[TCP]} &
+	TRANSMISSION CONTROL PROTOCOL
+	\newline\url{https://www.rfc-editor.org/rfc/rfc793}\\
+
 
  \end{longtable}
 
Signature · 1 line
2.19.1.6.gb485710b
#18 lore raw
In reply to Heng Qi

content.tex \subsubsection{Processing of Incoming Packets}\label{sec:Device Types / Network · line 4180

Quoted 8 lines · Heng Qi
content.tex +8 -0
+are defined below:
+
+\begin{lstlisting}
+#define VIRTIO_NET_HASH_REPORT_TUNNEL_NONE     0
+#define VIRTIO_NET_HASH_REPORT_GRE             1
+#define VIRTIO_NET_HASH_REPORT_VXLAN           2
+#define VIRTIO_NET_HASH_REPORT_GENEVE          3
+\end{lstlisting}

Btw this "are defined below" all over the place is just contributing to making the spec unnecesarily verbose. Simple "are:" will do.

Signature · 1 line
MST
author
#19 lore raw
Quoted 2 lines · Michael S. Tsirkin

Btw this "are defined below" all over the place is just contributing to making the spec unnecesarily verbose. Simple "are:" will do.

Sure. I'll fix it in the next version.

Thanks.

#20 lore raw
In reply to Heng Qi
Quoted 81 lines · Heng Qi

If the tunnel is used to encapsulate the packets, the hash calculated using the outer header of the receive packets is always fixed for the same flow packets, i.e. they will be steered to the same receive queue.

We add a feature bit VIRTIO_NET_F_HASH_TUNNEL and related bitmasks in \field{hash_types}, which instructs the device to calculate the hash using the inner headers of tunnel-encapsulated packets. Besides, values in \field{hash_report_tunnel} are added to report tunnel types.

---
v6:
1. Modify the wording of some sentences for clarity. @Michael S. Tsirkin
2. Fix some syntax issues. @Michael S. Tsirkin

v5:
1. Fix some syntax and capitalization issues. @Michael S. Tsirkin
2. Use encapsulated/encaptulation uniformly. @Michael S. Tsirkin
3. Move the links to introduction section. @Michael S. Tsirkin
4. Clarify some sentences. @Michael S. Tsirkin

v4:
1. Clarify some paragraphs. @Cornelia Huck
2. Fix the u8 type. @Cornelia Huck

v3:
1. Rename VIRTIO_NET_F_HASH_GRE_VXLAN_GENEVE_INNER to VIRTIO_NET_F_HASH_TUNNEL. @Jason Wang
2. Make things clearer. @Jason Wang @Michael S. Tsirkin
3. Keep the possibility to use inner hash for automatic receive steering. @Jason Wang
4. Add the "Tunnel packet" paragraph to avoid repeating the GRE etc. many times. @Michael S. Tsirkin

v2:
1. Add a feature bit for GRE/VXLAN/GENEVE inner hash. @Jason Wang
2. Chang \field{hash_tunnel} to \field{hash_report_tunnel}. @Jason Wang, @Michael S. Tsirkin

v1:
1. Remove the patch for the bitmask fix. @Michael S. Tsirkin
2. Clarify some paragraphs. @Jason Wang
3. Add \field{hash_tunnel} and VIRTIO_NET_HASH_REPORT_GRE. @Yuri Benditovich

 content.tex      | 191 +++++++++++++++++++++++++++++++++++++++++++++--
 introduction.tex |  19 +++++
 2 files changed, 203 insertions(+), 7 deletions(-)
content.tex +6 -1
index e863709..7845f6c 100644
@@ -3084,6 +3084,9 @@ \subsection{Feature bits}\label{sec:Device Types / Network Device / Feature bits
 \item[VIRTIO_NET_F_CTRL_MAC_ADDR(23)] Set MAC address through control
     channel.
 
+\item[VIRTIO_NET_F_HASH_TUNNEL(52)] Device supports inner
+    header hash for GRE, VXLAN and GENEVE tunnel-encapsulated packets.
+
 \item[VIRTIO_NET_F_NOTF_COAL(53)] Device supports notifications coalescing.
 
 \item[VIRTIO_NET_F_GUEST_USO4 (54)] Driver can receive USOv4 packets.
@@ -3095,7 +3098,8 @@ \subsection{Feature bits}\label{sec:Device Types / Network Device / Feature bits
  to several segments when each of these smaller packets has UDP header.
 
 \item[VIRTIO_NET_F_HASH_REPORT(57)] Device can report per-packet hash
-    value and a type of calculated hash.
+    value, a type of calculated hash, and, if VIRTIO_NET_F_HASH_TUNNEL
+    is negotiated, an encapsulation packet type.
 
 \item[VIRTIO_NET_F_GUEST_HDRLEN(59)] Driver can provide the exact \field{hdr_len}
     value. Device benefits from knowing the exact header length.
@@ -3140,6 +3144,7 @@ \subsubsection{Feature bit requirements}\label{sec:Device Types / Network Device
 \item[VIRTIO_NET_F_NOTF_COAL] Requires VIRTIO_NET_F_CTRL_VQ.
 \item[VIRTIO_NET_F_RSC_EXT] Requires VIRTIO_NET_F_HOST_TSO4 or VIRTIO_NET_F_HOST_TSO6.
 \item[VIRTIO_NET_F_RSS] Requires VIRTIO_NET_F_CTRL_VQ.
+\item[VIRTIO_NET_F_HASH_TUNNEL] Requires VIRTIO_NET_F_CTRL_VQ.
 \end{description}
 
 \subsubsection{Legacy Interface: Feature bits}\label{sec:Device Types / Network Device / Feature bits / Legacy Interface: Feature bits}

content.tex \subsection{Device Operation}\label{sec:Device Types / Network Device / Device O · line 3394

Quoted 5 lines
content.tex +1 -1
@@ -3386,7 +3391,8 @@ \subsection{Device Operation}\label{sec:Device Types / Network Device / Device O
         le16 num_buffers;
         le32 hash_value;        (Only if VIRTIO_NET_F_HASH_REPORT negotiated)
         le16 hash_report;       (Only if VIRTIO_NET_F_HASH_REPORT negotiated)
-        le16 padding_reserved;  (Only if VIRTIO_NET_F_HASH_REPORT negotiated)
+        u8 hash_report_tunnel;  (Only if VIRTIO_NET_F_HASH_REPORT negotiated, only valid of VIRTIO_NET_F_HASH_TUNNEL negotiated)

of -> if?

also let's add: otherwise reserved?

Quoted 35 lines
content.tex +18 -1
+        u8 padding_reserved;    (Only if VIRTIO_NET_F_HASH_REPORT negotiated)
 };
 \end{lstlisting}
 
@@ -3837,7 +3843,8 @@ \subsubsection{Processing of Incoming Packets}\label{sec:Device Types / Network
 A device attempts to calculate a per-packet hash in the following cases:
 \begin{itemize}
 \item The feature VIRTIO_NET_F_RSS was negotiated. The device uses the hash to determine the receive virtqueue to place incoming packets.
-\item The feature VIRTIO_NET_F_HASH_REPORT was negotiated. The device reports the hash value and the hash type with the packet.
+\item The feature VIRTIO_NET_F_HASH_REPORT was negotiated. The device reports the hash value and the hash type. If additionally
+VIRTIO_NET_F_HASH_TUNNEL was negotiated, the device reports the encapsulation type as well.
 \end{itemize}
 
 If the feature VIRTIO_NET_F_RSS was negotiated:
@@ -3863,8 +3870,36 @@ \subsubsection{Processing of Incoming Packets}\label{sec:Device Types / Network
  \ref{sec:Device Types / Network Device / Device Operation / Control Virtqueue / Automatic receive steering in multiqueue mode / Hash calculation}.
 \end{itemize}
 
+\subparagraph{Tunnel/Encapsulated packet}
+\label{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / Tunnel/Encapsulated packet}
+A tunnel packet is encapsulated from the original packet based on the tunneling
+protocol (only a single level of encapsulation is currently supported). The
+encapsulated packet contains an outer header and an inner header, and the device
+calculates the hash over either the inner header or the outer header.
+
+When the feature VIRTIO_NET_F_HASH_TUNNEL is negotiated and the corresponding
+encapsulation type is set in \field{hash_types}, the hash for a specific type of
+encapsulated packet is calculated over the inner as opposed to outer header.
 
 
+Supported encapsulation types are listed in \ref{sec:Device Types / Network Device /
+Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets /
+Supported/enabled hash types}.
+
+If both VIRTIO_NET_F_HASH_REPORT and VIRTIO_NET_F_HASH_TUNNEL are negotiated,

content.tex \subsubsection{Processing of Incoming Packets}\label{sec:Device Types / Network · line 3899

Quoted 10 lines
content.tex +10 -0
+the device can support inner hash calculation for \hyperref[intro:GRE]{[GRE]},
+\hyperref[intro:VXLAN]{[VXLAN]} and \hyperref[intro:GENEVE]{[GENEVE]}
+encapsulated packets, and the corresponding encapsulation type in \field{hash_types}
+is VIRTIO_NET_HASH_TYPE_{GRE, VXLAN, GENEVE}_INNER respectively. The value in
+\field{hash_report_tunnel} is VIRTIO_NET_HASH_REPORT_{NONE, GRE, VXLAN, GENEVE} respectively.
+
+If VIRTIO_NET_F_HASH_REPORT is negotiated but VIRTIO_NET_F_HASH_TUNNEL is not
+negotiated, the device calculates the hash over the outer header, and
+\field{hash_report} reports the hash type. \field{hash_report_tunnel}
+is no longer valid.

Does this mean that VIRTIO_NET_HASH_TYPE_{GRE, VXLAN, GENEVE}_INNER must not be set without VIRTIO_NET_F_HASH_TUNNEL?

If yes then we can add conformance statements to this end and then drop the
"If the feature VIRTIO_NET_F_HASH_TUNNEL is negotiated" all over the place just relying on hash type being set instead.

Also, note that it has to be ok for device to expose hash types without negotiating VIRTIO_NET_F_HASH_TUNNEL since they are in config space.
Idea: do we want to decouple these completely? have VIRTIO_NET_F_HASH_TUNNEL just add hash_report_tunnel and have hash types speak for themselves? It makes things nicely orthogonal in that if one does not care about knowing encapsulation type (e.g. for RSS) one can disable VIRTIO_NET_F_HASH_TUNNEL.
Objections? Jason?

Quoted 216 lines
content.tex +3 -0
+
 \subparagraph{Supported/enabled hash types}
 \label{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / Supported/enabled hash types}
+This paragraph relies on definitions from \hyperref[intro:IP]{[IP]},
+\hyperref[intro:UDP]{[UDP]} and \hyperref[intro:TCP]{[TCP]}.
 Hash types applicable for IPv4 packets:

\begin{lstlisting}
#define VIRTIO_NET_HASH_TYPE_IPv4 (1 << 0)

content.tex +146 -4
@@ -3884,6 +3919,22 @@ \subsubsection{Processing of Incoming Packets}\label{sec:Device Types / Network
 #define VIRTIO_NET_HASH_TYPE_UDP_EX            (1 << 8)
 \end{lstlisting}
 
+If the feature VIRTIO_NET_F_HASH_TUNNEL is negotiated, the encapsulation
+hash type below indicates that the hash is calculated over the inner
+header of the encapsulated packet:
+Hash type applicable for inner payload of the gre-encapsulated packet
+\begin{lstlisting}
+#define VIRTIO_NET_HASH_TYPE_GRE_INNER         (1 << 9)
+\end{lstlisting}
+Hash type applicable for inner payload of the vxlan-encapsulated packet
+\begin{lstlisting}
+#define VIRTIO_NET_HASH_TYPE_VXLAN_INNER       (1 << 10)
+\end{lstlisting}
+Hash type applicable for inner payload of the geneve-encapsulated packet
+\begin{lstlisting}
+#define VIRTIO_NET_HASH_TYPE_GENEVE_INNER      (1 << 11)
+\end{lstlisting}
+
 \subparagraph{IPv4 packets}
 \label{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / IPv4 packets}
 The device calculates the hash on IPv4 packets according to 'Enabled hash types' bitmask as follows:
@@ -3975,15 +4026,125 @@ \subsubsection{Processing of Incoming Packets}\label{sec:Device Types / Network
 (see \ref{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / IPv6 packets without extension header}).
 \end{itemize}
 
+\subparagraph{Inner payload of an encapsulated packet}
+\label{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / Inner payload of the encapsulated packet}
+If the feature VIRTIO_NET_F_HASH_TUNNEL is negotiated and the corresponding
+encapsulation hash type is set in \field{hash_types}, the device calculates the
+inner header hash on an encapsulated packet (See \ref{sec:Device Types
+/ Network Device / Device Operation / Processing of Incoming Packets /
+Hash calculation for incoming packets / Tunnel/Encapsulated packet}) as follows:
+
+The device calculates the hash on the inner IPv4 packet of an encapsulated packet
+according to 'Enabled hash types' bitmask as follows:
+\begin{itemize}
+  \item If VIRTIO_NET_HASH_TYPE_TCPv4 is set and the encapsulated packet has an inner
+	TCPv4 header, the hash is calculated over the following fields:
+        \begin{itemsize}
+          \item inner Source IP address
+          \item inner Destination IP address
+          \item inner Source TCP port
+          \item inner Destination TCP port
+        \end{itemsize}
+  \item Else if VIRTIO_NET_HASH_TYPE_UDPv4 is set and the encapsulated packet has an
+	inner UDPv4 header, the hash is calculated over the following fields:
+        \begin{itemsize}
+          \item inner Source IP address
+          \item inner Destination IP address
+          \item inner Source UDP port
+          \item inner Destination UDP port
+        \end{itemize}
+  \item Else if VIRTIO_NET_HASH_TYPE_IPv4 is set, the hash is calculated over the
+	following fields:
+        \begin{itemsize}
+          \item inner Source IP address
+          \item inner Destination IP address
+        \end{itemsize}
+  \item Else the device does not calculate the hash
+\end{itemize}
+
+The device calculates the hash on the inner IPv6 packet without an extension header
+of an encapsulated packet according to 'Enabled hash types' bitmask as follows:
+\begin{itemize}
+  \item If VIRTIO_NET_HASH_TYPE_TCPv6 is set and the encapsulated packet has an inner
+	TCPv6 header, the hash is calculated over the following fields:
+        \begin{itemsize}
+          \item inner Source IPv6 address
+          \item inner Destination IPv6 address
+          \item inner Source TCP port
+          \item inner Destination TCP port
+        \end{itemsize}
+  \item Else if VIRTIO_NET_HASH_TYPE_UDPv6 is set and the encapsulated packet has an
+	inner UDPv6 header, the hash is calculated over the following fields:
+        \begin{itemsize}
+          \item inner Source IPv6 address
+          \item inner Destination IPv6 address
+          \item inner Source UDP port
+          \item inner Destination UDP port
+        \end{itemize}
+  \item Else if VIRTIO_NET_HASH_TYPE_IPv6 is set, the hash is calculated over the
+	following fields:
+        \begin{itemsize}
+          \item inner Source IPv6 address
+          \item inner Destination IPv6 address
+        \end{itemsize}
+  \item Else the device does not calculate the hash
+\end{itemize}
+
+The device calculates the hash on the inner IPv6 packet with an extension header
+of an encapsulated packet according to 'Enabled hash types' bitmask as follows:
+\begin{itemsize}
+  \item If VIRTIO_NET_HASH_TYPE_TCP_EX is set and the encapsulated packet has an inner
+	TCPv6 header, the hash is calculated over the following fields:
+        \begin{itemize}
+          \item Home address from the home address option in the inner IPv6 destination
+              options header. If the inner extension header is not present, use the
+              inner Source IPv6 address.
+          \item IPv6 address that is contained in the Routing-Header-Type-2 from the
+              associated inner extension header. If the inner extension header is not
+              present, use the inner Destination IPv6 address.
+          \item inner Source TCP port
+          \item inner Destination TCP port
+        \end{itemize}
+  \item Else if VIRTIO_NET_HASH_TYPE_UDP_EX is set and the encapsulated packet has an inner
+	UDPv6 header, the hash is calculated over the following fields:
+        \begin{itemsize}
+          \item Home address from the home address option in the inner IPv6 destination
+              options header. If the inner extension header is not present, use the
+              inner Source IPv6 address.
+          \item IPv6 address that is contained in the Routing-Header-Type-2 from the
+              associated inner extension header. If the inner extension header is not
+              present, use the inner Destination IPv6 address.
+          \item inner Source UDP port
+          \item inner Destination UDP port
+        \end{itemize}
+  \item Else if VIRTIO_NET_HASH_TYPE_IP_EX is set, the hash is calculated over the
+	following fields:
+        \begin{itemsize}
+          \item Home address from the home address option in the inner IPv6 destination
+              options header. If the inner extension header is not present, use the
+              inner Source IPv6 address.
+          \item IPv6 address that is contained in the Routing-Header-Type-2 from the
+              associated inner extension header. If the inner extension header is not
+              present, use the inner Destination IPv6 address.
+        \end{itemize}
+  \item Else skip the inner IPv6 extension header and calculate the inner header hash as
+	defined for an encapsulated packet whose inner payload is an IPv6 packet without
+	an extension header.
+\end{itemsize}
+
 \paragraph{Hash reporting for incoming packets}
 \label{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash reporting for incoming packets}
 
-If VIRTIO_NET_F_HASH_REPORT was negotiated and
- the device has calculated the hash for the packet, the device fills \field{hash_report} with the report type of calculated hash
-and \field{hash_value} with the value of calculated hash.
+If VIRTIO_NET_F_HASH_REPORT was negotiated and the device has calculated the
+hash for the packet, the device fills \field{hash_report} with the report type
+of calculated hash, and \field{hash_value} with the value of calculated hash.
+Also, if VIRTIO_NET_F_HASH_TUNNEL was negotiated, the device needs to fill
+\field{hash_report_tunnel} with the report type of the encapsulated packet, and
+it is set to VIRTIO_NET_HASH_REPORT_TUNNEL_NONE for the unencapsulated packet.
 
 If VIRTIO_NET_F_HASH_REPORT was negotiated but due to any reason the
-hash was not calculated, the device sets \field{hash_report} to VIRTIO_NET_HASH_REPORT_NONE.
+hash was not calculated, the device sets \field{hash_report} to VIRTIO_NET_HASH_REPORT_NONE,
+and sets \field{hash_report_tunnel} to VIRTIO_NET_HASH_REPORT_TUNNEL_NONE.
 
 Possible values that the device can report in \field{hash_report} are defined below.
 They correspond to supported hash types defined in
@@ -4005,6 +4166,22 @@ \subsubsection{Processing of Incoming Packets}\label{sec:Device Types / Network
 #define VIRTIO_NET_HASH_REPORT_UDPv6_EX        9
 \end{lstlisting}
 
+If \field{hash_report} differs from VIRTIO_NET_HASH_REPORT_NONE,
+\field{hash_report_tunnel} can report the type of the encapsulated
+packet to the driver over the inner header hash calculation.
+Possible values that the device can report in \field{hash_report_tunnel}
+are defined below:
+
+\begin{lstlisting}
+#define VIRTIO_NET_HASH_REPORT_TUNNEL_NONE     0
+#define VIRTIO_NET_HASH_REPORT_GRE             1
+#define VIRTIO_NET_HASH_REPORT_VXLAN           2
+#define VIRTIO_NET_HASH_REPORT_GENEVE          3
+\end{lstlisting}
+
+They correspond to supported hash types defined in
+\ref{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / Supported/enabled hash types}.
+
 \subsubsection{Control Virtqueue}\label{sec:Device Types / Network Device / Device Operation / Control Virtqueue}
 
 The driver uses the control virtqueue (if VIRTIO_NET_F_CTRL_VQ is
introduction.tex +19 -0
index 287c5fc..ff01a9b 100644
@@ -98,6 +98,25 @@ \section{Normative References}\label{sec:Normative References}
 	\phantomsection\label{intro:SEC1}\textbf{[SEC1]} &
     Standards for Efficient Cryptography Group(SECG), ``SEC1: Elliptic Cureve Cryptography'', Version 1.0, September 2000.
 	\newline\url{https://www.secg.org/sec1-v2.pdf}\\
+	\phantomsection\label{intro:GRE}\textbf{[GRE]} &
+	Generic Routing Encapsulation
+	\newline\url{https://datatracker.ietf.org/doc/rfc2784/}\\
+	\phantomsection\label{intro:VXLAN}\textbf{[VXLAN]} &
+	Virtual eXtensible Local Area Network
+	\newline\url{https://datatracker.ietf.org/doc/rfc7348/}\\
+	\phantomsection\label{intro:GENEVE}\textbf{[GENEVE]} &
+	Generic Network Virtualization Encapsulation
+	\newline\url{https://datatracker.ietf.org/doc/rfc8926/}\\
+	\phantomsection\label{intro:IP}\textbf{[IP]} &
+	INTERNET PROTOCOL
+	\newline\url{https://www.rfc-editor.org/rfc/rfc791}\\
+	\phantomsection\label{intro:UDP}\textbf{[UDP]} &
+	User Datagram Protocol
+	\newline\url{https://www.rfc-editor.org/rfc/rfc768}\\
+	\phantomsection\label{intro:TCP}\textbf{[TCP]} &
+	TRANSMISSION CONTROL PROTOCOL
+	\newline\url{https://www.rfc-editor.org/rfc/rfc793}\\
+
 
 \end{longtable}
 
Signature · 1 line
2.19.1.6.gb485710b
author
#21 lore raw
Quoted 80 lines · Michael S. Tsirkin
Quoted 79 lines · Heng Qi

If the tunnel is used to encapsulate the packets, the hash calculated using the outer header of the receive packets is always fixed for the same flow packets, i.e. they will be steered to the same receive queue.

We add a feature bit VIRTIO_NET_F_HASH_TUNNEL and related bitmasks in \field{hash_types}, which instructs the device to calculate the hash using the inner headers of tunnel-encapsulated packets. Besides, values in \field{hash_report_tunnel} are added to report tunnel types.

---
v6:
1. Modify the wording of some sentences for clarity. @Michael S. Tsirkin
2. Fix some syntax issues. @Michael S. Tsirkin

v5:
1. Fix some syntax and capitalization issues. @Michael S. Tsirkin
2. Use encapsulated/encaptulation uniformly. @Michael S. Tsirkin
3. Move the links to introduction section. @Michael S. Tsirkin
4. Clarify some sentences. @Michael S. Tsirkin

v4:
1. Clarify some paragraphs. @Cornelia Huck
2. Fix the u8 type. @Cornelia Huck

v3:
1. Rename VIRTIO_NET_F_HASH_GRE_VXLAN_GENEVE_INNER to VIRTIO_NET_F_HASH_TUNNEL. @Jason Wang
2. Make things clearer. @Jason Wang @Michael S. Tsirkin
3. Keep the possibility to use inner hash for automatic receive steering. @Jason Wang
4. Add the "Tunnel packet" paragraph to avoid repeating the GRE etc. many times. @Michael S. Tsirkin

v2:
1. Add a feature bit for GRE/VXLAN/GENEVE inner hash. @Jason Wang
2. Chang \field{hash_tunnel} to \field{hash_report_tunnel}. @Jason Wang, @Michael S. Tsirkin

v1:
1. Remove the patch for the bitmask fix. @Michael S. Tsirkin
2. Clarify some paragraphs. @Jason Wang
3. Add \field{hash_tunnel} and VIRTIO_NET_HASH_REPORT_GRE. @Yuri Benditovich

 content.tex      | 191 +++++++++++++++++++++++++++++++++++++++++++++--
 introduction.tex |  19 +++++
 2 files changed, 203 insertions(+), 7 deletions(-)
content.tex +6 -1
index e863709..7845f6c 100644
@@ -3084,6 +3084,9 @@ \subsection{Feature bits}\label{sec:Device Types / Network Device / Feature bits
 \item[VIRTIO_NET_F_CTRL_MAC_ADDR(23)] Set MAC address through control
     channel.
 
+\item[VIRTIO_NET_F_HASH_TUNNEL(52)] Device supports inner
+    header hash for GRE, VXLAN and GENEVE tunnel-encapsulated packets.
+
 \item[VIRTIO_NET_F_NOTF_COAL(53)] Device supports notifications coalescing.
 
 \item[VIRTIO_NET_F_GUEST_USO4 (54)] Driver can receive USOv4 packets.
@@ -3095,7 +3098,8 @@ \subsection{Feature bits}\label{sec:Device Types / Network Device / Feature bits
  to several segments when each of these smaller packets has UDP header.
 
 \item[VIRTIO_NET_F_HASH_REPORT(57)] Device can report per-packet hash
-    value and a type of calculated hash.
+    value, a type of calculated hash, and, if VIRTIO_NET_F_HASH_TUNNEL
+    is negotiated, an encapsulation packet type.
 
 \item[VIRTIO_NET_F_GUEST_HDRLEN(59)] Driver can provide the exact \field{hdr_len}
     value. Device benefits from knowing the exact header length.
@@ -3140,6 +3144,7 @@ \subsubsection{Feature bit requirements}\label{sec:Device Types / Network Device
 \item[VIRTIO_NET_F_NOTF_COAL] Requires VIRTIO_NET_F_CTRL_VQ.
 \item[VIRTIO_NET_F_RSC_EXT] Requires VIRTIO_NET_F_HOST_TSO4 or VIRTIO_NET_F_HOST_TSO6.
 \item[VIRTIO_NET_F_RSS] Requires VIRTIO_NET_F_CTRL_VQ.
+\item[VIRTIO_NET_F_HASH_TUNNEL] Requires VIRTIO_NET_F_CTRL_VQ.
 \end{description}
 
Quoted 9 lines

content.tex \subsection{Device Operation}\label{sec:Device Types / Network Device / Device O · line 3394

Quoted 7 lines
content.tex +1 -1
 \subsubsection{Legacy Interface: Feature bits}\label{sec:Device Types / Network Device / Feature bits / Legacy Interface: Feature bits}
@@ -3386,7 +3391,8 @@ \subsection{Device Operation}\label{sec:Device Types / Network Device / Device O
         le16 num_buffers;
         le32 hash_value;        (Only if VIRTIO_NET_F_HASH_REPORT negotiated)
         le16 hash_report;       (Only if VIRTIO_NET_F_HASH_REPORT negotiated)
-        le16 padding_reserved;  (Only if VIRTIO_NET_F_HASH_REPORT negotiated)
+        u8 hash_report_tunnel;  (Only if VIRTIO_NET_F_HASH_REPORT negotiated, only valid of VIRTIO_NET_F_HASH_TUNNEL negotiated)

of -> if?

Sorry for the late reply.
It's Cornelia's suggestion, and 'of' seems to work fine.

Quoted 2 lines

also let's add: otherwise reserved?

Sure.

Quoted 42 lines
Quoted 42 lines
content.tex +23 -1
+        u8 padding_reserved;    (Only if VIRTIO_NET_F_HASH_REPORT negotiated)
 };
 \end{lstlisting}
 
@@ -3837,7 +3843,8 @@ \subsubsection{Processing of Incoming Packets}\label{sec:Device Types / Network
 A device attempts to calculate a per-packet hash in the following cases:
 \begin{itemize}
 \item The feature VIRTIO_NET_F_RSS was negotiated. The device uses the hash to determine the receive virtqueue to place incoming packets.
-\item The feature VIRTIO_NET_F_HASH_REPORT was negotiated. The device reports the hash value and the hash type with the packet.
+\item The feature VIRTIO_NET_F_HASH_REPORT was negotiated. The device reports the hash value and the hash type. If additionally
+VIRTIO_NET_F_HASH_TUNNEL was negotiated, the device reports the encapsulation type as well.
 \end{itemize}
 
 If the feature VIRTIO_NET_F_RSS was negotiated:
@@ -3863,8 +3870,36 @@ \subsubsection{Processing of Incoming Packets}\label{sec:Device Types / Network
  \ref{sec:Device Types / Network Device / Device Operation / Control Virtqueue / Automatic receive steering in multiqueue mode / Hash calculation}.
 \end{itemize}
 
+\subparagraph{Tunnel/Encapsulated packet}
+\label{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / Tunnel/Encapsulated packet}
+A tunnel packet is encapsulated from the original packet based on the tunneling
+protocol (only a single level of encapsulation is currently supported). The
+encapsulated packet contains an outer header and an inner header, and the device
+calculates the hash over either the inner header or the outer header.
+
+When the feature VIRTIO_NET_F_HASH_TUNNEL is negotiated and the corresponding
+encapsulation type is set in \field{hash_types}, the hash for a specific type of
+encapsulated packet is calculated over the inner as opposed to outer header.
 
 
+Supported encapsulation types are listed in \ref{sec:Device Types / Network Device /
+Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets /
+Supported/enabled hash types}.
+
+If both VIRTIO_NET_F_HASH_REPORT and VIRTIO_NET_F_HASH_TUNNEL are negotiated,
+the device can support inner hash calculation for \hyperref[intro:GRE]{[GRE]},
+\hyperref[intro:VXLAN]{[VXLAN]} and \hyperref[intro:GENEVE]{[GENEVE]}
+encapsulated packets, and the corresponding encapsulation type in \field{hash_types}
+is VIRTIO_NET_HASH_TYPE_{GRE, VXLAN, GENEVE}_INNER respectively. The value in
+\field{hash_report_tunnel} is VIRTIO_NET_HASH_REPORT_{NONE, GRE, VXLAN, GENEVE} respectively.
Quoted 9 lines

content.tex \subsubsection{Processing of Incoming Packets}\label{sec:Device Types / Network · line 3897

Quoted 5 lines
content.tex +5 -0
+
+If VIRTIO_NET_F_HASH_REPORT is negotiated but VIRTIO_NET_F_HASH_TUNNEL is not
+negotiated, the device calculates the hash over the outer header, and
+\field{hash_report} reports the hash type. \field{hash_report_tunnel}
+is no longer valid.

Does this mean that VIRTIO_NET_HASH_TYPE_{GRE, VXLAN, GENEVE}_INNER must not be set without VIRTIO_NET_F_HASH_TUNNEL?

Yes.

Quoted 4 lines

If yes then we can add conformance statements to this end and then drop the
"If the feature VIRTIO_NET_F_HASH_TUNNEL is negotiated" all over the place just relying on hash type being set instead.

Please see the response below.

Quoted 8 lines

Also, note that it has to be ok for device to expose hash types without negotiating VIRTIO_NET_F_HASH_TUNNEL since they are in config space.
Idea: do we want to decouple these completely? have VIRTIO_NET_F_HASH_TUNNEL just add hash_report_tunnel and have hash types speak for themselves? It makes things nicely orthogonal in that if one does not care about knowing encapsulation type (e.g. for RSS) one can disable VIRTIO_NET_F_HASH_TUNNEL.

This seems to have the following 3 problems:

1. This will break the specification. As described in the specification, any field
in the configuration space only exists when there is a corresponding feature bit:
"Device configuration space is generally used for rarely-changing or initialization-time parameters. Where configuration fields are optional, their existence is indicated by feature bits: ", which also applies to supported_hash_types.

2. If I'm not wrong, this seems to make the encapsulation hash types dependent on VIRTIO_NET_F_RSS, can the driver read supported_hash_types (including encapsulation hash types) if VIRTIO_NET_F_RSS is not negotiated? But that shouldn't be the case, we only use encapsulation hash types to provide guidance for hash calculation, but not necessarily use hash values to select receive queues (unless VIRTIO_NET_F_RSS is negotiated).

3. If all supported_hash_types speak for themselves, why do we need the VIRTIO_NET_F_HASH_TUNNEL feature, because as long as the encapsulation hash types exist and VIRTIO_NET_F_HASH_REPORT is negotiated, hash_report_tunnel exists correspondingly. "have VIRTIO_NET_F_HASH_TUNNEL just add hash_report_tunnel"? It seems like hash_report_tunnel will only work if the encapsulation hash types are present.

So, if I'm not wrong, the reasonable logic should be that VIRTIO_NET_F_HASH_TUNNEL is only used to declare that the device supports inner header hash and let the corresponding encapsulation hash type exist, when VIRTIO_NET_F_RSS is negotiated, we can use this hash value to select the receive queue. Considering the migration at the same time, if the dst device does not negotiate VIRTIO_NET_F_HASH_TUNNEL, the migration will fail; otherwise, use encapsulation hash types to determine whether the migration can succeed.

Thanks.

Quoted 236 lines

Objections? Jason?

Quoted 216 lines
content.tex +3 -0
+
 \subparagraph{Supported/enabled hash types}
 \label{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / Supported/enabled hash types}
+This paragraph relies on definitions from \hyperref[intro:IP]{[IP]},
+\hyperref[intro:UDP]{[UDP]} and \hyperref[intro:TCP]{[TCP]}.
 Hash types applicable for IPv4 packets:

\begin{lstlisting}
#define VIRTIO_NET_HASH_TYPE_IPv4 (1 << 0)

content.tex +146 -4
@@ -3884,6 +3919,22 @@ \subsubsection{Processing of Incoming Packets}\label{sec:Device Types / Network
 #define VIRTIO_NET_HASH_TYPE_UDP_EX            (1 << 8)
 \end{lstlisting}
 
+If the feature VIRTIO_NET_F_HASH_TUNNEL is negotiated, the encapsulation
+hash type below indicates that the hash is calculated over the inner
+header of the encapsulated packet:
+Hash type applicable for inner payload of the gre-encapsulated packet
+\begin{lstlisting}
+#define VIRTIO_NET_HASH_TYPE_GRE_INNER         (1 << 9)
+\end{lstlisting}
+Hash type applicable for inner payload of the vxlan-encapsulated packet
+\begin{lstlisting}
+#define VIRTIO_NET_HASH_TYPE_VXLAN_INNER       (1 << 10)
+\end{lstlisting}
+Hash type applicable for inner payload of the geneve-encapsulated packet
+\begin{lstlisting}
+#define VIRTIO_NET_HASH_TYPE_GENEVE_INNER      (1 << 11)
+\end{lstlisting}
+
 \subparagraph{IPv4 packets}
 \label{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / IPv4 packets}
 The device calculates the hash on IPv4 packets according to 'Enabled hash types' bitmask as follows:
@@ -3975,15 +4026,125 @@ \subsubsection{Processing of Incoming Packets}\label{sec:Device Types / Network
 (see \ref{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / IPv6 packets without extension header}).
 \end{itemize}
 
+\subparagraph{Inner payload of an encapsulated packet}
+\label{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / Inner payload of the encapsulated packet}
+If the feature VIRTIO_NET_F_HASH_TUNNEL is negotiated and the corresponding
+encapsulation hash type is set in \field{hash_types}, the device calculates the
+inner header hash on an encapsulated packet (See \ref{sec:Device Types
+/ Network Device / Device Operation / Processing of Incoming Packets /
+Hash calculation for incoming packets / Tunnel/Encapsulated packet}) as follows:
+
+The device calculates the hash on the inner IPv4 packet of an encapsulated packet
+according to 'Enabled hash types' bitmask as follows:
+\begin{itemize}
+  \item If VIRTIO_NET_HASH_TYPE_TCPv4 is set and the encapsulated packet has an inner
+	TCPv4 header, the hash is calculated over the following fields:
+        \begin{itemsize}
+          \item inner Source IP address
+          \item inner Destination IP address
+          \item inner Source TCP port
+          \item inner Destination TCP port
+        \end{itemsize}
+  \item Else if VIRTIO_NET_HASH_TYPE_UDPv4 is set and the encapsulated packet has an
+	inner UDPv4 header, the hash is calculated over the following fields:
+        \begin{itemsize}
+          \item inner Source IP address
+          \item inner Destination IP address
+          \item inner Source UDP port
+          \item inner Destination UDP port
+        \end{itemize}
+  \item Else if VIRTIO_NET_HASH_TYPE_IPv4 is set, the hash is calculated over the
+	following fields:
+        \begin{itemsize}
+          \item inner Source IP address
+          \item inner Destination IP address
+        \end{itemsize}
+  \item Else the device does not calculate the hash
+\end{itemize}
+
+The device calculates the hash on the inner IPv6 packet without an extension header
+of an encapsulated packet according to 'Enabled hash types' bitmask as follows:
+\begin{itemize}
+  \item If VIRTIO_NET_HASH_TYPE_TCPv6 is set and the encapsulated packet has an inner
+	TCPv6 header, the hash is calculated over the following fields:
+        \begin{itemsize}
+          \item inner Source IPv6 address
+          \item inner Destination IPv6 address
+          \item inner Source TCP port
+          \item inner Destination TCP port
+        \end{itemsize}
+  \item Else if VIRTIO_NET_HASH_TYPE_UDPv6 is set and the encapsulated packet has an
+	inner UDPv6 header, the hash is calculated over the following fields:
+        \begin{itemsize}
+          \item inner Source IPv6 address
+          \item inner Destination IPv6 address
+          \item inner Source UDP port
+          \item inner Destination UDP port
+        \end{itemize}
+  \item Else if VIRTIO_NET_HASH_TYPE_IPv6 is set, the hash is calculated over the
+	following fields:
+        \begin{itemsize}
+          \item inner Source IPv6 address
+          \item inner Destination IPv6 address
+        \end{itemsize}
+  \item Else the device does not calculate the hash
+\end{itemize}
+
+The device calculates the hash on the inner IPv6 packet with an extension header
+of an encapsulated packet according to 'Enabled hash types' bitmask as follows:
+\begin{itemsize}
+  \item If VIRTIO_NET_HASH_TYPE_TCP_EX is set and the encapsulated packet has an inner
+	TCPv6 header, the hash is calculated over the following fields:
+        \begin{itemize}
+          \item Home address from the home address option in the inner IPv6 destination
+              options header. If the inner extension header is not present, use the
+              inner Source IPv6 address.
+          \item IPv6 address that is contained in the Routing-Header-Type-2 from the
+              associated inner extension header. If the inner extension header is not
+              present, use the inner Destination IPv6 address.
+          \item inner Source TCP port
+          \item inner Destination TCP port
+        \end{itemize}
+  \item Else if VIRTIO_NET_HASH_TYPE_UDP_EX is set and the encapsulated packet has an inner
+	UDPv6 header, the hash is calculated over the following fields:
+        \begin{itemsize}
+          \item Home address from the home address option in the inner IPv6 destination
+              options header. If the inner extension header is not present, use the
+              inner Source IPv6 address.
+          \item IPv6 address that is contained in the Routing-Header-Type-2 from the
+              associated inner extension header. If the inner extension header is not
+              present, use the inner Destination IPv6 address.
+          \item inner Source UDP port
+          \item inner Destination UDP port
+        \end{itemize}
+  \item Else if VIRTIO_NET_HASH_TYPE_IP_EX is set, the hash is calculated over the
+	following fields:
+        \begin{itemsize}
+          \item Home address from the home address option in the inner IPv6 destination
+              options header. If the inner extension header is not present, use the
+              inner Source IPv6 address.
+          \item IPv6 address that is contained in the Routing-Header-Type-2 from the
+              associated inner extension header. If the inner extension header is not
+              present, use the inner Destination IPv6 address.
+        \end{itemize}
+  \item Else skip the inner IPv6 extension header and calculate the inner header hash as
+	defined for an encapsulated packet whose inner payload is an IPv6 packet without
+	an extension header.
+\end{itemsize}
+
 \paragraph{Hash reporting for incoming packets}
 \label{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash reporting for incoming packets}
 
-If VIRTIO_NET_F_HASH_REPORT was negotiated and
- the device has calculated the hash for the packet, the device fills \field{hash_report} with the report type of calculated hash
-and \field{hash_value} with the value of calculated hash.
+If VIRTIO_NET_F_HASH_REPORT was negotiated and the device has calculated the
+hash for the packet, the device fills \field{hash_report} with the report type
+of calculated hash, and \field{hash_value} with the value of calculated hash.
+Also, if VIRTIO_NET_F_HASH_TUNNEL was negotiated, the device needs to fill
+\field{hash_report_tunnel} with the report type of the encapsulated packet, and
+it is set to VIRTIO_NET_HASH_REPORT_TUNNEL_NONE for the unencapsulated packet.
 
 If VIRTIO_NET_F_HASH_REPORT was negotiated but due to any reason the
-hash was not calculated, the device sets \field{hash_report} to VIRTIO_NET_HASH_REPORT_NONE.
+hash was not calculated, the device sets \field{hash_report} to VIRTIO_NET_HASH_REPORT_NONE,
+and sets \field{hash_report_tunnel} to VIRTIO_NET_HASH_REPORT_TUNNEL_NONE.
 
 Possible values that the device can report in \field{hash_report} are defined below.
 They correspond to supported hash types defined in
@@ -4005,6 +4166,22 @@ \subsubsection{Processing of Incoming Packets}\label{sec:Device Types / Network
 #define VIRTIO_NET_HASH_REPORT_UDPv6_EX        9
 \end{lstlisting}
 
+If \field{hash_report} differs from VIRTIO_NET_HASH_REPORT_NONE,
+\field{hash_report_tunnel} can report the type of the encapsulated
+packet to the driver over the inner header hash calculation.
+Possible values that the device can report in \field{hash_report_tunnel}
+are defined below:
+
+\begin{lstlisting}
+#define VIRTIO_NET_HASH_REPORT_TUNNEL_NONE     0
+#define VIRTIO_NET_HASH_REPORT_GRE             1
+#define VIRTIO_NET_HASH_REPORT_VXLAN           2
+#define VIRTIO_NET_HASH_REPORT_GENEVE          3
+\end{lstlisting}
+
+They correspond to supported hash types defined in
+\ref{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / Supported/enabled hash types}.
+
 \subsubsection{Control Virtqueue}\label{sec:Device Types / Network Device / Device Operation / Control Virtqueue}
 
 The driver uses the control virtqueue (if VIRTIO_NET_F_CTRL_VQ is
introduction.tex +19 -0
index 287c5fc..ff01a9b 100644
@@ -98,6 +98,25 @@ \section{Normative References}\label{sec:Normative References}
 	\phantomsection\label{intro:SEC1}\textbf{[SEC1]} &
     Standards for Efficient Cryptography Group(SECG), ``SEC1: Elliptic Cureve Cryptography'', Version 1.0, September 2000.
 	\newline\url{https://www.secg.org/sec1-v2.pdf}\\
+	\phantomsection\label{intro:GRE}\textbf{[GRE]} &
+	Generic Routing Encapsulation
+	\newline\url{https://datatracker.ietf.org/doc/rfc2784/}\\
+	\phantomsection\label{intro:VXLAN}\textbf{[VXLAN]} &
+	Virtual eXtensible Local Area Network
+	\newline\url{https://datatracker.ietf.org/doc/rfc7348/}\\
+	\phantomsection\label{intro:GENEVE}\textbf{[GENEVE]} &
+	Generic Network Virtualization Encapsulation
+	\newline\url{https://datatracker.ietf.org/doc/rfc8926/}\\
+	\phantomsection\label{intro:IP}\textbf{[IP]} &
+	INTERNET PROTOCOL
+	\newline\url{https://www.rfc-editor.org/rfc/rfc791}\\
+	\phantomsection\label{intro:UDP}\textbf{[UDP]} &
+	User Datagram Protocol
+	\newline\url{https://www.rfc-editor.org/rfc/rfc768}\\
+	\phantomsection\label{intro:TCP}\textbf{[TCP]} &
+	TRANSMISSION CONTROL PROTOCOL
+	\newline\url{https://www.rfc-editor.org/rfc/rfc793}\\
+
 
 \end{longtable}
 
Signature · 1 line
2.19.1.6.gb485710b

This publicly archived list offers a means to provide input to the OASIS Virtual I/O Device (VIRTIO) TC.

In order to verify user consent to the Feedback License terms and to minimize spam in the list archive, subscription is required before posting.

Subscribe: [email protected]
Unsubscribe: [email protected]
List help: [email protected]
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/

#22 lore raw
Quoted 11 lines · Heng Qi
Quoted 9 lines · Michael S. Tsirkin

content.tex \subsection{Device Operation}\label{sec:Device Types / Network Device / Device O · line 3394

Quoted 6 lines · Heng Qi
content.tex +1 -1
@@ -3386,7 +3391,8 @@ \subsection{Device Operation}\label{sec:Device Types / Network Device / Device O
         le16 num_buffers;
         le32 hash_value;        (Only if VIRTIO_NET_F_HASH_REPORT negotiated)
         le16 hash_report;       (Only if VIRTIO_NET_F_HASH_REPORT negotiated)
-        le16 padding_reserved;  (Only if VIRTIO_NET_F_HASH_REPORT negotiated)
+        u8 hash_report_tunnel;  (Only if VIRTIO_NET_F_HASH_REPORT negotiated, only valid of VIRTIO_NET_F_HASH_TUNNEL negotiated)

of -> if?

Quoted 2 lines

Sorry for the late reply.
It's Cornelia's suggestion, and 'of' seems to work fine.

I think that was simply a typo, should have been 'if'.

---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]

author
#23 lore raw
Quoted 13 lines · Cornelia Huck
Quoted 11 lines · Heng Qi
Quoted 9 lines · Michael S. Tsirkin

content.tex \subsection{Device Operation}\label{sec:Device Types / Network Device / Device O · line 3394

Quoted 6 lines · Heng Qi
content.tex +1 -1
@@ -3386,7 +3391,8 @@ \subsection{Device Operation}\label{sec:Device Types / Network Device / Device O
         le16 num_buffers;
         le32 hash_value;        (Only if VIRTIO_NET_F_HASH_REPORT negotiated)
         le16 hash_report;       (Only if VIRTIO_NET_F_HASH_REPORT negotiated)
-        le16 padding_reserved;  (Only if VIRTIO_NET_F_HASH_REPORT negotiated)
+        u8 hash_report_tunnel;  (Only if VIRTIO_NET_F_HASH_REPORT negotiated, only valid of VIRTIO_NET_F_HASH_TUNNEL negotiated)

of -> if?

Quoted 4 lines
Quoted 2 lines

Sorry for the late reply.
It's Cornelia's suggestion, and 'of' seems to work fine.

I think that was simply a typo, should have been 'if'.

Sorry for the late reply due to vacation.

It is my mistake, sorry.

Thanks.

Quoted 5 lines

---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]

#24 lore raw
In reply to Heng Qi
Quoted 178 lines · Heng Qi
Quoted 80 lines · Michael S. Tsirkin
Quoted 79 lines · Heng Qi

If the tunnel is used to encapsulate the packets, the hash calculated using the outer header of the receive packets is always fixed for the same flow packets, i.e. they will be steered to the same receive queue.

We add a feature bit VIRTIO_NET_F_HASH_TUNNEL and related bitmasks in \field{hash_types}, which instructs the device to calculate the hash using the inner headers of tunnel-encapsulated packets. Besides, values in \field{hash_report_tunnel} are added to report tunnel types.

---
v6:
1. Modify the wording of some sentences for clarity. @Michael S. Tsirkin
2. Fix some syntax issues. @Michael S. Tsirkin

v5:
1. Fix some syntax and capitalization issues. @Michael S. Tsirkin
2. Use encapsulated/encaptulation uniformly. @Michael S. Tsirkin
3. Move the links to introduction section. @Michael S. Tsirkin
4. Clarify some sentences. @Michael S. Tsirkin

v4:
1. Clarify some paragraphs. @Cornelia Huck
2. Fix the u8 type. @Cornelia Huck

v3:
1. Rename VIRTIO_NET_F_HASH_GRE_VXLAN_GENEVE_INNER to VIRTIO_NET_F_HASH_TUNNEL. @Jason Wang
2. Make things clearer. @Jason Wang @Michael S. Tsirkin
3. Keep the possibility to use inner hash for automatic receive steering. @Jason Wang
4. Add the "Tunnel packet" paragraph to avoid repeating the GRE etc. many times. @Michael S. Tsirkin

v2:
1. Add a feature bit for GRE/VXLAN/GENEVE inner hash. @Jason Wang
2. Chang \field{hash_tunnel} to \field{hash_report_tunnel}. @Jason Wang, @Michael S. Tsirkin

v1:
1. Remove the patch for the bitmask fix. @Michael S. Tsirkin
2. Clarify some paragraphs. @Jason Wang
3. Add \field{hash_tunnel} and VIRTIO_NET_HASH_REPORT_GRE. @Yuri Benditovich

  content.tex      | 191 +++++++++++++++++++++++++++++++++++++++++++++--
  introduction.tex |  19 +++++
  2 files changed, 203 insertions(+), 7 deletions(-)
content.tex +6 -1
index e863709..7845f6c 100644
@@ -3084,6 +3084,9 @@ \subsection{Feature bits}\label{sec:Device Types / Network Device / Feature bits
  \item[VIRTIO_NET_F_CTRL_MAC_ADDR(23)] Set MAC address through control
      channel.
 
+\item[VIRTIO_NET_F_HASH_TUNNEL(52)] Device supports inner
+    header hash for GRE, VXLAN and GENEVE tunnel-encapsulated packets.
+
  \item[VIRTIO_NET_F_NOTF_COAL(53)] Device supports notifications coalescing.
 
  \item[VIRTIO_NET_F_GUEST_USO4 (54)] Driver can receive USOv4 packets.
@@ -3095,7 +3098,8 @@ \subsection{Feature bits}\label{sec:Device Types / Network Device / Feature bits
   to several segments when each of these smaller packets has UDP header.
 
  \item[VIRTIO_NET_F_HASH_REPORT(57)] Device can report per-packet hash
-    value and a type of calculated hash.
+    value, a type of calculated hash, and, if VIRTIO_NET_F_HASH_TUNNEL
+    is negotiated, an encapsulation packet type.
 
  \item[VIRTIO_NET_F_GUEST_HDRLEN(59)] Driver can provide the exact \field{hdr_len}
      value. Device benefits from knowing the exact header length.
@@ -3140,6 +3144,7 @@ \subsubsection{Feature bit requirements}\label{sec:Device Types / Network Device
  \item[VIRTIO_NET_F_NOTF_COAL] Requires VIRTIO_NET_F_CTRL_VQ.
  \item[VIRTIO_NET_F_RSC_EXT] Requires VIRTIO_NET_F_HOST_TSO4 or VIRTIO_NET_F_HOST_TSO6.
  \item[VIRTIO_NET_F_RSS] Requires VIRTIO_NET_F_CTRL_VQ.
+\item[VIRTIO_NET_F_HASH_TUNNEL] Requires VIRTIO_NET_F_CTRL_VQ.
  \end{description}
 
Quoted 8 lines

content.tex \subsection{Device Operation}\label{sec:Device Types / Network Device / Device O · line 3394

Quoted 7 lines
content.tex +1 -1
  \subsubsection{Legacy Interface: Feature bits}\label{sec:Device Types / Network Device / Feature bits / Legacy Interface: Feature bits}
@@ -3386,7 +3391,8 @@ \subsection{Device Operation}\label{sec:Device Types / Network Device / Device O
          le16 num_buffers;
          le32 hash_value;        (Only if VIRTIO_NET_F_HASH_REPORT negotiated)
          le16 hash_report;       (Only if VIRTIO_NET_F_HASH_REPORT negotiated)
-        le16 padding_reserved;  (Only if VIRTIO_NET_F_HASH_REPORT negotiated)
+        u8 hash_report_tunnel;  (Only if VIRTIO_NET_F_HASH_REPORT negotiated, only valid of VIRTIO_NET_F_HASH_TUNNEL negotiated)

of -> if?

Sorry for the late reply.
It's Cornelia's suggestion, and 'of' seems to work fine.

Quoted 1 line

also let's add: otherwise reserved?

Sure.

Quoted 39 lines
Quoted 39 lines
content.tex +22 -1
+        u8 padding_reserved;    (Only if VIRTIO_NET_F_HASH_REPORT negotiated)
  };
  \end{lstlisting}
 
@@ -3837,7 +3843,8 @@ \subsubsection{Processing of Incoming Packets}\label{sec:Device Types / Network
  A device attempts to calculate a per-packet hash in the following cases:
  \begin{itemize}
  \item The feature VIRTIO_NET_F_RSS was negotiated. The device uses the hash to determine the receive virtqueue to place incoming packets.
-\item The feature VIRTIO_NET_F_HASH_REPORT was negotiated. The device reports the hash value and the hash type with the packet.
+\item The feature VIRTIO_NET_F_HASH_REPORT was negotiated. The device reports the hash value and the hash type. If additionally
+VIRTIO_NET_F_HASH_TUNNEL was negotiated, the device reports the encapsulation type as well.
  \end{itemize}
 
  If the feature VIRTIO_NET_F_RSS was negotiated:
@@ -3863,8 +3870,36 @@ \subsubsection{Processing of Incoming Packets}\label{sec:Device Types / Network
   \ref{sec:Device Types / Network Device / Device Operation / Control Virtqueue / Automatic receive steering in multiqueue mode / Hash calculation}.
  \end{itemize}
 
+\subparagraph{Tunnel/Encapsulated packet}
+\label{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / Tunnel/Encapsulated packet}
+A tunnel packet is encapsulated from the original packet based on the tunneling
+protocol (only a single level of encapsulation is currently supported). The
+encapsulated packet contains an outer header and an inner header, and the device
+calculates the hash over either the inner header or the outer header.
+
+When the feature VIRTIO_NET_F_HASH_TUNNEL is negotiated and the corresponding
+encapsulation type is set in \field{hash_types}, the hash for a specific type of
+encapsulated packet is calculated over the inner as opposed to outer header.
 
+Supported encapsulation types are listed in \ref{sec:Device Types / Network Device /
+Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets /
+Supported/enabled hash types}.
+
+If both VIRTIO_NET_F_HASH_REPORT and VIRTIO_NET_F_HASH_TUNNEL are negotiated,
+the device can support inner hash calculation for \hyperref[intro:GRE]{[GRE]},
+\hyperref[intro:VXLAN]{[VXLAN]} and \hyperref[intro:GENEVE]{[GENEVE]}
+encapsulated packets, and the corresponding encapsulation type in \field{hash_types}
+is VIRTIO_NET_HASH_TYPE_{GRE, VXLAN, GENEVE}_INNER respectively. The value in
Quoted 9 lines

content.tex \subsubsection{Processing of Incoming Packets}\label{sec:Device Types / Network · line 3897

Quoted 6 lines
content.tex +6 -0
+\field{hash_report_tunnel} is VIRTIO_NET_HASH_REPORT_{NONE, GRE, VXLAN, GENEVE} respectively.
+
+If VIRTIO_NET_F_HASH_REPORT is negotiated but VIRTIO_NET_F_HASH_TUNNEL is not
+negotiated, the device calculates the hash over the outer header, and
+\field{hash_report} reports the hash type. \field{hash_report_tunnel}
+is no longer valid.

Does this mean that VIRTIO_NET_HASH_TYPE_{GRE, VXLAN, GENEVE}_INNER must not be set without VIRTIO_NET_F_HASH_TUNNEL?

Yes.

Quoted 4 lines

If yes then we can add conformance statements to this end and then drop the
"If the feature VIRTIO_NET_F_HASH_TUNNEL is negotiated" all over the place just relying on hash type being set instead.

Please see the response below.

Quoted 7 lines

Also, note that it has to be ok for device to expose hash types without negotiating VIRTIO_NET_F_HASH_TUNNEL since they are in config space.
Idea: do we want to decouple these completely? have VIRTIO_NET_F_HASH_TUNNEL just add hash_report_tunnel and have hash types speak for themselves? It makes things nicely orthogonal in that if one does not care about knowing encapsulation type (e.g. for RSS) one can disable VIRTIO_NET_F_HASH_TUNNEL.

This seems to have the following 3 problems:

1. This will break the specification. As described in the specification, any field
in the configuration space only exists when there is a corresponding feature bit:
"Device configuration space is generally used for rarely-changing or initialization-time parameters. Where configuration fields are optional, their existence is indicated by feature bits: ", which also applies to supported_hash_types.

2. If I'm not wrong, this seems to make the encapsulation hash types dependent on VIRTIO_NET_F_RSS, can the driver read supported_hash_types (including encapsulation hash types) if VIRTIO_NET_F_RSS is not negotiated? But that shouldn't be the case, we only use encapsulation hash types to provide guidance for hash calculation, but not necessarily use hash values to select receive queues (unless VIRTIO_NET_F_RSS is negotiated).

3. If all supported_hash_types speak for themselves, why do we need the VIRTIO_NET_F_HASH_TUNNEL feature, because as long as the encapsulation hash types exist and VIRTIO_NET_F_HASH_REPORT is negotiated, hash_report_tunnel exists correspondingly. "have VIRTIO_NET_F_HASH_TUNNEL just add hash_report_tunnel"? It seems like hash_report_tunnel will only work if the encapsulation hash types are present.

Quoted 6 lines

So, if I'm not wrong, the reasonable logic should be that VIRTIO_NET_F_HASH_TUNNEL is only used to declare that the device supports inner header hash and let the corresponding encapsulation hash type exist, when VIRTIO_NET_F_RSS is negotiated, we can use this hash value to select the receive queue. Considering the migration at the same time, if the dst device does not negotiate VIRTIO_NET_F_HASH_TUNNEL, the migration will fail; otherwise, use encapsulation hash types to determine whether the migration can succeed.

I think it might be better to have consistency:

1) If VIRTIO_NET_F_HASH_TUNNEL introduces a new field in vnet header, is it better to have a new config filed in the config space?

or

2) If VIRTIO_NET_F_HASH_TUNNEL doesn't introduce new config file, should we try to reuse hash_report?

1) seems better and cleaner to me.

More below

Quoted 18 lines

Thanks.

Quoted 15 lines

Objections? Jason?

Quoted 12 lines
content.tex +3 -0
+
  \subparagraph{Supported/enabled hash types}
  \label{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / Supported/enabled hash types}
+This paragraph relies on definitions from \hyperref[intro:IP]{[IP]},
+\hyperref[intro:UDP]{[UDP]} and \hyperref[intro:TCP]{[TCP]}.
  Hash types applicable for IPv4 packets:
  \begin{lstlisting}
  #define VIRTIO_NET_HASH_TYPE_IPv4              (1 << 0)
content.tex +0 -0
@@ -3884,6 +3919,22 @@ \subsubsection{Processing of Incoming Packets}\label{sec:Device Types / Network
  #define VIRTIO_NET_HASH_TYPE_UDP_EX            (1 << 8)
  \end{lstlisting}
 
Quoted 15 lines
Quoted 7 lines
Quoted 7 lines
content.tex +7 -0
+If the feature VIRTIO_NET_F_HASH_TUNNEL is negotiated, the encapsulation
+hash type below indicates that the hash is calculated over the inner
+header of the encapsulated packet:
+Hash type applicable for inner payload of the gre-encapsulated packet
+\begin{lstlisting}
+#define VIRTIO_NET_HASH_TYPE_GRE_INNER         (1 << 9)
+\end{lstlisting}
Quoted 8 lines

content.tex \subsubsection{Processing of Incoming Packets}\label{sec:Device Types / Network · line 3936

Quoted 8 lines
content.tex +8 -0
+Hash type applicable for inner payload of the vxlan-encapsulated packet
+\begin{lstlisting}
+#define VIRTIO_NET_HASH_TYPE_VXLAN_INNER       (1 << 10)
+\end{lstlisting}
+Hash type applicable for inner payload of the geneve-encapsulated packet
+\begin{lstlisting}
+#define VIRTIO_NET_HASH_TYPE_GENEVE_INNER      (1 << 11)
+\end{lstlisting}

Just notice that this seems to break the equation:

VIRTIO_NET_HASH_TYPE_XXX = 1 « (VIRTIO_NET_HASH_REPORT_XXX - 1)

?

Thanks

Quoted 205 lines
Quoted 205 lines
Quoted 189 lines
content.tex +131 -4
+
  \subparagraph{IPv4 packets}
  \label{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / IPv4 packets}
  The device calculates the hash on IPv4 packets according to 'Enabled hash types' bitmask as follows:
@@ -3975,15 +4026,125 @@ \subsubsection{Processing of Incoming Packets}\label{sec:Device Types / Network
  (see \ref{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / IPv6 packets without extension header}).
  \end{itemize}
 
+\subparagraph{Inner payload of an encapsulated packet}
+\label{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / Inner payload of the encapsulated packet}
+If the feature VIRTIO_NET_F_HASH_TUNNEL is negotiated and the corresponding
+encapsulation hash type is set in \field{hash_types}, the device calculates the
+inner header hash on an encapsulated packet (See \ref{sec:Device Types
+/ Network Device / Device Operation / Processing of Incoming Packets /
+Hash calculation for incoming packets / Tunnel/Encapsulated packet}) as follows:
+
+The device calculates the hash on the inner IPv4 packet of an encapsulated packet
+according to 'Enabled hash types' bitmask as follows:
+\begin{itemize}
+  \item If VIRTIO_NET_HASH_TYPE_TCPv4 is set and the encapsulated packet has an inner
+	TCPv4 header, the hash is calculated over the following fields:
+        \begin{itemsize}
+          \item inner Source IP address
+          \item inner Destination IP address
+          \item inner Source TCP port
+          \item inner Destination TCP port
+        \end{itemsize}
+  \item Else if VIRTIO_NET_HASH_TYPE_UDPv4 is set and the encapsulated packet has an
+	inner UDPv4 header, the hash is calculated over the following fields:
+        \begin{itemsize}
+          \item inner Source IP address
+          \item inner Destination IP address
+          \item inner Source UDP port
+          \item inner Destination UDP port
+        \end{itemize}
+  \item Else if VIRTIO_NET_HASH_TYPE_IPv4 is set, the hash is calculated over the
+	following fields:
+        \begin{itemsize}
+          \item inner Source IP address
+          \item inner Destination IP address
+        \end{itemsize}
+  \item Else the device does not calculate the hash
+\end{itemize}
+
+The device calculates the hash on the inner IPv6 packet without an extension header
+of an encapsulated packet according to 'Enabled hash types' bitmask as follows:
+\begin{itemize}
+  \item If VIRTIO_NET_HASH_TYPE_TCPv6 is set and the encapsulated packet has an inner
+	TCPv6 header, the hash is calculated over the following fields:
+        \begin{itemsize}
+          \item inner Source IPv6 address
+          \item inner Destination IPv6 address
+          \item inner Source TCP port
+          \item inner Destination TCP port
+        \end{itemsize}
+  \item Else if VIRTIO_NET_HASH_TYPE_UDPv6 is set and the encapsulated packet has an
+	inner UDPv6 header, the hash is calculated over the following fields:
+        \begin{itemsize}
+          \item inner Source IPv6 address
+          \item inner Destination IPv6 address
+          \item inner Source UDP port
+          \item inner Destination UDP port
+        \end{itemize}
+  \item Else if VIRTIO_NET_HASH_TYPE_IPv6 is set, the hash is calculated over the
+	following fields:
+        \begin{itemsize}
+          \item inner Source IPv6 address
+          \item inner Destination IPv6 address
+        \end{itemsize}
+  \item Else the device does not calculate the hash
+\end{itemize}
+
+The device calculates the hash on the inner IPv6 packet with an extension header
+of an encapsulated packet according to 'Enabled hash types' bitmask as follows:
+\begin{itemsize}
+  \item If VIRTIO_NET_HASH_TYPE_TCP_EX is set and the encapsulated packet has an inner
+	TCPv6 header, the hash is calculated over the following fields:
+        \begin{itemize}
+          \item Home address from the home address option in the inner IPv6 destination
+              options header. If the inner extension header is not present, use the
+              inner Source IPv6 address.
+          \item IPv6 address that is contained in the Routing-Header-Type-2 from the
+              associated inner extension header. If the inner extension header is not
+              present, use the inner Destination IPv6 address.
+          \item inner Source TCP port
+          \item inner Destination TCP port
+        \end{itemize}
+  \item Else if VIRTIO_NET_HASH_TYPE_UDP_EX is set and the encapsulated packet has an inner
+	UDPv6 header, the hash is calculated over the following fields:
+        \begin{itemsize}
+          \item Home address from the home address option in the inner IPv6 destination
+              options header. If the inner extension header is not present, use the
+              inner Source IPv6 address.
+          \item IPv6 address that is contained in the Routing-Header-Type-2 from the
+              associated inner extension header. If the inner extension header is not
+              present, use the inner Destination IPv6 address.
+          \item inner Source UDP port
+          \item inner Destination UDP port
+        \end{itemize}
+  \item Else if VIRTIO_NET_HASH_TYPE_IP_EX is set, the hash is calculated over the
+	following fields:
+        \begin{itemsize}
+          \item Home address from the home address option in the inner IPv6 destination
+              options header. If the inner extension header is not present, use the
+              inner Source IPv6 address.
+          \item IPv6 address that is contained in the Routing-Header-Type-2 from the
+              associated inner extension header. If the inner extension header is not
+              present, use the inner Destination IPv6 address.
+        \end{itemize}
+  \item Else skip the inner IPv6 extension header and calculate the inner header hash as
+	defined for an encapsulated packet whose inner payload is an IPv6 packet without
+	an extension header.
+\end{itemsize}
+
  \paragraph{Hash reporting for incoming packets}
  \label{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash reporting for incoming packets}
 
-If VIRTIO_NET_F_HASH_REPORT was negotiated and
- the device has calculated the hash for the packet, the device fills \field{hash_report} with the report type of calculated hash
-and \field{hash_value} with the value of calculated hash.
+If VIRTIO_NET_F_HASH_REPORT was negotiated and the device has calculated the
+hash for the packet, the device fills \field{hash_report} with the report type
+of calculated hash, and \field{hash_value} with the value of calculated hash.
+Also, if VIRTIO_NET_F_HASH_TUNNEL was negotiated, the device needs to fill
+\field{hash_report_tunnel} with the report type of the encapsulated packet, and
+it is set to VIRTIO_NET_HASH_REPORT_TUNNEL_NONE for the unencapsulated packet.
 
  If VIRTIO_NET_F_HASH_REPORT was negotiated but due to any reason the
-hash was not calculated, the device sets \field{hash_report} to VIRTIO_NET_HASH_REPORT_NONE.
+hash was not calculated, the device sets \field{hash_report} to VIRTIO_NET_HASH_REPORT_NONE,
+and sets \field{hash_report_tunnel} to VIRTIO_NET_HASH_REPORT_TUNNEL_NONE.
 
  Possible values that the device can report in \field{hash_report} are defined below.
  They correspond to supported hash types defined in
@@ -4005,6 +4166,22 @@ \subsubsection{Processing of Incoming Packets}\label{sec:Device Types / Network
  #define VIRTIO_NET_HASH_REPORT_UDPv6_EX        9
  \end{lstlisting}
 
+If \field{hash_report} differs from VIRTIO_NET_HASH_REPORT_NONE,
+\field{hash_report_tunnel} can report the type of the encapsulated
+packet to the driver over the inner header hash calculation.
+Possible values that the device can report in \field{hash_report_tunnel}
+are defined below:
+
+\begin{lstlisting}
+#define VIRTIO_NET_HASH_REPORT_TUNNEL_NONE     0
+#define VIRTIO_NET_HASH_REPORT_GRE             1
+#define VIRTIO_NET_HASH_REPORT_VXLAN           2
+#define VIRTIO_NET_HASH_REPORT_GENEVE          3
+\end{lstlisting}
+
+They correspond to supported hash types defined in
+\ref{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / Supported/enabled hash types}.
+
  \subsubsection{Control Virtqueue}\label{sec:Device Types / Network Device / Device Operation / Control Virtqueue}
 
  The driver uses the control virtqueue (if VIRTIO_NET_F_CTRL_VQ is
introduction.tex +19 -0
index 287c5fc..ff01a9b 100644
@@ -98,6 +98,25 @@ \section{Normative References}\label{sec:Normative References}
  	\phantomsection\label{intro:SEC1}\textbf{[SEC1]} &
      Standards for Efficient Cryptography Group(SECG), ``SEC1: Elliptic Cureve Cryptography'', Version 1.0, September 2000.
  	\newline\url{https://www.secg.org/sec1-v2.pdf}\\
+	\phantomsection\label{intro:GRE}\textbf{[GRE]} &
+	Generic Routing Encapsulation
+	\newline\url{https://datatracker.ietf.org/doc/rfc2784/}\\
+	\phantomsection\label{intro:VXLAN}\textbf{[VXLAN]} &
+	Virtual eXtensible Local Area Network
+	\newline\url{https://datatracker.ietf.org/doc/rfc7348/}\\
+	\phantomsection\label{intro:GENEVE}\textbf{[GENEVE]} &
+	Generic Network Virtualization Encapsulation
+	\newline\url{https://datatracker.ietf.org/doc/rfc8926/}\\
+	\phantomsection\label{intro:IP}\textbf{[IP]} &
+	INTERNET PROTOCOL
+	\newline\url{https://www.rfc-editor.org/rfc/rfc791}\\
+	\phantomsection\label{intro:UDP}\textbf{[UDP]} &
+	User Datagram Protocol
+	\newline\url{https://www.rfc-editor.org/rfc/rfc768}\\
+	\phantomsection\label{intro:TCP}\textbf{[TCP]} &
+	TRANSMISSION CONTROL PROTOCOL
+	\newline\url{https://www.rfc-editor.org/rfc/rfc793}\\
+
 
  \end{longtable}
 
Signature · 1 line
2.19.1.6.gb485710b

This publicly archived list offers a means to provide input to the OASIS Virtual I/O Device (VIRTIO) TC.

In order to verify user consent to the Feedback License terms and to minimize spam in the list archive, subscription is required before posting.

Subscribe: [email protected]
Unsubscribe: [email protected]
List help: [email protected]
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/

author
#25 lore raw
Quoted 185 lines · Jason Wang
Quoted 170 lines · Heng Qi
Quoted 75 lines · Michael S. Tsirkin
Quoted 74 lines · Heng Qi

If the tunnel is used to encapsulate the packets, the hash calculated using the outer header of the receive packets is always fixed for the same flow packets, i.e. they will be steered to the same receive queue.

We add a feature bit VIRTIO_NET_F_HASH_TUNNEL and related bitmasks in \field{hash_types}, which instructs the device to calculate the hash using the inner headers of tunnel-encapsulated packets. Besides, values in \field{hash_report_tunnel} are added to report tunnel types.

---
v6:
1. Modify the wording of some sentences for clarity. @Michael S. Tsirkin
2. Fix some syntax issues. @Michael S. Tsirkin

v5:
1. Fix some syntax and capitalization issues. @Michael S. Tsirkin
2. Use encapsulated/encaptulation uniformly. @Michael S. Tsirkin
3. Move the links to introduction section. @Michael S. Tsirkin
4. Clarify some sentences. @Michael S. Tsirkin

v4:
1. Clarify some paragraphs. @Cornelia Huck
2. Fix the u8 type. @Cornelia Huck

v3:
1. Rename VIRTIO_NET_F_HASH_GRE_VXLAN_GENEVE_INNER to VIRTIO_NET_F_HASH_TUNNEL. @Jason Wang
2. Make things clearer. @Jason Wang @Michael S. Tsirkin
3. Keep the possibility to use inner hash for automatic receive steering. @Jason Wang
4. Add the "Tunnel packet" paragraph to avoid repeating the GRE etc. many times. @Michael S. Tsirkin

v2:
1. Add a feature bit for GRE/VXLAN/GENEVE inner hash. @Jason Wang
2. Chang \field{hash_tunnel} to \field{hash_report_tunnel}. @Jason Wang, @Michael S. Tsirkin

v1:
1. Remove the patch for the bitmask fix. @Michael S. Tsirkin
2. Clarify some paragraphs. @Jason Wang
3. Add \field{hash_tunnel} and VIRTIO_NET_HASH_REPORT_GRE. @Yuri Benditovich

 content.tex      | 191 +++++++++++++++++++++++++++++++++++++++++++++--
 introduction.tex |  19 +++++
 2 files changed, 203 insertions(+), 7 deletions(-)
content.tex +6 -1
index e863709..7845f6c 100644
@@ -3084,6 +3084,9 @@ \subsection{Feature bits}\label{sec:Device Types / Network Device / Feature bits
 \item[VIRTIO_NET_F_CTRL_MAC_ADDR(23)] Set MAC address through control
     channel.
+\item[VIRTIO_NET_F_HASH_TUNNEL(52)] Device supports inner
+    header hash for GRE, VXLAN and GENEVE tunnel-encapsulated packets.
+
 \item[VIRTIO_NET_F_NOTF_COAL(53)] Device supports notifications coalescing.
 \item[VIRTIO_NET_F_GUEST_USO4 (54)] Driver can receive USOv4 packets.
@@ -3095,7 +3098,8 @@ \subsection{Feature bits}\label{sec:Device Types / Network Device / Feature bits
  to several segments when each of these smaller packets has UDP header.
 \item[VIRTIO_NET_F_HASH_REPORT(57)] Device can report per-packet hash
-    value and a type of calculated hash.
+    value, a type of calculated hash, and, if VIRTIO_NET_F_HASH_TUNNEL
+    is negotiated, an encapsulation packet type.
 \item[VIRTIO_NET_F_GUEST_HDRLEN(59)] Driver can provide the exact \field{hdr_len}
     value. Device benefits from knowing the exact header length.
@@ -3140,6 +3144,7 @@ \subsubsection{Feature bit requirements}\label{sec:Device Types / Network Device
 \item[VIRTIO_NET_F_NOTF_COAL] Requires VIRTIO_NET_F_CTRL_VQ.
 \item[VIRTIO_NET_F_RSC_EXT] Requires VIRTIO_NET_F_HOST_TSO4 or VIRTIO_NET_F_HOST_TSO6.
 \item[VIRTIO_NET_F_RSS] Requires VIRTIO_NET_F_CTRL_VQ.
+\item[VIRTIO_NET_F_HASH_TUNNEL] Requires VIRTIO_NET_F_CTRL_VQ.
 \end{description}
Quoted 8 lines

content.tex \subsection{Device Operation}\label{sec:Device Types / Network Device / Device O · line 3394

Quoted 7 lines
content.tex +1 -1
 \subsubsection{Legacy Interface: Feature bits}\label{sec:Device Types / Network Device / Feature bits / Legacy Interface: Feature bits}
@@ -3386,7 +3391,8 @@ \subsection{Device Operation}\label{sec:Device Types / Network Device / Device O
         le16 num_buffers;
         le32 hash_value;        (Only if VIRTIO_NET_F_HASH_REPORT negotiated)
         le16 hash_report;       (Only if VIRTIO_NET_F_HASH_REPORT negotiated)
-        le16 padding_reserved;  (Only if VIRTIO_NET_F_HASH_REPORT negotiated)
+        u8 hash_report_tunnel;  (Only if VIRTIO_NET_F_HASH_REPORT negotiated, only valid of VIRTIO_NET_F_HASH_TUNNEL negotiated)

of -> if?

Sorry for the late reply.
It's Cornelia's suggestion, and 'of' seems to work fine.

Quoted 1 line

also let's add: otherwise reserved?

Sure.

Quoted 36 lines
Quoted 36 lines
content.tex +22 -1
+        u8 padding_reserved;    (Only if VIRTIO_NET_F_HASH_REPORT negotiated)
 };
 \end{lstlisting}
@@ -3837,7 +3843,8 @@ \subsubsection{Processing of Incoming Packets}\label{sec:Device Types / Network
 A device attempts to calculate a per-packet hash in the following cases:
 \begin{itemize}
 \item The feature VIRTIO_NET_F_RSS was negotiated. The device uses the hash to determine the receive virtqueue to place incoming packets.
-\item The feature VIRTIO_NET_F_HASH_REPORT was negotiated. The device reports the hash value and the hash type with the packet.
+\item The feature VIRTIO_NET_F_HASH_REPORT was negotiated. The device reports the hash value and the hash type. If additionally
+VIRTIO_NET_F_HASH_TUNNEL was negotiated, the device reports the encapsulation type as well.
 \end{itemize}
 If the feature VIRTIO_NET_F_RSS was negotiated:
@@ -3863,8 +3870,36 @@ \subsubsection{Processing of Incoming Packets}\label{sec:Device Types / Network
  \ref{sec:Device Types / Network Device / Device Operation / Control Virtqueue / Automatic receive steering in multiqueue mode / Hash calculation}.
 \end{itemize}
+\subparagraph{Tunnel/Encapsulated packet}
+\label{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / Tunnel/Encapsulated packet}
+A tunnel packet is encapsulated from the original packet based on the tunneling
+protocol (only a single level of encapsulation is currently supported). The
+encapsulated packet contains an outer header and an inner header, and the device
+calculates the hash over either the inner header or the outer header.
+
+When the feature VIRTIO_NET_F_HASH_TUNNEL is negotiated and the corresponding
+encapsulation type is set in \field{hash_types}, the hash for a specific type of
+encapsulated packet is calculated over the inner as opposed to outer header.
 
+Supported encapsulation types are listed in \ref{sec:Device Types / Network Device /
+Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets /
+Supported/enabled hash types}.
+
+If both VIRTIO_NET_F_HASH_REPORT and VIRTIO_NET_F_HASH_TUNNEL are negotiated,
+the device can support inner hash calculation for \hyperref[intro:GRE]{[GRE]},
+\hyperref[intro:VXLAN]{[VXLAN]} and \hyperref[intro:GENEVE]{[GENEVE]}
+encapsulated packets, and the corresponding encapsulation type in \field{hash_types}
+is VIRTIO_NET_HASH_TYPE_{GRE, VXLAN, GENEVE}_INNER respectively. The value in
Quoted 9 lines

content.tex \subsubsection{Processing of Incoming Packets}\label{sec:Device Types / Network · line 3897

Quoted 6 lines
content.tex +6 -0
+\field{hash_report_tunnel} is VIRTIO_NET_HASH_REPORT_{NONE, GRE, VXLAN, GENEVE} respectively.
+
+If VIRTIO_NET_F_HASH_REPORT is negotiated but VIRTIO_NET_F_HASH_TUNNEL is not
+negotiated, the device calculates the hash over the outer header, and
+\field{hash_report} reports the hash type. \field{hash_report_tunnel}
+is no longer valid.

Does this mean that VIRTIO_NET_HASH_TYPE_{GRE, VXLAN, GENEVE}_INNER must not be set without VIRTIO_NET_F_HASH_TUNNEL?

Yes.

Quoted 4 lines

If yes then we can add conformance statements to this end and then drop the
"If the feature VIRTIO_NET_F_HASH_TUNNEL is negotiated" all over the place just relying on hash type being set instead.

Please see the response below.

Quoted 7 lines

Also, note that it has to be ok for device to expose hash types without negotiating VIRTIO_NET_F_HASH_TUNNEL since they are in config space.
Idea: do we want to decouple these completely? have VIRTIO_NET_F_HASH_TUNNEL just add hash_report_tunnel and have hash types speak for themselves? It makes things nicely orthogonal in that if one does not care about knowing encapsulation type (e.g. for RSS) one can disable VIRTIO_NET_F_HASH_TUNNEL.

This seems to have the following 3 problems:

1. This will break the specification. As described in the specification, any field
in the configuration space only exists when there is a corresponding feature bit:
"Device configuration space is generally used for rarely-changing or initialization-time parameters. Where configuration fields are optional, their existence is indicated by feature bits: ", which also applies to supported_hash_types.

2. If I'm not wrong, this seems to make the encapsulation hash types dependent on VIRTIO_NET_F_RSS, can the driver read supported_hash_types (including encapsulation hash types) if VIRTIO_NET_F_RSS is not negotiated? But that shouldn't be the case, we only use encapsulation hash types to provide guidance for hash calculation, but not necessarily use hash values to select receive queues (unless VIRTIO_NET_F_RSS is negotiated).

3. If all supported_hash_types speak for themselves, why do we need the VIRTIO_NET_F_HASH_TUNNEL feature, because as long as the encapsulation hash types exist and VIRTIO_NET_F_HASH_REPORT is negotiated, hash_report_tunnel exists correspondingly. "have VIRTIO_NET_F_HASH_TUNNEL just add hash_report_tunnel"? It seems like hash_report_tunnel will only work if the encapsulation hash types are present.

Quoted 6 lines

So, if I'm not wrong, the reasonable logic should be that VIRTIO_NET_F_HASH_TUNNEL is only used to declare that the device supports inner header hash and let the corresponding encapsulation hash type exist, when VIRTIO_NET_F_RSS is negotiated, we can use this hash value to select the receive queue. Considering the migration at the same time, if the dst device does not negotiate VIRTIO_NET_F_HASH_TUNNEL, the migration will fail; otherwise, use encapsulation hash types to determine whether the migration can succeed.

I think it might be better to have consistency:

1) If VIRTIO_NET_F_HASH_TUNNEL introduces a new field in vnet header, is it better to have a new config filed in the config space?

Quoted 6 lines

or

2) If VIRTIO_NET_F_HASH_TUNNEL doesn't introduce new config file, should we try to reuse hash_report?

1) seems better and cleaner to me.

Sorry for the late reply due to vacation.

Good point, it's clearer to use the VIRTIO_NET_F_HASH_TUNNEL feature to control all information related to inner header hash.

Quoted 40 lines

More below

Quoted 24 lines

Thanks.

Quoted 21 lines

Objections? Jason?

Quoted 18 lines
content.tex +10 -0
+
 \subparagraph{Supported/enabled hash types}
 \label{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / Supported/enabled hash types}
+This paragraph relies on definitions from \hyperref[intro:IP]{[IP]},
+\hyperref[intro:UDP]{[UDP]} and \hyperref[intro:TCP]{[TCP]}.
 Hash types applicable for IPv4 packets:
 \begin{lstlisting}
 #define VIRTIO_NET_HASH_TYPE_IPv4              (1 << 0)
@@ -3884,6 +3919,22 @@ \subsubsection{Processing of Incoming Packets}\label{sec:Device Types / Network
 #define VIRTIO_NET_HASH_TYPE_UDP_EX            (1 << 8)
 \end{lstlisting}
+If the feature VIRTIO_NET_F_HASH_TUNNEL is negotiated, the encapsulation
+hash type below indicates that the hash is calculated over the inner
+header of the encapsulated packet:
+Hash type applicable for inner payload of the gre-encapsulated packet
+\begin{lstlisting}
+#define VIRTIO_NET_HASH_TYPE_GRE_INNER         (1 << 9)
+\end{lstlisting}
Quoted 8 lines
Quoted 8 lines

content.tex \subsubsection{Processing of Incoming Packets}\label{sec:Device Types / Network · line 3936

Quoted 8 lines
content.tex +8 -0
+Hash type applicable for inner payload of the vxlan-encapsulated packet
+\begin{lstlisting}
+#define VIRTIO_NET_HASH_TYPE_VXLAN_INNER       (1 << 10)
+\end{lstlisting}
+Hash type applicable for inner payload of the geneve-encapsulated packet
+\begin{lstlisting}
+#define VIRTIO_NET_HASH_TYPE_GENEVE_INNER      (1 << 11)
+\end{lstlisting}

Just notice that this seems to break the equation:

Quoted 4 lines

VIRTIO_NET_HASH_TYPE_XXX = 1 « (VIRTIO_NET_HASH_REPORT_XXX - 1)

?

Yes, we have discussed it here https://lists.oasis-open.org/archives/virtio-dev/202211/msg00189.html , as you suggested above, it seems better to use the new config space field.

Thanks.

Quoted 205 lines

Thanks

Quoted 189 lines
Quoted 189 lines
Quoted 181 lines
content.tex +131 -4
+
 \subparagraph{IPv4 packets}
 \label{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / IPv4 packets}
 The device calculates the hash on IPv4 packets according to 'Enabled hash types' bitmask as follows:
@@ -3975,15 +4026,125 @@ \subsubsection{Processing of Incoming Packets}\label{sec:Device Types / Network
 (see \ref{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / IPv6 packets without extension header}).
 \end{itemize}
+\subparagraph{Inner payload of an encapsulated packet}
+\label{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / Inner payload of the encapsulated packet}
+If the feature VIRTIO_NET_F_HASH_TUNNEL is negotiated and the corresponding
+encapsulation hash type is set in \field{hash_types}, the device calculates the
+inner header hash on an encapsulated packet (See \ref{sec:Device Types
+/ Network Device / Device Operation / Processing of Incoming Packets /
+Hash calculation for incoming packets / Tunnel/Encapsulated packet}) as follows:
+
+The device calculates the hash on the inner IPv4 packet of an encapsulated packet
+according to 'Enabled hash types' bitmask as follows:
+\begin{itemize}
+  \item If VIRTIO_NET_HASH_TYPE_TCPv4 is set and the encapsulated packet has an inner
+	TCPv4 header, the hash is calculated over the following fields:
+        \begin{itemsize}
+          \item inner Source IP address
+          \item inner Destination IP address
+          \item inner Source TCP port
+          \item inner Destination TCP port
+        \end{itemsize}
+  \item Else if VIRTIO_NET_HASH_TYPE_UDPv4 is set and the encapsulated packet has an
+	inner UDPv4 header, the hash is calculated over the following fields:
+        \begin{itemsize}
+          \item inner Source IP address
+          \item inner Destination IP address
+          \item inner Source UDP port
+          \item inner Destination UDP port
+        \end{itemize}
+  \item Else if VIRTIO_NET_HASH_TYPE_IPv4 is set, the hash is calculated over the
+	following fields:
+        \begin{itemsize}
+          \item inner Source IP address
+          \item inner Destination IP address
+        \end{itemsize}
+  \item Else the device does not calculate the hash
+\end{itemize}
+
+The device calculates the hash on the inner IPv6 packet without an extension header
+of an encapsulated packet according to 'Enabled hash types' bitmask as follows:
+\begin{itemize}
+  \item If VIRTIO_NET_HASH_TYPE_TCPv6 is set and the encapsulated packet has an inner
+	TCPv6 header, the hash is calculated over the following fields:
+        \begin{itemsize}
+          \item inner Source IPv6 address
+          \item inner Destination IPv6 address
+          \item inner Source TCP port
+          \item inner Destination TCP port
+        \end{itemsize}
+  \item Else if VIRTIO_NET_HASH_TYPE_UDPv6 is set and the encapsulated packet has an
+	inner UDPv6 header, the hash is calculated over the following fields:
+        \begin{itemsize}
+          \item inner Source IPv6 address
+          \item inner Destination IPv6 address
+          \item inner Source UDP port
+          \item inner Destination UDP port
+        \end{itemize}
+  \item Else if VIRTIO_NET_HASH_TYPE_IPv6 is set, the hash is calculated over the
+	following fields:
+        \begin{itemsize}
+          \item inner Source IPv6 address
+          \item inner Destination IPv6 address
+        \end{itemsize}
+  \item Else the device does not calculate the hash
+\end{itemize}
+
+The device calculates the hash on the inner IPv6 packet with an extension header
+of an encapsulated packet according to 'Enabled hash types' bitmask as follows:
+\begin{itemsize}
+  \item If VIRTIO_NET_HASH_TYPE_TCP_EX is set and the encapsulated packet has an inner
+	TCPv6 header, the hash is calculated over the following fields:
+        \begin{itemize}
+          \item Home address from the home address option in the inner IPv6 destination
+              options header. If the inner extension header is not present, use the
+              inner Source IPv6 address.
+          \item IPv6 address that is contained in the Routing-Header-Type-2 from the
+              associated inner extension header. If the inner extension header is not
+              present, use the inner Destination IPv6 address.
+          \item inner Source TCP port
+          \item inner Destination TCP port
+        \end{itemize}
+  \item Else if VIRTIO_NET_HASH_TYPE_UDP_EX is set and the encapsulated packet has an inner
+	UDPv6 header, the hash is calculated over the following fields:
+        \begin{itemsize}
+          \item Home address from the home address option in the inner IPv6 destination
+              options header. If the inner extension header is not present, use the
+              inner Source IPv6 address.
+          \item IPv6 address that is contained in the Routing-Header-Type-2 from the
+              associated inner extension header. If the inner extension header is not
+              present, use the inner Destination IPv6 address.
+          \item inner Source UDP port
+          \item inner Destination UDP port
+        \end{itemize}
+  \item Else if VIRTIO_NET_HASH_TYPE_IP_EX is set, the hash is calculated over the
+	following fields:
+        \begin{itemsize}
+          \item Home address from the home address option in the inner IPv6 destination
+              options header. If the inner extension header is not present, use the
+              inner Source IPv6 address.
+          \item IPv6 address that is contained in the Routing-Header-Type-2 from the
+              associated inner extension header. If the inner extension header is not
+              present, use the inner Destination IPv6 address.
+        \end{itemize}
+  \item Else skip the inner IPv6 extension header and calculate the inner header hash as
+	defined for an encapsulated packet whose inner payload is an IPv6 packet without
+	an extension header.
+\end{itemsize}
+
 \paragraph{Hash reporting for incoming packets}
 \label{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash reporting for incoming packets}
-If VIRTIO_NET_F_HASH_REPORT was negotiated and
- the device has calculated the hash for the packet, the device fills \field{hash_report} with the report type of calculated hash
-and \field{hash_value} with the value of calculated hash.
+If VIRTIO_NET_F_HASH_REPORT was negotiated and the device has calculated the
+hash for the packet, the device fills \field{hash_report} with the report type
+of calculated hash, and \field{hash_value} with the value of calculated hash.
+Also, if VIRTIO_NET_F_HASH_TUNNEL was negotiated, the device needs to fill
+\field{hash_report_tunnel} with the report type of the encapsulated packet, and
+it is set to VIRTIO_NET_HASH_REPORT_TUNNEL_NONE for the unencapsulated packet.
 If VIRTIO_NET_F_HASH_REPORT was negotiated but due to any reason the
-hash was not calculated, the device sets \field{hash_report} to VIRTIO_NET_HASH_REPORT_NONE.
+hash was not calculated, the device sets \field{hash_report} to VIRTIO_NET_HASH_REPORT_NONE,
+and sets \field{hash_report_tunnel} to VIRTIO_NET_HASH_REPORT_TUNNEL_NONE.
 Possible values that the device can report in \field{hash_report} are defined below.
 They correspond to supported hash types defined in
@@ -4005,6 +4166,22 @@ \subsubsection{Processing of Incoming Packets}\label{sec:Device Types / Network
 #define VIRTIO_NET_HASH_REPORT_UDPv6_EX        9
 \end{lstlisting}
+If \field{hash_report} differs from VIRTIO_NET_HASH_REPORT_NONE,
+\field{hash_report_tunnel} can report the type of the encapsulated
+packet to the driver over the inner header hash calculation.
+Possible values that the device can report in \field{hash_report_tunnel}
+are defined below:
+
+\begin{lstlisting}
+#define VIRTIO_NET_HASH_REPORT_TUNNEL_NONE     0
+#define VIRTIO_NET_HASH_REPORT_GRE             1
+#define VIRTIO_NET_HASH_REPORT_VXLAN           2
+#define VIRTIO_NET_HASH_REPORT_GENEVE          3
+\end{lstlisting}
+
+They correspond to supported hash types defined in
+\ref{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / Supported/enabled hash types}.
+
 \subsubsection{Control Virtqueue}\label{sec:Device Types / Network Device / Device Operation / Control Virtqueue}
 The driver uses the control virtqueue (if VIRTIO_NET_F_CTRL_VQ is
introduction.tex +19 -0
index 287c5fc..ff01a9b 100644
@@ -98,6 +98,25 @@ \section{Normative References}\label{sec:Normative References}
 	\phantomsection\label{intro:SEC1}\textbf{[SEC1]} &
     Standards for Efficient Cryptography Group(SECG), ``SEC1: Elliptic Cureve Cryptography'', Version 1.0, September 2000.
 	\newline\url{https://www.secg.org/sec1-v2.pdf}\\
+	\phantomsection\label{intro:GRE}\textbf{[GRE]} &
+	Generic Routing Encapsulation
+	\newline\url{https://datatracker.ietf.org/doc/rfc2784/}\\
+	\phantomsection\label{intro:VXLAN}\textbf{[VXLAN]} &
+	Virtual eXtensible Local Area Network
+	\newline\url{https://datatracker.ietf.org/doc/rfc7348/}\\
+	\phantomsection\label{intro:GENEVE}\textbf{[GENEVE]} &
+	Generic Network Virtualization Encapsulation
+	\newline\url{https://datatracker.ietf.org/doc/rfc8926/}\\
+	\phantomsection\label{intro:IP}\textbf{[IP]} &
+	INTERNET PROTOCOL
+	\newline\url{https://www.rfc-editor.org/rfc/rfc791}\\
+	\phantomsection\label{intro:UDP}\textbf{[UDP]} &
+	User Datagram Protocol
+	\newline\url{https://www.rfc-editor.org/rfc/rfc768}\\
+	\phantomsection\label{intro:TCP}\textbf{[TCP]} &
+	TRANSMISSION CONTROL PROTOCOL
+	\newline\url{https://www.rfc-editor.org/rfc/rfc793}\\
+
 \end{longtable}
Signature · 1 line
2.19.1.6.gb485710b

This publicly archived list offers a means to provide input to the OASIS Virtual I/O Device (VIRTIO) TC.

In order to verify user consent to the Feedback License terms and to minimize spam in the list archive, subscription is required before posting.

Quoted 8 lines
Quoted 8 lines

Subscribe: [email protected]
Unsubscribe: [email protected]
List help: [email protected]
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/

---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]

author
#26 lore raw
Quoted 184 lines · Heng Qi
Quoted 183 lines · Jason Wang
Quoted 168 lines · Heng Qi
Quoted 75 lines · Michael S. Tsirkin
Quoted 74 lines · Heng Qi

If the tunnel is used to encapsulate the packets, the hash calculated using the outer header of the receive packets is always fixed for the same flow packets, i.e. they will be steered to the same receive queue.

We add a feature bit VIRTIO_NET_F_HASH_TUNNEL and related bitmasks in \field{hash_types}, which instructs the device to calculate the hash using the inner headers of tunnel-encapsulated packets. Besides, values in \field{hash_report_tunnel} are added to report tunnel types.

---
v6:
1. Modify the wording of some sentences for clarity. @Michael S. Tsirkin
2. Fix some syntax issues. @Michael S. Tsirkin

v5:
1. Fix some syntax and capitalization issues. @Michael S. Tsirkin
2. Use encapsulated/encaptulation uniformly. @Michael S. Tsirkin
3. Move the links to introduction section. @Michael S. Tsirkin
4. Clarify some sentences. @Michael S. Tsirkin

v4:
1. Clarify some paragraphs. @Cornelia Huck
2. Fix the u8 type. @Cornelia Huck

v3:
1. Rename VIRTIO_NET_F_HASH_GRE_VXLAN_GENEVE_INNER to VIRTIO_NET_F_HASH_TUNNEL. @Jason Wang
2. Make things clearer. @Jason Wang @Michael S. Tsirkin
3. Keep the possibility to use inner hash for automatic receive steering. @Jason Wang
4. Add the "Tunnel packet" paragraph to avoid repeating the GRE etc. many times. @Michael S. Tsirkin

v2:
1. Add a feature bit for GRE/VXLAN/GENEVE inner hash. @Jason Wang
2. Chang \field{hash_tunnel} to \field{hash_report_tunnel}. @Jason Wang, @Michael S. Tsirkin

v1:
1. Remove the patch for the bitmask fix. @Michael S. Tsirkin
2. Clarify some paragraphs. @Jason Wang
3. Add \field{hash_tunnel} and VIRTIO_NET_HASH_REPORT_GRE. @Yuri Benditovich

  content.tex      | 191 +++++++++++++++++++++++++++++++++++++++++++++--
  introduction.tex |  19 +++++
  2 files changed, 203 insertions(+), 7 deletions(-)
content.tex +6 -1
index e863709..7845f6c 100644
@@ -3084,6 +3084,9 @@ \subsection{Feature bits}\label{sec:Device Types / Network Device / Feature bits
  \item[VIRTIO_NET_F_CTRL_MAC_ADDR(23)] Set MAC address through control
      channel.
+\item[VIRTIO_NET_F_HASH_TUNNEL(52)] Device supports inner
+    header hash for GRE, VXLAN and GENEVE tunnel-encapsulated packets.
+
  \item[VIRTIO_NET_F_NOTF_COAL(53)] Device supports notifications coalescing.
  \item[VIRTIO_NET_F_GUEST_USO4 (54)] Driver can receive USOv4 packets.
@@ -3095,7 +3098,8 @@ \subsection{Feature bits}\label{sec:Device Types / Network Device / Feature bits
   to several segments when each of these smaller packets has UDP header.
  \item[VIRTIO_NET_F_HASH_REPORT(57)] Device can report per-packet hash
-    value and a type of calculated hash.
+    value, a type of calculated hash, and, if VIRTIO_NET_F_HASH_TUNNEL
+    is negotiated, an encapsulation packet type.
  \item[VIRTIO_NET_F_GUEST_HDRLEN(59)] Driver can provide the exact \field{hdr_len}
      value. Device benefits from knowing the exact header length.
@@ -3140,6 +3144,7 @@ \subsubsection{Feature bit requirements}\label{sec:Device Types / Network Device
  \item[VIRTIO_NET_F_NOTF_COAL] Requires VIRTIO_NET_F_CTRL_VQ.
  \item[VIRTIO_NET_F_RSC_EXT] Requires VIRTIO_NET_F_HOST_TSO4 or VIRTIO_NET_F_HOST_TSO6.
  \item[VIRTIO_NET_F_RSS] Requires VIRTIO_NET_F_CTRL_VQ.
+\item[VIRTIO_NET_F_HASH_TUNNEL] Requires VIRTIO_NET_F_CTRL_VQ.
  \end{description}
Quoted 8 lines

content.tex \subsection{Device Operation}\label{sec:Device Types / Network Device / Device O · line 3394

Quoted 7 lines
content.tex +1 -1
  \subsubsection{Legacy Interface: Feature bits}\label{sec:Device Types / Network Device / Feature bits / Legacy Interface: Feature bits}
@@ -3386,7 +3391,8 @@ \subsection{Device Operation}\label{sec:Device Types / Network Device / Device O
          le16 num_buffers;
          le32 hash_value;        (Only if VIRTIO_NET_F_HASH_REPORT negotiated)
          le16 hash_report;       (Only if VIRTIO_NET_F_HASH_REPORT negotiated)
-        le16 padding_reserved;  (Only if VIRTIO_NET_F_HASH_REPORT negotiated)
+        u8 hash_report_tunnel;  (Only if VIRTIO_NET_F_HASH_REPORT negotiated, only valid of VIRTIO_NET_F_HASH_TUNNEL negotiated)

of -> if?

Sorry for the late reply.
It's Cornelia's suggestion, and 'of' seems to work fine.

Quoted 1 line

also let's add: otherwise reserved?

Sure.

Quoted 34 lines
Quoted 34 lines
content.tex +22 -1
+        u8 padding_reserved;    (Only if VIRTIO_NET_F_HASH_REPORT negotiated)
  };
  \end{lstlisting}
@@ -3837,7 +3843,8 @@ \subsubsection{Processing of Incoming Packets}\label{sec:Device Types / Network
  A device attempts to calculate a per-packet hash in the following cases:
  \begin{itemize}
  \item The feature VIRTIO_NET_F_RSS was negotiated. The device uses the hash to determine the receive virtqueue to place incoming packets.
-\item The feature VIRTIO_NET_F_HASH_REPORT was negotiated. The device reports the hash value and the hash type with the packet.
+\item The feature VIRTIO_NET_F_HASH_REPORT was negotiated. The device reports the hash value and the hash type. If additionally
+VIRTIO_NET_F_HASH_TUNNEL was negotiated, the device reports the encapsulation type as well.
  \end{itemize}
  If the feature VIRTIO_NET_F_RSS was negotiated:
@@ -3863,8 +3870,36 @@ \subsubsection{Processing of Incoming Packets}\label{sec:Device Types / Network
   \ref{sec:Device Types / Network Device / Device Operation / Control Virtqueue / Automatic receive steering in multiqueue mode / Hash calculation}.
  \end{itemize}
+\subparagraph{Tunnel/Encapsulated packet}
+\label{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / Tunnel/Encapsulated packet}
+A tunnel packet is encapsulated from the original packet based on the tunneling
+protocol (only a single level of encapsulation is currently supported). The
+encapsulated packet contains an outer header and an inner header, and the device
+calculates the hash over either the inner header or the outer header.
+
+When the feature VIRTIO_NET_F_HASH_TUNNEL is negotiated and the corresponding
+encapsulation type is set in \field{hash_types}, the hash for a specific type of
+encapsulated packet is calculated over the inner as opposed to outer header.
+Supported encapsulation types are listed in \ref{sec:Device Types / Network Device /
+Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets /
+Supported/enabled hash types}.
+
+If both VIRTIO_NET_F_HASH_REPORT and VIRTIO_NET_F_HASH_TUNNEL are negotiated,
+the device can support inner hash calculation for \hyperref[intro:GRE]{[GRE]},
+\hyperref[intro:VXLAN]{[VXLAN]} and \hyperref[intro:GENEVE]{[GENEVE]}
+encapsulated packets, and the corresponding encapsulation type in \field{hash_types}
+is VIRTIO_NET_HASH_TYPE_{GRE, VXLAN, GENEVE}_INNER respectively. The value in
Quoted 9 lines

content.tex \subsubsection{Processing of Incoming Packets}\label{sec:Device Types / Network · line 3897

Quoted 6 lines
content.tex +6 -0
+\field{hash_report_tunnel} is VIRTIO_NET_HASH_REPORT_{NONE, GRE, VXLAN, GENEVE} respectively.
+
+If VIRTIO_NET_F_HASH_REPORT is negotiated but VIRTIO_NET_F_HASH_TUNNEL is not
+negotiated, the device calculates the hash over the outer header, and
+\field{hash_report} reports the hash type. \field{hash_report_tunnel}
+is no longer valid.

Does this mean that VIRTIO_NET_HASH_TYPE_{GRE, VXLAN, GENEVE}_INNER must not be set without VIRTIO_NET_F_HASH_TUNNEL?

Yes.

Quoted 4 lines

If yes then we can add conformance statements to this end and then drop the
"If the feature VIRTIO_NET_F_HASH_TUNNEL is negotiated" all over the place just relying on hash type being set instead.

Please see the response below.

Quoted 7 lines

Also, note that it has to be ok for device to expose hash types without negotiating VIRTIO_NET_F_HASH_TUNNEL since they are in config space.
Idea: do we want to decouple these completely? have VIRTIO_NET_F_HASH_TUNNEL just add hash_report_tunnel and have hash types speak for themselves? It makes things nicely orthogonal in that if one does not care about knowing encapsulation type (e.g. for RSS) one can disable VIRTIO_NET_F_HASH_TUNNEL.

This seems to have the following 3 problems:

1. This will break the specification. As described in the specification, any field
in the configuration space only exists when there is a corresponding feature bit:
"Device configuration space is generally used for rarely-changing or initialization-time parameters. Where configuration fields are optional, their existence is indicated by feature bits: ", which also applies to supported_hash_types.

2. If I'm not wrong, this seems to make the encapsulation hash types dependent on VIRTIO_NET_F_RSS, can the driver read supported_hash_types (including encapsulation hash types) if VIRTIO_NET_F_RSS is not negotiated? But that shouldn't be the case, we only use encapsulation hash types to provide guidance for hash calculation, but not necessarily use hash values to select receive queues (unless VIRTIO_NET_F_RSS is negotiated).

3. If all supported_hash_types speak for themselves, why do we need the VIRTIO_NET_F_HASH_TUNNEL feature, because as long as the encapsulation hash types exist and VIRTIO_NET_F_HASH_REPORT is negotiated, hash_report_tunnel exists correspondingly. "have VIRTIO_NET_F_HASH_TUNNEL just add hash_report_tunnel"? It seems like hash_report_tunnel will only work if the encapsulation hash types are present.

Quoted 6 lines

So, if I'm not wrong, the reasonable logic should be that VIRTIO_NET_F_HASH_TUNNEL is only used to declare that the device supports inner header hash and let the corresponding encapsulation hash type exist, when VIRTIO_NET_F_RSS is negotiated, we can use this hash value to select the receive queue. Considering the migration at the same time, if the dst device does not negotiate VIRTIO_NET_F_HASH_TUNNEL, the migration will fail; otherwise, use encapsulation hash types to determine whether the migration can succeed.

I think it might be better to have consistency:

1) If VIRTIO_NET_F_HASH_TUNNEL introduces a new field in vnet header, is it better to have a new config filed in the config space?

or

Quoted 8 lines
Quoted 4 lines

2) If VIRTIO_NET_F_HASH_TUNNEL doesn't introduce new config file, should we try to reuse hash_report?

1) seems better and cleaner to me.

Sorry for the late reply due to vacation.

Good point, it's clearer to use the VIRTIO_NET_F_HASH_TUNNEL feature to control all information related to inner header hash.

Hi, Parav.

Do you think we need both hash_types and hash_tunnel_types?

Thanks.

Quoted 253 lines
Quoted 38 lines

More below

Quoted 23 lines

Thanks.

Quoted 21 lines

Objections? Jason?

Quoted 18 lines
content.tex +10 -0
+
  \subparagraph{Supported/enabled hash types}
  \label{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / Supported/enabled hash types}
+This paragraph relies on definitions from \hyperref[intro:IP]{[IP]},
+\hyperref[intro:UDP]{[UDP]} and \hyperref[intro:TCP]{[TCP]}.
  Hash types applicable for IPv4 packets:
  \begin{lstlisting}
  #define VIRTIO_NET_HASH_TYPE_IPv4              (1 << 0)
@@ -3884,6 +3919,22 @@ \subsubsection{Processing of Incoming Packets}\label{sec:Device Types / Network
  #define VIRTIO_NET_HASH_TYPE_UDP_EX            (1 << 8)
  \end{lstlisting}
+If the feature VIRTIO_NET_F_HASH_TUNNEL is negotiated, the encapsulation
+hash type below indicates that the hash is calculated over the inner
+header of the encapsulated packet:
+Hash type applicable for inner payload of the gre-encapsulated packet
+\begin{lstlisting}
+#define VIRTIO_NET_HASH_TYPE_GRE_INNER         (1 << 9)
+\end{lstlisting}
Quoted 8 lines
Quoted 8 lines

content.tex \subsubsection{Processing of Incoming Packets}\label{sec:Device Types / Network · line 3936

Quoted 8 lines
content.tex +8 -0
+Hash type applicable for inner payload of the vxlan-encapsulated packet
+\begin{lstlisting}
+#define VIRTIO_NET_HASH_TYPE_VXLAN_INNER       (1 << 10)
+\end{lstlisting}
+Hash type applicable for inner payload of the geneve-encapsulated packet
+\begin{lstlisting}
+#define VIRTIO_NET_HASH_TYPE_GENEVE_INNER      (1 << 11)
+\end{lstlisting}

Just notice that this seems to break the equation:

Quoted 4 lines

VIRTIO_NET_HASH_TYPE_XXX = 1 « (VIRTIO_NET_HASH_REPORT_XXX - 1)

?

Yes, we have discussed it here https://lists.oasis-open.org/archives/virtio-dev/202211/msg00189.html , as you suggested above, it seems better to use the new config space field.

Thanks.

Quoted 200 lines

Thanks

Quoted 188 lines
Quoted 188 lines
Quoted 181 lines
content.tex +131 -4
+
  \subparagraph{IPv4 packets}
  \label{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / IPv4 packets}
  The device calculates the hash on IPv4 packets according to 'Enabled hash types' bitmask as follows:
@@ -3975,15 +4026,125 @@ \subsubsection{Processing of Incoming Packets}\label{sec:Device Types / Network
  (see \ref{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / IPv6 packets without extension header}).
  \end{itemize}
+\subparagraph{Inner payload of an encapsulated packet}
+\label{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / Inner payload of the encapsulated packet}
+If the feature VIRTIO_NET_F_HASH_TUNNEL is negotiated and the corresponding
+encapsulation hash type is set in \field{hash_types}, the device calculates the
+inner header hash on an encapsulated packet (See \ref{sec:Device Types
+/ Network Device / Device Operation / Processing of Incoming Packets /
+Hash calculation for incoming packets / Tunnel/Encapsulated packet}) as follows:
+
+The device calculates the hash on the inner IPv4 packet of an encapsulated packet
+according to 'Enabled hash types' bitmask as follows:
+\begin{itemize}
+  \item If VIRTIO_NET_HASH_TYPE_TCPv4 is set and the encapsulated packet has an inner
+	TCPv4 header, the hash is calculated over the following fields:
+        \begin{itemsize}
+          \item inner Source IP address
+          \item inner Destination IP address
+          \item inner Source TCP port
+          \item inner Destination TCP port
+        \end{itemsize}
+  \item Else if VIRTIO_NET_HASH_TYPE_UDPv4 is set and the encapsulated packet has an
+	inner UDPv4 header, the hash is calculated over the following fields:
+        \begin{itemsize}
+          \item inner Source IP address
+          \item inner Destination IP address
+          \item inner Source UDP port
+          \item inner Destination UDP port
+        \end{itemize}
+  \item Else if VIRTIO_NET_HASH_TYPE_IPv4 is set, the hash is calculated over the
+	following fields:
+        \begin{itemsize}
+          \item inner Source IP address
+          \item inner Destination IP address
+        \end{itemsize}
+  \item Else the device does not calculate the hash
+\end{itemize}
+
+The device calculates the hash on the inner IPv6 packet without an extension header
+of an encapsulated packet according to 'Enabled hash types' bitmask as follows:
+\begin{itemize}
+  \item If VIRTIO_NET_HASH_TYPE_TCPv6 is set and the encapsulated packet has an inner
+	TCPv6 header, the hash is calculated over the following fields:
+        \begin{itemsize}
+          \item inner Source IPv6 address
+          \item inner Destination IPv6 address
+          \item inner Source TCP port
+          \item inner Destination TCP port
+        \end{itemsize}
+  \item Else if VIRTIO_NET_HASH_TYPE_UDPv6 is set and the encapsulated packet has an
+	inner UDPv6 header, the hash is calculated over the following fields:
+        \begin{itemsize}
+          \item inner Source IPv6 address
+          \item inner Destination IPv6 address
+          \item inner Source UDP port
+          \item inner Destination UDP port
+        \end{itemize}
+  \item Else if VIRTIO_NET_HASH_TYPE_IPv6 is set, the hash is calculated over the
+	following fields:
+        \begin{itemsize}
+          \item inner Source IPv6 address
+          \item inner Destination IPv6 address
+        \end{itemsize}
+  \item Else the device does not calculate the hash
+\end{itemize}
+
+The device calculates the hash on the inner IPv6 packet with an extension header
+of an encapsulated packet according to 'Enabled hash types' bitmask as follows:
+\begin{itemsize}
+  \item If VIRTIO_NET_HASH_TYPE_TCP_EX is set and the encapsulated packet has an inner
+	TCPv6 header, the hash is calculated over the following fields:
+        \begin{itemize}
+          \item Home address from the home address option in the inner IPv6 destination
+              options header. If the inner extension header is not present, use the
+              inner Source IPv6 address.
+          \item IPv6 address that is contained in the Routing-Header-Type-2 from the
+              associated inner extension header. If the inner extension header is not
+              present, use the inner Destination IPv6 address.
+          \item inner Source TCP port
+          \item inner Destination TCP port
+        \end{itemize}
+  \item Else if VIRTIO_NET_HASH_TYPE_UDP_EX is set and the encapsulated packet has an inner
+	UDPv6 header, the hash is calculated over the following fields:
+        \begin{itemsize}
+          \item Home address from the home address option in the inner IPv6 destination
+              options header. If the inner extension header is not present, use the
+              inner Source IPv6 address.
+          \item IPv6 address that is contained in the Routing-Header-Type-2 from the
+              associated inner extension header. If the inner extension header is not
+              present, use the inner Destination IPv6 address.
+          \item inner Source UDP port
+          \item inner Destination UDP port
+        \end{itemize}
+  \item Else if VIRTIO_NET_HASH_TYPE_IP_EX is set, the hash is calculated over the
+	following fields:
+        \begin{itemsize}
+          \item Home address from the home address option in the inner IPv6 destination
+              options header. If the inner extension header is not present, use the
+              inner Source IPv6 address.
+          \item IPv6 address that is contained in the Routing-Header-Type-2 from the
+              associated inner extension header. If the inner extension header is not
+              present, use the inner Destination IPv6 address.
+        \end{itemize}
+  \item Else skip the inner IPv6 extension header and calculate the inner header hash as
+	defined for an encapsulated packet whose inner payload is an IPv6 packet without
+	an extension header.
+\end{itemsize}
+
  \paragraph{Hash reporting for incoming packets}
  \label{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash reporting for incoming packets}
-If VIRTIO_NET_F_HASH_REPORT was negotiated and
- the device has calculated the hash for the packet, the device fills \field{hash_report} with the report type of calculated hash
-and \field{hash_value} with the value of calculated hash.
+If VIRTIO_NET_F_HASH_REPORT was negotiated and the device has calculated the
+hash for the packet, the device fills \field{hash_report} with the report type
+of calculated hash, and \field{hash_value} with the value of calculated hash.
+Also, if VIRTIO_NET_F_HASH_TUNNEL was negotiated, the device needs to fill
+\field{hash_report_tunnel} with the report type of the encapsulated packet, and
+it is set to VIRTIO_NET_HASH_REPORT_TUNNEL_NONE for the unencapsulated packet.
  If VIRTIO_NET_F_HASH_REPORT was negotiated but due to any reason the
-hash was not calculated, the device sets \field{hash_report} to VIRTIO_NET_HASH_REPORT_NONE.
+hash was not calculated, the device sets \field{hash_report} to VIRTIO_NET_HASH_REPORT_NONE,
+and sets \field{hash_report_tunnel} to VIRTIO_NET_HASH_REPORT_TUNNEL_NONE.
  Possible values that the device can report in \field{hash_report} are defined below.
  They correspond to supported hash types defined in
@@ -4005,6 +4166,22 @@ \subsubsection{Processing of Incoming Packets}\label{sec:Device Types / Network
  #define VIRTIO_NET_HASH_REPORT_UDPv6_EX        9
  \end{lstlisting}
+If \field{hash_report} differs from VIRTIO_NET_HASH_REPORT_NONE,
+\field{hash_report_tunnel} can report the type of the encapsulated
+packet to the driver over the inner header hash calculation.
+Possible values that the device can report in \field{hash_report_tunnel}
+are defined below:
+
+\begin{lstlisting}
+#define VIRTIO_NET_HASH_REPORT_TUNNEL_NONE     0
+#define VIRTIO_NET_HASH_REPORT_GRE             1
+#define VIRTIO_NET_HASH_REPORT_VXLAN           2
+#define VIRTIO_NET_HASH_REPORT_GENEVE          3
+\end{lstlisting}
+
+They correspond to supported hash types defined in
+\ref{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / Supported/enabled hash types}.
+
  \subsubsection{Control Virtqueue}\label{sec:Device Types / Network Device / Device Operation / Control Virtqueue}
  The driver uses the control virtqueue (if VIRTIO_NET_F_CTRL_VQ is
introduction.tex +19 -0
index 287c5fc..ff01a9b 100644
@@ -98,6 +98,25 @@ \section{Normative References}\label{sec:Normative References}
  	\phantomsection\label{intro:SEC1}\textbf{[SEC1]} &
      Standards for Efficient Cryptography Group(SECG), ``SEC1: Elliptic Cureve Cryptography'', Version 1.0, September 2000.
  	\newline\url{https://www.secg.org/sec1-v2.pdf}\\
+	\phantomsection\label{intro:GRE}\textbf{[GRE]} &
+	Generic Routing Encapsulation
+	\newline\url{https://datatracker.ietf.org/doc/rfc2784/}\\
+	\phantomsection\label{intro:VXLAN}\textbf{[VXLAN]} &
+	Virtual eXtensible Local Area Network
+	\newline\url{https://datatracker.ietf.org/doc/rfc7348/}\\
+	\phantomsection\label{intro:GENEVE}\textbf{[GENEVE]} &
+	Generic Network Virtualization Encapsulation
+	\newline\url{https://datatracker.ietf.org/doc/rfc8926/}\\
+	\phantomsection\label{intro:IP}\textbf{[IP]} &
+	INTERNET PROTOCOL
+	\newline\url{https://www.rfc-editor.org/rfc/rfc791}\\
+	\phantomsection\label{intro:UDP}\textbf{[UDP]} &
+	User Datagram Protocol
+	\newline\url{https://www.rfc-editor.org/rfc/rfc768}\\
+	\phantomsection\label{intro:TCP}\textbf{[TCP]} &
+	TRANSMISSION CONTROL PROTOCOL
+	\newline\url{https://www.rfc-editor.org/rfc/rfc793}\\
+
  \end{longtable}
Signature · 1 line
2.19.1.6.gb485710b

This publicly archived list offers a means to provide input to the OASIS Virtual I/O Device (VIRTIO) TC.

In order to verify user consent to the Feedback License terms and to minimize spam in the list archive, subscription is required before posting.

Quoted 8 lines
Quoted 8 lines

Subscribe: [email protected]
Unsubscribe: [email protected]
List help: [email protected]
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/

Quoted 3 lines

---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]

---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]

#27 lore raw
Quoted 272 lines

From: Heng Qi <[email protected]>
Sent: Tuesday, February 7, 2023 9:31 PM

Quoted 7 lines · Heng Qi
Quoted 6 lines · Jason Wang
Quoted 5 lines · Heng Qi
Quoted 4 lines · Michael S. Tsirkin
Quoted 3 lines · Heng Qi

If the tunnel is used to encapsulate the packets, the hash calculated using the outer header of the receive packets is always fixed for the same flow packets, i.e. they will be steered to the same

receive queue.

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

We add a feature bit VIRTIO_NET_F_HASH_TUNNEL and related

bitmasks

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

in \field{hash_types}, which instructs the device to calculate the hash using the inner headers of tunnel-encapsulated packets. Besides, values in \field{hash_report_tunnel} are added to report tunnel

types.

Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines

Fixes:
https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2F
github.com%2Foasis-tcs%2Fvirtio-

spec%2Fissues%2F151&data=05%7C01%7

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

Cparav%40nvidia.com%7C12be6c51b3c04860c01608db097c796a%7C43083d15 7

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

27340c1b7db39efd9ccc17a%7C0%7C0%7C638114202991221369%7CUnknown %7CT

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

WFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJ

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

XVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=6feFgy94LMCYfG0NKPlHCr1AvkC OH4v9

Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines

%2BnLomXklNqs%3D&reserved=0

---
v6:
1. Modify the wording of some sentences for clarity. @Michael S.

Tsirkin

Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines

2. Fix some syntax issues. @Michael S. Tsirkin

v5:
1. Fix some syntax and capitalization issues. @Michael S. Tsirkin
2. Use encapsulated/encaptulation uniformly. @Michael S. Tsirkin
3. Move the links to introduction section. @Michael S. Tsirkin
4. Clarify some sentences. @Michael S. Tsirkin

Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines

v4:
1. Clarify some paragraphs. @Cornelia Huck
2. Fix the u8 type. @Cornelia Huck

v3:
1. Rename VIRTIO_NET_F_HASH_GRE_VXLAN_GENEVE_INNER to

VIRTIO_NET_F_HASH_TUNNEL. @Jason Wang

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

2. Make things clearer. @Jason Wang @Michael S. Tsirkin
3. Keep the possibility to use inner hash for automatic receive steering.

@Jason Wang

Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines

4. Add the "Tunnel packet" paragraph to avoid repeating the GRE
etc. many times. @Michael S. Tsirkin

v2:
1. Add a feature bit for GRE/VXLAN/GENEVE inner hash. @Jason Wang
2. Chang \field{hash_tunnel} to \field{hash_report_tunnel}.
@Jason Wang, @Michael S. Tsirkin

Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines

v1:
1. Remove the patch for the bitmask fix. @Michael S. Tsirkin
2. Clarify some paragraphs. @Jason Wang
3. Add \field{hash_tunnel} and VIRTIO_NET_HASH_REPORT_GRE.

@Yuri

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

Benditovich

  content.tex      | 191

+++++++++++++++++++++++++++++++++++++++++++++--

Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines

content.tex \subsection{Feature bits}\label{sec:Device

Quoted 8 lines
  introduction.tex |  19 +++++
  2 files changed, 203 insertions(+), 7 deletions(-)
content.tex index e863709..7845f6c renamed from content.tex +0 -0

100644

content.tex +0 -0
@@ -3084,6 +3084,9 @@ \subsection{Feature bits}\label{sec:Device

Types / Network Device / Feature bits

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

content.tex \subsection{Feature bits}\label{sec:Device · line 3084

Quoted 1 line
content.tex +0 -0
  \item[VIRTIO_NET_F_CTRL_MAC_ADDR(23)] Set MAC address through

control

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

content.tex \subsection{Feature bits}\label{sec:Device Types / Network Device / Feature bits

Quoted 3 lines
content.tex +2 -0
      channel.
+\item[VIRTIO_NET_F_HASH_TUNNEL(52)] Device supports inner
+    header hash for GRE, VXLAN and GENEVE tunnel-encapsulated

packets.

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

content.tex \subsection{Feature bits}\label{sec:Device · line 3089

Quoted 2 lines
content.tex +1 -0
+
  \item[VIRTIO_NET_F_NOTF_COAL(53)] Device supports notifications

coalescing.

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

content.tex \subsection{Feature bits}\label{sec:Device · line 3090

Quoted 1 line
content.tex +0 -0
  \item[VIRTIO_NET_F_GUEST_USO4 (54)] Driver can receive USOv4

packets.

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

content.tex \subsection{Feature bits}\label{sec:Device

Quoted 1 line
content.tex +0 -0
@@ -3095,7 +3098,8 @@ \subsection{Feature bits}\label{sec:Device

Types / Network Device / Feature bits

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

content.tex \subsection{Feature bits}\label{sec:Device · line 3098

Quoted 1 line
content.tex +0 -0
   to several segments when each of these smaller packets has UDP

header.

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

content.tex \subsection{Feature bits}\label{sec:Device · line 3099

Quoted 1 line
content.tex +0 -0
  \item[VIRTIO_NET_F_HASH_REPORT(57)] Device can report per-packet

hash

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

content.tex \subsection{Feature bits}\label{sec:Device · line 3100

Quoted 2 lines
content.tex +1 -1
-    value and a type of calculated hash.
+    value, a type of calculated hash, and, if

VIRTIO_NET_F_HASH_TUNNEL

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

content.tex \subsection{Feature bits}\label{sec:Device · line 3102

Quoted 2 lines
content.tex +1 -0
+    is negotiated, an encapsulation packet type.
  \item[VIRTIO_NET_F_GUEST_HDRLEN(59)] Driver can provide the exact

\field{hdr_len}

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

content.tex \subsubsection{Feature bit

Quoted 2 lines
content.tex +0 -0
      value. Device benefits from knowing the exact header length.
@@ -3140,6 +3144,7 @@ \subsubsection{Feature bit

requirements}\label{sec:Device Types / Network Device

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

content.tex \subsubsection{Feature bit · line 3145

Quoted 2 lines
content.tex +0 -0
  \item[VIRTIO_NET_F_NOTF_COAL] Requires VIRTIO_NET_F_CTRL_VQ.
  \item[VIRTIO_NET_F_RSC_EXT] Requires VIRTIO_NET_F_HOST_TSO4 or

VIRTIO_NET_F_HOST_TSO6.

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

content.tex \subsubsection{Feature bit · line 3147

Quoted 2 lines
content.tex +1 -0
  \item[VIRTIO_NET_F_RSS] Requires VIRTIO_NET_F_CTRL_VQ.
+\item[VIRTIO_NET_F_HASH_TUNNEL] Requires

VIRTIO_NET_F_CTRL_VQ.

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
content.tex +0 -0
  \end{description}
  \subsubsection{Legacy Interface: Feature bits}\label{sec:Device
 Types / Network Device / Feature bits / Legacy Interface: Feature bits}
(quoted diff) +0 -0
@@ -3386,7 +3391,8 @@ \subsection{Device Operation}\label{sec:Device

Types / Network Device / Device O

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

le16 num_buffers;
le32 hash_value; (Only if VIRTIO_NET_F_HASH_REPORT

negotiated)

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

le16 hash_report; (Only if VIRTIO_NET_F_HASH_REPORT

negotiated)

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

- le16 padding_reserved; (Only if VIRTIO_NET_F_HASH_REPORT

negotiated)

Quoted 10 lines
Quoted 10 lines
Quoted 10 lines
Quoted 3 lines
Quoted 2 lines

+ u8 hash_report_tunnel; (Only if VIRTIO_NET_F_HASH_REPORT
+ negotiated, only valid of VIRTIO_NET_F_HASH_TUNNEL negotiated)

of -> if?

Sorry for the late reply.
It's Cornelia's suggestion, and 'of' seems to work fine.

Quoted 1 line

also let's add: otherwise reserved?

Sure.

Quoted 1 line
Quoted 1 line

+ u8 padding_reserved; (Only if VIRTIO_NET_F_HASH_REPORT

negotiated)

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

content.tex \subsubsection{Processing of Incoming

Quoted 3 lines
  };
  \end{lstlisting}
content.tex +0 -0
@@ -3837,7 +3843,8 @@ \subsubsection{Processing of Incoming

Packets}\label{sec:Device Types / Network

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

content.tex \subsubsection{Processing of Incoming Packets}\label{sec:Device Types / Network

Quoted 3 lines
content.tex +0 -0
  A device attempts to calculate a per-packet hash in the following cases:
  \begin{itemize}
  \item The feature VIRTIO_NET_F_RSS was negotiated. The device uses

the hash to determine the receive virtqueue to place incoming packets.

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

content.tex \subsubsection{Processing of Incoming · line 3840

Quoted 1 line
content.tex +0 -1
-\item The feature VIRTIO_NET_F_HASH_REPORT was negotiated. The

device reports the hash value and the hash type with the packet.

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

content.tex \subsubsection{Processing of Incoming · line 3847

Quoted 2 lines
content.tex +2 -0
+\item The feature VIRTIO_NET_F_HASH_REPORT was negotiated. The
+device reports the hash value and the hash type. If additionally

VIRTIO_NET_F_HASH_TUNNEL was negotiated, the device reports the encapsulation type as well.

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

content.tex \subsubsection{Processing of Incoming

Quoted 3 lines
content.tex +0 -0
  \end{itemize}
  If the feature VIRTIO_NET_F_RSS was negotiated:
@@ -3863,8 +3870,36 @@ \subsubsection{Processing of Incoming

Packets}\label{sec:Device Types / Network

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

content.tex \subsubsection{Processing of Incoming · line 3870

Quoted 1 line
content.tex +0 -0
   \ref{sec:Device Types / Network Device / Device Operation / Control

Virtqueue / Automatic receive steering in multiqueue mode / Hash calculation}.

Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines

content.tex \subsubsection{Processing of Incoming · line 3877

Quoted 7 lines
content.tex +6 -0
  \end{itemize}
+\subparagraph{Tunnel/Encapsulated packet} \label{sec:Device Types
+/ Network Device / Device Operation / Processing of Incoming
+Packets / Hash calculation for incoming packets /
+Tunnel/Encapsulated packet} A tunnel packet is encapsulated from
+the original packet based on the tunneling protocol (only a
+single level of encapsulation is currently supported). The encapsulated

packet contains an outer header and an inner header, and the device calculates the hash over either the inner header or the outer header.

Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines

content.tex \subsubsection{Processing of Incoming · line 3881

Quoted 4 lines
content.tex +4 -0
+
+When the feature VIRTIO_NET_F_HASH_TUNNEL is negotiated and the
+corresponding encapsulation type is set in \field{hash_types},
+the hash for a specific type of encapsulated packet is calculated over

the inner as opposed to outer header.

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

content.tex \subsubsection{Processing of Incoming · line 3884

Quoted 3 lines
content.tex +3 -0
+Supported encapsulation types are listed in \ref{sec:Device Types
+/ Network Device / Device Operation / Processing of Incoming
+Packets / Hash calculation for incoming packets / Supported/enabled

hash types}.

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

content.tex \subsubsection{Processing of Incoming · line 3886

Quoted 2 lines
content.tex +2 -0
+
+If both VIRTIO_NET_F_HASH_REPORT and

VIRTIO_NET_F_HASH_TUNNEL are

Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines

content.tex \subsubsection{Processing of Incoming · line 3891

Quoted 5 lines
content.tex +5 -0
+negotiated, the device can support inner hash calculation for
+\hyperref[intro:GRE]{[GRE]}, \hyperref[intro:VXLAN]{[VXLAN]} and
+\hyperref[intro:GENEVE]{[GENEVE]} encapsulated packets, and the
+corresponding encapsulation type in \field{hash_types} is
+VIRTIO_NET_HASH_TYPE_{GRE, VXLAN, GENEVE}_INNER respectively.

The value in \field{hash_report_tunnel} is VIRTIO_NET_HASH_REPORT_{NONE, GRE, VXLAN, GENEVE} respectively.

Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines

content.tex \subsubsection{Processing of Incoming · line 3896

Quoted 5 lines
content.tex +5 -0
+
+If VIRTIO_NET_F_HASH_REPORT is negotiated but
+VIRTIO_NET_F_HASH_TUNNEL is not negotiated, the device calculates
+the hash over the outer header, and \field{hash_report} reports
+the hash type. \field{hash_report_tunnel} is no longer valid.

Does this mean that VIRTIO_NET_HASH_TYPE_{GRE, VXLAN,

GENEVE}_INNER

Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 2 lines

must not be set without VIRTIO_NET_F_HASH_TUNNEL?

Yes.

Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 3 lines

If yes then we can add conformance statements to this end and then drop the "If the feature VIRTIO_NET_F_HASH_TUNNEL is negotiated" all over the place just relying on hash type being set instead.

Please see the response below.

Quoted 3 lines

Also, note that it has to be ok for device to expose hash types without negotiating VIRTIO_NET_F_HASH_TUNNEL since they are in config space.
Idea: do we want to decouple these completely? have

VIRTIO_NET_F_HASH_TUNNEL

Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 4 lines

just add hash_report_tunnel and have hash types speak for themselves? It makes things nicely orthogonal in that if one does not care about knowing encapsulation type (e.g. for RSS) one can disable VIRTIO_NET_F_HASH_TUNNEL.

This seems to have the following 3 problems:

1. This will break the specification. As described in the specification, any

field

Quoted 1 line
Quoted 1 line
Quoted 1 line

in the configuration space only exists when there is a corresponding

feature bit:

Quoted 1 line
Quoted 1 line
Quoted 1 line

"Device configuration space is generally used for rarely-changing or

initialization-time parameters.

Quoted 1 line
Quoted 1 line
Quoted 1 line

Where configuration fields are optional, their existence is indicated by

feature bits: ",

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

which also applies to supported_hash_types.

2. If I'm not wrong, this seems to make the encapsulation hash types

dependent on

Quoted 1 line
Quoted 1 line
Quoted 1 line

VIRTIO_NET_F_RSS, can the driver read supported_hash_types

(including encapsulation

Quoted 1 line
Quoted 1 line
Quoted 1 line

hash types) if VIRTIO_NET_F_RSS is not negotiated? But that shouldn't be

the case,

Quoted 1 line
Quoted 1 line
Quoted 1 line

we only use encapsulation hash types to provide guidance for hash

calculation, but not

Quoted 1 line
Quoted 1 line
Quoted 1 line

necessarily use hash values to select receive queues (unless

VIRTIO_NET_F_RSS is negotiated).

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

3. If all supported_hash_types speak for themselves, why do we need the

VIRTIO_NET_F_HASH_TUNNEL

Quoted 1 line
Quoted 1 line
Quoted 1 line

feature, because as long as the encapsulation hash types exist and

VIRTIO_NET_F_HASH_REPORT is

Quoted 1 line
Quoted 1 line
Quoted 1 line

negotiated, hash_report_tunnel exists correspondingly. "have

VIRTIO_NET_F_HASH_TUNNEL just add

Quoted 1 line
Quoted 1 line
Quoted 1 line

hash_report_tunnel"? It seems like hash_report_tunnel will only work if

the encapsulation hash

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

types are present.

So, if I'm not wrong, the reasonable logic should be that

VIRTIO_NET_F_HASH_TUNNEL is only used

Quoted 1 line
Quoted 1 line
Quoted 1 line

to declare that the device supports inner header hash and let the

corresponding encapsulation

Quoted 1 line
Quoted 1 line
Quoted 1 line

hash type exist, when VIRTIO_NET_F_RSS is negotiated, we can use this

hash value to select the

Quoted 1 line
Quoted 1 line
Quoted 1 line

receive queue. Considering the migration at the same time, if the dst device

does not negotiate

Quoted 1 line
Quoted 1 line
Quoted 1 line

VIRTIO_NET_F_HASH_TUNNEL, the migration will fail; otherwise, use

encapsulation hash types to

Quoted 13 lines
Quoted 13 lines
Quoted 1 line

determine whether the migration can succeed.

I think it might be better to have consistency:

1) If VIRTIO_NET_F_HASH_TUNNEL introduces a new field in vnet header, is it better to have a new config filed in the config space?

or

2) If VIRTIO_NET_F_HASH_TUNNEL doesn't introduce new config file, should we try to reuse hash_report?

1) seems better and cleaner to me.

Quoted 8 lines
Quoted 4 lines

Sorry for the late reply due to vacation.

Good point, it's clearer to use the VIRTIO_NET_F_HASH_TUNNEL feature to control all information related to inner header hash.

Hi, Parav.

Do you think we need both hash_types and hash_tunnel_types?

In struct virtio_net_config we need two fields.
a. supported_hash_types (already exists)
b. supported_hash_tunnel_type -> bitmap indicating for which outer headers, inner hash calculation is supported.

In struct virtio_net_hdr we need two fields.
a. hash_report (already exists)
b. hash_tunnel_type 8 bits -> absolute value indicating which outer header exists when inner header hash calculated. You already have it in your patch named as hash_report_tunnel.
May be better to name as hash_report_tunnel_type to make it clearer that its type.

author
#28 lore raw
Quoted 265 lines · Parav Pandit
Quoted 265 lines

From: Heng Qi <[email protected]>
Sent: Tuesday, February 7, 2023 9:31 PM

Quoted 7 lines · Heng Qi
Quoted 6 lines · Jason Wang
Quoted 5 lines · Heng Qi
Quoted 4 lines · Michael S. Tsirkin
Quoted 3 lines · Heng Qi

If the tunnel is used to encapsulate the packets, the hash calculated using the outer header of the receive packets is always fixed for the same flow packets, i.e. they will be steered to the same

receive queue.

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

We add a feature bit VIRTIO_NET_F_HASH_TUNNEL and related

bitmasks

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

in \field{hash_types}, which instructs the device to calculate the hash using the inner headers of tunnel-encapsulated packets. Besides, values in \field{hash_report_tunnel} are added to report tunnel

types.

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

Fixes:
https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2F
github.com%2Foasis-tcs%2Fvirtio-

spec%2Fissues%2F151&data=05%7C01%7 Cparav%40nvidia.com%7C12be6c51b3c04860c01608db097c796a%7C43083d15 7 27340c1b7db39efd9ccc17a%7C0%7C0%7C638114202991221369%7CUnknown %7CT WFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJ XVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=6feFgy94LMCYfG0NKPlHCr1AvkC OH4v9

Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines

%2BnLomXklNqs%3D&reserved=0

---
v6:
1. Modify the wording of some sentences for clarity. @Michael S.

Tsirkin

Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines

2. Fix some syntax issues. @Michael S. Tsirkin

v5:
1. Fix some syntax and capitalization issues. @Michael S. Tsirkin
2. Use encapsulated/encaptulation uniformly. @Michael S. Tsirkin
3. Move the links to introduction section. @Michael S. Tsirkin
4. Clarify some sentences. @Michael S. Tsirkin

Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines

v4:
1. Clarify some paragraphs. @Cornelia Huck
2. Fix the u8 type. @Cornelia Huck

v3:
1. Rename VIRTIO_NET_F_HASH_GRE_VXLAN_GENEVE_INNER to

VIRTIO_NET_F_HASH_TUNNEL. @Jason Wang

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

2. Make things clearer. @Jason Wang @Michael S. Tsirkin
3. Keep the possibility to use inner hash for automatic receive steering.

@Jason Wang

Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines

4. Add the "Tunnel packet" paragraph to avoid repeating the GRE
etc. many times. @Michael S. Tsirkin

v2:
1. Add a feature bit for GRE/VXLAN/GENEVE inner hash. @Jason Wang
2. Chang \field{hash_tunnel} to \field{hash_report_tunnel}.
@Jason Wang, @Michael S. Tsirkin

Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines

v1:
1. Remove the patch for the bitmask fix. @Michael S. Tsirkin
2. Clarify some paragraphs. @Jason Wang
3. Add \field{hash_tunnel} and VIRTIO_NET_HASH_REPORT_GRE.

@Yuri

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

Benditovich

   content.tex      | 191

+++++++++++++++++++++++++++++++++++++++++++++--

Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines

content.tex \subsection{Feature bits}\label{sec:Device

Quoted 8 lines
   introduction.tex |  19 +++++
   2 files changed, 203 insertions(+), 7 deletions(-)
content.tex index e863709..7845f6c renamed from content.tex +0 -0

100644

content.tex +0 -0
@@ -3084,6 +3084,9 @@ \subsection{Feature bits}\label{sec:Device

Types / Network Device / Feature bits

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

content.tex \subsection{Feature bits}\label{sec:Device · line 3084

Quoted 1 line
content.tex +0 -0
   \item[VIRTIO_NET_F_CTRL_MAC_ADDR(23)] Set MAC address through

control

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

content.tex \subsection{Feature bits}\label{sec:Device · line 3087

Quoted 3 lines
content.tex +2 -0
       channel.
+\item[VIRTIO_NET_F_HASH_TUNNEL(52)] Device supports inner
+    header hash for GRE, VXLAN and GENEVE tunnel-encapsulated

packets.

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

content.tex \subsection{Feature bits}\label{sec:Device · line 3089

Quoted 2 lines
content.tex +1 -0
+
   \item[VIRTIO_NET_F_NOTF_COAL(53)] Device supports notifications

coalescing.

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

content.tex \subsection{Feature bits}\label{sec:Device · line 3090

Quoted 1 line
content.tex +0 -0
   \item[VIRTIO_NET_F_GUEST_USO4 (54)] Driver can receive USOv4

packets.

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

content.tex \subsection{Feature bits}\label{sec:Device

Quoted 1 line
content.tex +0 -0
@@ -3095,7 +3098,8 @@ \subsection{Feature bits}\label{sec:Device

Types / Network Device / Feature bits

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

content.tex \subsection{Feature bits}\label{sec:Device · line 3098

Quoted 1 line
content.tex +0 -0
    to several segments when each of these smaller packets has UDP

header.

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

content.tex \subsection{Feature bits}\label{sec:Device · line 3099

Quoted 1 line
content.tex +0 -0
   \item[VIRTIO_NET_F_HASH_REPORT(57)] Device can report per-packet

hash

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

content.tex \subsection{Feature bits}\label{sec:Device · line 3100

Quoted 2 lines
content.tex +1 -1
-    value and a type of calculated hash.
+    value, a type of calculated hash, and, if

VIRTIO_NET_F_HASH_TUNNEL

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

content.tex \subsection{Feature bits}\label{sec:Device · line 3102

Quoted 2 lines
content.tex +1 -0
+    is negotiated, an encapsulation packet type.
   \item[VIRTIO_NET_F_GUEST_HDRLEN(59)] Driver can provide the exact

\field{hdr_len}

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

content.tex \subsubsection{Feature bit

Quoted 2 lines
content.tex +0 -0
       value. Device benefits from knowing the exact header length.
@@ -3140,6 +3144,7 @@ \subsubsection{Feature bit

requirements}\label{sec:Device Types / Network Device

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

content.tex \subsubsection{Feature bit · line 3145

Quoted 2 lines
content.tex +0 -0
   \item[VIRTIO_NET_F_NOTF_COAL] Requires VIRTIO_NET_F_CTRL_VQ.
   \item[VIRTIO_NET_F_RSC_EXT] Requires VIRTIO_NET_F_HOST_TSO4 or

VIRTIO_NET_F_HOST_TSO6.

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

content.tex \subsubsection{Feature bit · line 3147

Quoted 2 lines
content.tex +1 -0
   \item[VIRTIO_NET_F_RSS] Requires VIRTIO_NET_F_CTRL_VQ.
+\item[VIRTIO_NET_F_HASH_TUNNEL] Requires

VIRTIO_NET_F_CTRL_VQ.

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
content.tex +0 -0
   \end{description}
   \subsubsection{Legacy Interface: Feature bits}\label{sec:Device
 Types / Network Device / Feature bits / Legacy Interface: Feature bits}
(quoted diff) +0 -0
@@ -3386,7 +3391,8 @@ \subsection{Device Operation}\label{sec:Device

Types / Network Device / Device O

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

le16 num_buffers;
le32 hash_value; (Only if VIRTIO_NET_F_HASH_REPORT

negotiated)

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

le16 hash_report; (Only if VIRTIO_NET_F_HASH_REPORT

negotiated)

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

- le16 padding_reserved; (Only if VIRTIO_NET_F_HASH_REPORT

negotiated)

Quoted 10 lines
Quoted 10 lines
Quoted 10 lines
Quoted 3 lines
Quoted 2 lines

+ u8 hash_report_tunnel; (Only if VIRTIO_NET_F_HASH_REPORT
+ negotiated, only valid of VIRTIO_NET_F_HASH_TUNNEL negotiated)

of -> if?

Sorry for the late reply.
It's Cornelia's suggestion, and 'of' seems to work fine.

Quoted 1 line

also let's add: otherwise reserved?

Sure.

Quoted 1 line
Quoted 1 line

+ u8 padding_reserved; (Only if VIRTIO_NET_F_HASH_REPORT

negotiated)

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

content.tex \subsubsection{Processing of Incoming

Quoted 3 lines
   };
   \end{lstlisting}
content.tex +0 -0
@@ -3837,7 +3843,8 @@ \subsubsection{Processing of Incoming

Packets}\label{sec:Device Types / Network

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

content.tex \subsubsection{Processing of Incoming · line 3845

Quoted 3 lines
content.tex +0 -0
   A device attempts to calculate a per-packet hash in the following cases:
   \begin{itemize}
   \item The feature VIRTIO_NET_F_RSS was negotiated. The device uses

the hash to determine the receive virtqueue to place incoming packets.

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

content.tex \subsubsection{Processing of Incoming · line 3840

Quoted 1 line
content.tex +0 -1
-\item The feature VIRTIO_NET_F_HASH_REPORT was negotiated. The

device reports the hash value and the hash type with the packet.

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

content.tex \subsubsection{Processing of Incoming · line 3847

Quoted 2 lines
content.tex +2 -0
+\item The feature VIRTIO_NET_F_HASH_REPORT was negotiated. The
+device reports the hash value and the hash type. If additionally

VIRTIO_NET_F_HASH_TUNNEL was negotiated, the device reports the encapsulation type as well.

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

content.tex \subsubsection{Processing of Incoming

Quoted 3 lines
content.tex +0 -0
   \end{itemize}
   If the feature VIRTIO_NET_F_RSS was negotiated:
@@ -3863,8 +3870,36 @@ \subsubsection{Processing of Incoming

Packets}\label{sec:Device Types / Network

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

content.tex \subsubsection{Processing of Incoming · line 3870

Quoted 1 line
content.tex +0 -0
    \ref{sec:Device Types / Network Device / Device Operation / Control

Virtqueue / Automatic receive steering in multiqueue mode / Hash calculation}.

Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines

content.tex \subsubsection{Processing of Incoming · line 3877

Quoted 7 lines
content.tex +6 -0
   \end{itemize}
+\subparagraph{Tunnel/Encapsulated packet} \label{sec:Device Types
+/ Network Device / Device Operation / Processing of Incoming
+Packets / Hash calculation for incoming packets /
+Tunnel/Encapsulated packet} A tunnel packet is encapsulated from
+the original packet based on the tunneling protocol (only a
+single level of encapsulation is currently supported). The encapsulated

packet contains an outer header and an inner header, and the device calculates the hash over either the inner header or the outer header.

Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines

content.tex \subsubsection{Processing of Incoming · line 3881

Quoted 4 lines
content.tex +4 -0
+
+When the feature VIRTIO_NET_F_HASH_TUNNEL is negotiated and the
+corresponding encapsulation type is set in \field{hash_types},
+the hash for a specific type of encapsulated packet is calculated over

the inner as opposed to outer header.

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

content.tex \subsubsection{Processing of Incoming · line 3884

Quoted 3 lines
content.tex +3 -0
+Supported encapsulation types are listed in \ref{sec:Device Types
+/ Network Device / Device Operation / Processing of Incoming
+Packets / Hash calculation for incoming packets / Supported/enabled

hash types}.

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

content.tex \subsubsection{Processing of Incoming · line 3886

Quoted 2 lines
content.tex +2 -0
+
+If both VIRTIO_NET_F_HASH_REPORT and

VIRTIO_NET_F_HASH_TUNNEL are

Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines

content.tex \subsubsection{Processing of Incoming · line 3891

Quoted 5 lines
content.tex +5 -0
+negotiated, the device can support inner hash calculation for
+\hyperref[intro:GRE]{[GRE]}, \hyperref[intro:VXLAN]{[VXLAN]} and
+\hyperref[intro:GENEVE]{[GENEVE]} encapsulated packets, and the
+corresponding encapsulation type in \field{hash_types} is
+VIRTIO_NET_HASH_TYPE_{GRE, VXLAN, GENEVE}_INNER respectively.

The value in \field{hash_report_tunnel} is VIRTIO_NET_HASH_REPORT_{NONE, GRE, VXLAN, GENEVE} respectively.

Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines

content.tex \subsubsection{Processing of Incoming · line 3896

Quoted 5 lines
content.tex +5 -0
+
+If VIRTIO_NET_F_HASH_REPORT is negotiated but
+VIRTIO_NET_F_HASH_TUNNEL is not negotiated, the device calculates
+the hash over the outer header, and \field{hash_report} reports
+the hash type. \field{hash_report_tunnel} is no longer valid.

Does this mean that VIRTIO_NET_HASH_TYPE_{GRE, VXLAN,

GENEVE}_INNER

Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 2 lines

must not be set without VIRTIO_NET_F_HASH_TUNNEL?

Yes.

Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 3 lines

If yes then we can add conformance statements to this end and then drop the "If the feature VIRTIO_NET_F_HASH_TUNNEL is negotiated" all over the place just relying on hash type being set instead.

Please see the response below.

Quoted 3 lines

Also, note that it has to be ok for device to expose hash types without negotiating VIRTIO_NET_F_HASH_TUNNEL since they are in config space.
Idea: do we want to decouple these completely? have

VIRTIO_NET_F_HASH_TUNNEL

Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 4 lines

just add hash_report_tunnel and have hash types speak for themselves? It makes things nicely orthogonal in that if one does not care about knowing encapsulation type (e.g. for RSS) one can disable VIRTIO_NET_F_HASH_TUNNEL.

This seems to have the following 3 problems:

1. This will break the specification. As described in the specification, any

field

Quoted 1 line
Quoted 1 line
Quoted 1 line

in the configuration space only exists when there is a corresponding

feature bit:

Quoted 1 line
Quoted 1 line
Quoted 1 line

"Device configuration space is generally used for rarely-changing or

initialization-time parameters.

Quoted 1 line
Quoted 1 line
Quoted 1 line

Where configuration fields are optional, their existence is indicated by

feature bits: ",

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

which also applies to supported_hash_types.

2. If I'm not wrong, this seems to make the encapsulation hash types

dependent on

Quoted 1 line
Quoted 1 line
Quoted 1 line

VIRTIO_NET_F_RSS, can the driver read supported_hash_types

(including encapsulation

Quoted 1 line
Quoted 1 line
Quoted 1 line

hash types) if VIRTIO_NET_F_RSS is not negotiated? But that shouldn't be

the case,

Quoted 1 line
Quoted 1 line
Quoted 1 line

we only use encapsulation hash types to provide guidance for hash

calculation, but not

Quoted 1 line
Quoted 1 line
Quoted 1 line

necessarily use hash values to select receive queues (unless

VIRTIO_NET_F_RSS is negotiated).

Quoted 1 line
Quoted 1 line
Quoted 1 line

3. If all supported_hash_types speak for themselves, why do we need the

VIRTIO_NET_F_HASH_TUNNEL

Quoted 1 line
Quoted 1 line
Quoted 1 line

feature, because as long as the encapsulation hash types exist and

VIRTIO_NET_F_HASH_REPORT is

Quoted 1 line
Quoted 1 line
Quoted 1 line

negotiated, hash_report_tunnel exists correspondingly. "have

VIRTIO_NET_F_HASH_TUNNEL just add

Quoted 1 line
Quoted 1 line
Quoted 1 line

hash_report_tunnel"? It seems like hash_report_tunnel will only work if

the encapsulation hash

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

types are present.

So, if I'm not wrong, the reasonable logic should be that

VIRTIO_NET_F_HASH_TUNNEL is only used

Quoted 1 line
Quoted 1 line
Quoted 1 line

to declare that the device supports inner header hash and let the

corresponding encapsulation

Quoted 1 line
Quoted 1 line
Quoted 1 line

hash type exist, when VIRTIO_NET_F_RSS is negotiated, we can use this

hash value to select the

Quoted 1 line
Quoted 1 line
Quoted 1 line

receive queue. Considering the migration at the same time, if the dst device

does not negotiate

Quoted 1 line
Quoted 1 line
Quoted 1 line

VIRTIO_NET_F_HASH_TUNNEL, the migration will fail; otherwise, use

encapsulation hash types to

Quoted 14 lines
Quoted 6 lines
Quoted 1 line

determine whether the migration can succeed.

I think it might be better to have consistency:

1) If VIRTIO_NET_F_HASH_TUNNEL introduces a new field in vnet header, is it better to have a new config filed in the config space?

Quoted 6 lines

or

2) If VIRTIO_NET_F_HASH_TUNNEL doesn't introduce new config file, should we try to reuse hash_report?

1) seems better and cleaner to me.

Sorry for the late reply due to vacation.

Quoted 8 lines
Quoted 5 lines
Quoted 2 lines

Good point, it's clearer to use the VIRTIO_NET_F_HASH_TUNNEL feature to control all information related to inner header hash.

Hi, Parav.

Do you think we need both hash_types and hash_tunnel_types?

In struct virtio_net_config we need two fields.
a. supported_hash_types (already exists)
b. supported_hash_tunnel_type -> bitmap indicating for which outer headers, inner hash calculation is supported.

Thanks for the suggestion, we seem to have reached an agreement.

Quoted 6 lines

In struct virtio_net_hdr we need two fields.
a. hash_report (already exists)
b. hash_tunnel_type 8 bits -> absolute value indicating which outer header exists when inner header hash calculated. You already have it in your patch named as hash_report_tunnel.
May be better to name as hash_report_tunnel_type to make it clearer that its type.

Sure.

Thanks for your reply.

#29 lore raw
Quoted 2 lines

From: Heng Qi <[email protected]>
Sent: Tuesday, February 7, 2023 10:25 PM

[..]

Quoted 13 lines
Quoted 5 lines
Quoted 2 lines

Do you think we need both hash_types and hash_tunnel_types?

In struct virtio_net_config we need two fields.
a. supported_hash_types (already exists) b. supported_hash_tunnel_type -> bitmap indicating for which outer headers, inner hash calculation is

supported.

Thanks for the suggestion, we seem to have reached an agreement.

Quoted 4 lines

In struct virtio_net_hdr we need two fields.
a. hash_report (already exists)
b. hash_tunnel_type 8 bits -> absolute value indicating which outer header

Quoted 8 lines

exists when inner header hash calculated.

Quoted 2 lines

You already have it in your patch named as hash_report_tunnel. May be better to name as hash_report_tunnel_type to make it clearer that its

type.

Sure.

Thanks for your reply.

I had one last question. Why do we need to inform the hash_report_tunnel_type of the outer header in the virtio_net_hdr? Is this for debug? Or is there a use case that will process this value?

author
#30 lore raw
Quoted 17 lines · Parav Pandit
Quoted 2 lines

From: Heng Qi <[email protected]>
Sent: Tuesday, February 7, 2023 10:25 PM

[..]

Quoted 14 lines
Quoted 4 lines
Quoted 1 line

Do you think we need both hash_types and hash_tunnel_types?

In struct virtio_net_config we need two fields.
a. supported_hash_types (already exists) b. supported_hash_tunnel_type -> bitmap indicating for which outer headers, inner hash calculation is

supported.

Thanks for the suggestion, we seem to have reached an agreement.

Quoted 3 lines

In struct virtio_net_hdr we need two fields.
a. hash_report (already exists)
b. hash_tunnel_type 8 bits -> absolute value indicating which outer header

exists when inner header hash calculated.

Quoted 2 lines

You already have it in your patch named as hash_report_tunnel. May be better to name as hash_report_tunnel_type to make it clearer that its

Quoted 7 lines
Quoted 5 lines

type.

Sure.

Thanks for your reply.

I had one last question. Why do we need to inform the hash_report_tunnel_type of the outer header in the virtio_net_hdr? Is this for debug? Or is there a use case that will process this value?

The driver may use it to do some statistical information, or do some rx classification based on the rx hash, and we'd better not hide information from the driver.

Thanks.

#31 lore raw
Quoted 31 lines

From: Heng Qi <[email protected]>
Sent: Wednesday, February 8, 2023 1:11 AM

Quoted 25 lines · Parav Pandit
Quoted 2 lines

From: Heng Qi <[email protected]>
Sent: Tuesday, February 7, 2023 10:25 PM

[..]

Quoted 22 lines
Quoted 6 lines
Quoted 1 line

Do you think we need both hash_types and hash_tunnel_types?

In struct virtio_net_config we need two fields.
a. supported_hash_types (already exists) b. supported_hash_tunnel_type
-> bitmap indicating for which outer headers, inner hash calculation -> is

supported.

Thanks for the suggestion, we seem to have reached an agreement.

Quoted 4 lines

In struct virtio_net_hdr we need two fields.
a. hash_report (already exists)
b. hash_tunnel_type 8 bits -> absolute value indicating which outer header

exists when inner header hash calculated.

Quoted 3 lines

You already have it in your patch named as hash_report_tunnel. May be better to name as hash_report_tunnel_type to make it clearer that its

type.

Sure.

Quoted 9 lines
Quoted 2 lines
Quoted 1 line

Thanks for your reply.

I had one last question. Why do we need to inform the

hash_report_tunnel_type of the outer header in the virtio_net_hdr?

Quoted 1 line

Is this for debug? Or is there a use case that will process this value?

The driver may use it to do some statistical information, or do some rx classification based on the rx hash, and we'd better not hide information from the driver.

Statistical information is better gathered via stats, instead of adding such code in driver data path.

It is not about hiding.
It has onus on the data path to detect the tunnel type and place that virtio_net_hdr. So lets find one use of it in data path which sw entity will use it.

author
#32 lore raw
Quoted 31 lines · Parav Pandit
Quoted 31 lines

From: Heng Qi <[email protected]>
Sent: Wednesday, February 8, 2023 1:11 AM

Quoted 26 lines · Parav Pandit
Quoted 2 lines

From: Heng Qi <[email protected]>
Sent: Tuesday, February 7, 2023 10:25 PM

[..]

Quoted 23 lines
Quoted 6 lines
Quoted 1 line

Do you think we need both hash_types and hash_tunnel_types?

In struct virtio_net_config we need two fields.
a. supported_hash_types (already exists) b. supported_hash_tunnel_type
-> bitmap indicating for which outer headers, inner hash calculation -> is

supported.

Thanks for the suggestion, we seem to have reached an agreement.

Quoted 4 lines

In struct virtio_net_hdr we need two fields.
a. hash_report (already exists)
b. hash_tunnel_type 8 bits -> absolute value indicating which outer header

exists when inner header hash calculated.

Quoted 3 lines

You already have it in your patch named as hash_report_tunnel. May be better to name as hash_report_tunnel_type to make it clearer that its

type.

Sure.

Thanks for your reply.

Quoted 8 lines
Quoted 7 lines
Quoted 1 line

I had one last question. Why do we need to inform the

hash_report_tunnel_type of the outer header in the virtio_net_hdr?

Quoted 1 line

Is this for debug? Or is there a use case that will process this value?

The driver may use it to do some statistical information, or do some rx classification based on the rx hash, and we'd better not hide information from the driver.

Statistical information is better gathered via stats, instead of adding such code in driver data path.

A cautionary note : the source of hash_report_tunnel comes from the discussion here https://lists.oasis-open.org/archives/virtio-dev/202211/msg00063.html . If I understand correctly, if virtio-net-hdr also has hash_report_tunnel, how does the driver do statistics?

CC Yuri for some opinions.

Thanks.

Quoted 6 lines

It is not about hiding.
It has onus on the data path to detect the tunnel type and place that virtio_net_hdr. So lets find one use of it in data path which sw entity will use it.

This publicly archived list offers a means to provide input to the OASIS Virtual I/O Device (VIRTIO) TC.

In order to verify user consent to the Feedback License terms and to minimize spam in the list archive, subscription is required before posting.

Subscribe: [email protected]
Unsubscribe: [email protected]
List help: [email protected]
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/

#33 lore raw
Quoted 13 lines

From: Heng Qi <[email protected]>
Sent: Thursday, February 9, 2023 12:21 AM

Quoted 9 lines
Quoted 8 lines
Quoted 2 lines
Quoted 1 line

Thanks for your reply.

I had one last question. Why do we need to inform the

hash_report_tunnel_type of the outer header in the virtio_net_hdr?

Quoted 1 line

Is this for debug? Or is there a use case that will process this value?

The driver may use it to do some statistical information, or do some rx classification based on the rx hash, and we'd better not hide information from the driver.

Statistical information is better gathered via stats, instead of adding such code

Quoted 8 lines

in driver data path.

A cautionary note : the source of hash_report_tunnel comes from the discussion here
https://lists.oasis-open.org/archives/virtio-dev/202211/msg00063.html

If I understand correctly, if virtio-net-hdr also has hash_report_tunnel, how does the driver do statistics?

I am asking to avoid having hash_report_tunnel in virtio_net_hdr if it is only for the statistics purpose.
If it has more use than just statistics, I like to understand its use in rx packet processing flow, if there is any and to describe in commit log too.

Below [1] says about driver doing some rx classification. It is unclear on inner or outer. The response is not very clear to me, about how this field is useful.

[1] https://lists.oasis-open.org/archives/virtio-dev/202211/msg00063.html

If driver needs to know the statistics of it, it can query them from the device by introducing statistics cmd.

author
#34 lore raw
Quoted 11 lines · Parav Pandit
Quoted 11 lines

From: Heng Qi <[email protected]>
Sent: Thursday, February 9, 2023 12:21 AM

Quoted 9 lines
Quoted 8 lines
Quoted 2 lines
Quoted 1 line

Thanks for your reply.

I had one last question. Why do we need to inform the

hash_report_tunnel_type of the outer header in the virtio_net_hdr?

Quoted 1 line

Is this for debug? Or is there a use case that will process this value?

The driver may use it to do some statistical information, or do some rx classification based on the rx hash, and we'd better not hide information from the driver.

Statistical information is better gathered via stats, instead of adding such code

Quoted 8 lines
Quoted 7 lines

in driver data path.

A cautionary note : the source of hash_report_tunnel comes from the discussion here
https://lists.oasis-open.org/archives/virtio-dev/202211/msg00063.html
If I understand correctly, if virtio-net-hdr also has hash_report_tunnel, how does the driver do statistics?

I am asking to avoid having hash_report_tunnel in virtio_net_hdr if it is only for the statistics purpose.

Parav, I don't seem to follow this.

Note that the hash_report(le16) and padding reserved(le16) fields only exist when VIRTIO_NET_F_HASH_REPORT is negotiated, and hash_report_tunnel only uses the first 8 bits of padding_reserved, which does not add any overhead.

Whether to reuse hash_report or use the new hash_report_tunnel initially has been discussed: https://lists.oasis-open.org/archives/virtio-dev/202211/msg00065.html .

Quoted 1 line

If it has more use than just statistics, I like to understand its use in rx packet processing flow, if there is any and to describe in commit log too.

I think there are still some important points, such as the kernel or driver forwards the packet,
and the hash value is used when selecting the queue, but at this time they want to know whether
the hash value comes from the inner or outer header.
What if it doesn't want to forward based on the hash calculated by the inner header, but only uses the hash calculated by the outer header?

Thanks.

Quoted 6 lines

Below [1] says about driver doing some rx classification. It is unclear on inner or outer. The response is not very clear to me, about how this field is useful.

[1] https://lists.oasis-open.org/archives/virtio-dev/202211/msg00063.html

If driver needs to know the statistics of it, it can query them from the device by introducing statistics cmd.

author
#35 lore raw

Any more discussions or comments? Please let me know if this is OK with you. and the v8 version is released at
[1] https://lists.oasis-open.org/archives/virtio-dev/202302/msg00094.html

Thanks.

Quoted 54 lines · Heng Qi
Quoted 11 lines · Parav Pandit
Quoted 11 lines

From: Heng Qi <[email protected]>
Sent: Thursday, February 9, 2023 12:21 AM

Quoted 9 lines
Quoted 8 lines
Quoted 2 lines
Quoted 1 line

Thanks for your reply.

I had one last question. Why do we need to inform the

hash_report_tunnel_type of the outer header in the virtio_net_hdr?

Quoted 1 line

Is this for debug? Or is there a use case that will process this value?

The driver may use it to do some statistical information, or do some rx classification based on the rx hash, and we'd better not hide information from the driver.

Statistical information is better gathered via stats, instead of adding such code

Quoted 8 lines
Quoted 7 lines

in driver data path.

A cautionary note : the source of hash_report_tunnel comes from the discussion here
https://lists.oasis-open.org/archives/virtio-dev/202211/msg00063.html
If I understand correctly, if virtio-net-hdr also has hash_report_tunnel, how does the driver do statistics?

I am asking to avoid having hash_report_tunnel in virtio_net_hdr if it is only for the statistics purpose.

Parav, I don't seem to follow this.

Note that the hash_report(le16) and padding reserved(le16) fields only exist when VIRTIO_NET_F_HASH_REPORT is negotiated, and hash_report_tunnel only uses the first 8 bits of padding_reserved, which does not add any overhead.

Whether to reuse hash_report or use the new hash_report_tunnel initially has been discussed: https://lists.oasis-open.org/archives/virtio-dev/202211/msg00065.html .

Quoted 1 line

If it has more use than just statistics, I like to understand its use in rx packet processing flow, if there is any and to describe in commit log too.

I think there are still some important points, such as the kernel or driver forwards the packet,
and the hash value is used when selecting the queue, but at this time they want to know whether
the hash value comes from the inner or outer header.
What if it doesn't want to forward based on the hash calculated by the inner header, but only uses the hash calculated by the outer header?

Thanks.

Quoted 6 lines

Below [1] says about driver doing some rx classification. It is unclear on inner or outer. The response is not very clear to me, about how this field is useful.

[1] https://lists.oasis-open.org/archives/virtio-dev/202211/msg00063.html

If driver needs to know the statistics of it, it can query them from the device by introducing statistics cmd.

This publicly archived list offers a means to provide input to the OASIS Virtual I/O Device (VIRTIO) TC.

In order to verify user consent to the Feedback License terms and to minimize spam in the list archive, subscription is required before posting.

Quoted 8 lines

Subscribe: [email protected]
Unsubscribe: [email protected]
List help: [email protected]
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/

This publicly archived list offers a means to provide input to the OASIS Virtual I/O Device (VIRTIO) TC.

In order to verify user consent to the Feedback License terms and to minimize spam in the list archive, subscription is required before posting.

Subscribe: [email protected]
Unsubscribe: [email protected]
List help: [email protected]
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/

#36 lore raw
In reply to Parav Pandit
Quoted 20 lines · Parav Pandit
Quoted 2 lines

From: Heng Qi <[email protected]>
Sent: Tuesday, February 7, 2023 10:25 PM

[..]

Quoted 16 lines
Quoted 5 lines
Quoted 2 lines

Do you think we need both hash_types and hash_tunnel_types?

In struct virtio_net_config we need two fields.
a. supported_hash_types (already exists) b. supported_hash_tunnel_type -> bitmap indicating for which outer headers, inner hash calculation is

supported.

Thanks for the suggestion, we seem to have reached an agreement.

Quoted 4 lines

In struct virtio_net_hdr we need two fields.
a. hash_report (already exists)
b. hash_tunnel_type 8 bits -> absolute value indicating which outer header

exists when inner header hash calculated.

Quoted 2 lines

You already have it in your patch named as hash_report_tunnel. May be better to name as hash_report_tunnel_type to make it clearer that its

Quoted 8 lines
Quoted 5 lines

type.

Sure.

Thanks for your reply.

I had one last question. Why do we need to inform the hash_report_tunnel_type of the outer header in the virtio_net_hdr? Is this for debug? Or is there a use case that will process this value?

Well we have hash_report which is kind of similar (and also kind of pointless but I think it's there because WHQL wants it). Maybe we can steal some bits from there instead of a new field?

I have a follow up question though: are we only hashing the inner header or both inner and outer header? Somewhat confused on this.

In fact, CC Yuri for thoughts and suggestions from windows side of things.

Signature · 1 line
MST
#37 lore raw
Quoted 33 lines

From: Michael S. Tsirkin <[email protected]>
Sent: Wednesday, February 8, 2023 8:32 AM

Quoted 28 lines · Parav Pandit
Quoted 2 lines

From: Heng Qi <[email protected]>
Sent: Tuesday, February 7, 2023 10:25 PM

[..]

Quoted 24 lines
Quoted 7 lines
Quoted 2 lines

Do you think we need both hash_types and hash_tunnel_types?

In struct virtio_net_config we need two fields. a. supported_hash_types (already exists) b. supported_hash_tunnel_type
-> bitmap indicating for which outer headers, inner hash -> calculation is

supported.

Thanks for the suggestion, we seem to have reached an agreement.

Quoted 5 lines

In struct virtio_net_hdr we need two fields.
a. hash_report (already exists)
b. hash_tunnel_type 8 bits -> absolute value indicating which outer header

exists when inner header hash calculated.

Quoted 3 lines

You already have it in your patch named as hash_report_tunnel. May be better to name as hash_report_tunnel_type to make it clearer that its

type.

Sure.

Quoted 8 lines
Quoted 3 lines
Quoted 1 line

Thanks for your reply.

I had one last question. Why do we need to inform the

hash_report_tunnel_type of the outer header in the virtio_net_hdr?

Quoted 1 line

Is this for debug? Or is there a use case that will process this value?

Well we have hash_report which is kind of similar (and also kind of pointless but I think it's there because WHQL wants it).

Hash_report is useful. It tells hash_value is in which namespace (ipv4-tcp/ipv4 udp etc). OS can use this value to find tcp connection in a given namespace.

Quoted 3 lines

Maybe we can steal some bits
from there instead of a new field?

I do not have problem adding extra bits. I just don't find that just telling that its vxlan or nvgre to the OS is useful. If OS needs to know about outer header details, it needs to know the VNI information than just telling vxlan.

Quoted 4 lines

I have a follow up question though: are we only hashing the inner header or both inner and outer header? Somewhat confused on this.

I understood as inner header. But worth to describe it. May be there. Need to read v8 patch.

This publicly archived list offers a means to provide input to the OASIS Virtual I/O Device (VIRTIO) TC.

In order to verify user consent to the Feedback License terms and to minimize spam in the list archive, subscription is required before posting.

Subscribe: [email protected]
Unsubscribe: [email protected]
List help: [email protected]
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/

#38 lore raw
Quoted 41 lines · Parav Pandit
Quoted 33 lines

From: Michael S. Tsirkin <[email protected]>
Sent: Wednesday, February 8, 2023 8:32 AM

Quoted 28 lines · Parav Pandit
Quoted 2 lines

From: Heng Qi <[email protected]>
Sent: Tuesday, February 7, 2023 10:25 PM

[..]

Quoted 24 lines
Quoted 7 lines
Quoted 2 lines

Do you think we need both hash_types and hash_tunnel_types?

In struct virtio_net_config we need two fields. a. supported_hash_types (already exists) b. supported_hash_tunnel_type
-> bitmap indicating for which outer headers, inner hash -> calculation is

supported.

Thanks for the suggestion, we seem to have reached an agreement.

Quoted 5 lines

In struct virtio_net_hdr we need two fields.
a. hash_report (already exists)
b. hash_tunnel_type 8 bits -> absolute value indicating which outer header

exists when inner header hash calculated.

Quoted 3 lines

You already have it in your patch named as hash_report_tunnel. May be better to name as hash_report_tunnel_type to make it clearer that its

type.

Sure.

Quoted 8 lines
Quoted 3 lines
Quoted 1 line

Thanks for your reply.

I had one last question. Why do we need to inform the

hash_report_tunnel_type of the outer header in the virtio_net_hdr?

Quoted 1 line

Is this for debug? Or is there a use case that will process this value?

Well we have hash_report which is kind of similar (and also kind of pointless but I think it's there because WHQL wants it).

Quoted 8 lines

Hash_report is useful. It tells hash_value is in which namespace (ipv4-tcp/ipv4 udp etc). OS can use this value to find tcp connection in a given namespace.

Quoted 3 lines

Maybe we can steal some bits
from there instead of a new field?

I do not have problem adding extra bits. I just don't find that just telling that its vxlan or nvgre to the OS is useful. If OS needs to know about outer header details, it needs to know the VNI information than just telling vxlan.

This does make sense.

Quoted 5 lines
Quoted 4 lines

I have a follow up question though: are we only hashing the inner header or both inner and outer header? Somewhat confused on this.

I understood as inner header. But worth to describe it. May be there. Need to read v8 patch.

Hmm. I just realized that there's a security problem with hashing just the inner header: it allow users inside the tunnel control queueing outside. By observing packet loss some information leaks between tunnels.

Ideas for solving this they all involve hashing both inner and outer header:
1- report two sets of hashes. overkill?
2- hash both headers together
2- add salt. can come from driver or device itself

More ideas?

Signature · 18 lines
MST


This publicly archived list offers a means to provide input to the
OASIS Virtual I/O Device (VIRTIO) TC.

In order to verify user consent to the Feedback License terms and
to minimize spam in the list archive, subscription is required
before posting.

Subscribe: [email protected]
Unsubscribe: [email protected]
List help: [email protected]
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/
#39 lore raw
Quoted 63 lines

From: Michael S. Tsirkin <[email protected]>
Sent: Wednesday, February 8, 2023 8:52 AM

Quoted 34 lines · Parav Pandit
Quoted 34 lines

From: Michael S. Tsirkin <[email protected]>
Sent: Wednesday, February 8, 2023 8:32 AM

Quoted 29 lines · Parav Pandit
Quoted 2 lines

From: Heng Qi <[email protected]>
Sent: Tuesday, February 7, 2023 10:25 PM

[..]

Quoted 24 lines
Quoted 7 lines
Quoted 2 lines

Do you think we need both hash_types and hash_tunnel_types?

In struct virtio_net_config we need two fields. a. supported_hash_types (already exists) b. supported_hash_tunnel_type
-> bitmap indicating for which outer headers, inner hash -> calculation is

supported.

Thanks for the suggestion, we seem to have reached an agreement.

Quoted 4 lines

In struct virtio_net_hdr we need two fields.
a. hash_report (already exists) b. hash_tunnel_type 8 bits -> absolute value indicating which outer header

exists when inner header hash calculated.

Quoted 3 lines

You already have it in your patch named as hash_report_tunnel. May be better to name as hash_report_tunnel_type to make it clearer that its

type.

Sure.

Thanks for your reply.

Quoted 7 lines
Quoted 6 lines
Quoted 1 line

I had one last question. Why do we need to inform the

hash_report_tunnel_type of the outer header in the virtio_net_hdr?

Quoted 1 line

Is this for debug? Or is there a use case that will process this value?

Well we have hash_report which is kind of similar (and also kind of pointless but I think it's there because WHQL wants it).

Hash_report is useful. It tells hash_value is in which namespace (ipv4-tcp/ipv4

udp etc).

Quoted 6 lines

OS can use this value to find tcp connection in a given namespace.

Quoted 3 lines

Maybe we can steal some bits
from there instead of a new field?

I do not have problem adding extra bits. I just don't find that just telling that

its vxlan or nvgre to the OS is useful.

Quoted 1 line

If OS needs to know about outer header details, it needs to know the VNI

information than just telling vxlan.

This does make sense.

Quoted 4 lines
Quoted 4 lines

I have a follow up question though: are we only hashing the inner header or both inner and outer header? Somewhat confused on this.

Quoted 7 lines
Quoted 1 line

I understood as inner header. But worth to describe it. May be there. Need to

read v8 patch.

Hmm. I just realized that there's a security problem with hashing just the inner
header: it allow users inside the tunnel control queueing outside. By observing packet loss some information leaks between tunnels.

I likely didn't understand. Can you please explain?

Queuing is always done on the inner header with/without encapsulation. Hash is always reported for inner header.
It is only adding the ability to hash even when outer header exists.

If queuing to be decided based on outer header (hash), then that is different. Hashing both inner and outer in a flat q structure unlikely works, right? Because both hashes can result in different q selection.

Quoted 11 lines

Ideas for solving this they all involve hashing both inner and outer header:
1- report two sets of hashes. overkill?
2- hash both headers together
2- add salt. can come from driver or device itself

More ideas?

--
MST

#40 lore raw
Quoted 70 lines · Parav Pandit
Quoted 63 lines

From: Michael S. Tsirkin <[email protected]>
Sent: Wednesday, February 8, 2023 8:52 AM

Quoted 34 lines · Parav Pandit
Quoted 34 lines

From: Michael S. Tsirkin <[email protected]>
Sent: Wednesday, February 8, 2023 8:32 AM

Quoted 29 lines · Parav Pandit
Quoted 2 lines

From: Heng Qi <[email protected]>
Sent: Tuesday, February 7, 2023 10:25 PM

[..]

Quoted 24 lines
Quoted 7 lines
Quoted 2 lines

Do you think we need both hash_types and hash_tunnel_types?

In struct virtio_net_config we need two fields. a. supported_hash_types (already exists) b. supported_hash_tunnel_type
-> bitmap indicating for which outer headers, inner hash -> calculation is

supported.

Thanks for the suggestion, we seem to have reached an agreement.

Quoted 4 lines

In struct virtio_net_hdr we need two fields.
a. hash_report (already exists) b. hash_tunnel_type 8 bits -> absolute value indicating which outer header

exists when inner header hash calculated.

Quoted 3 lines

You already have it in your patch named as hash_report_tunnel. May be better to name as hash_report_tunnel_type to make it clearer that its

type.

Sure.

Thanks for your reply.

Quoted 7 lines
Quoted 6 lines
Quoted 1 line

I had one last question. Why do we need to inform the

hash_report_tunnel_type of the outer header in the virtio_net_hdr?

Quoted 1 line

Is this for debug? Or is there a use case that will process this value?

Well we have hash_report which is kind of similar (and also kind of pointless but I think it's there because WHQL wants it).

Hash_report is useful. It tells hash_value is in which namespace (ipv4-tcp/ipv4

udp etc).

Quoted 6 lines

OS can use this value to find tcp connection in a given namespace.

Quoted 3 lines

Maybe we can steal some bits
from there instead of a new field?

I do not have problem adding extra bits. I just don't find that just telling that

its vxlan or nvgre to the OS is useful.

Quoted 1 line

If OS needs to know about outer header details, it needs to know the VNI

information than just telling vxlan.

This does make sense.

Quoted 4 lines
Quoted 4 lines

I have a follow up question though: are we only hashing the inner header or both inner and outer header? Somewhat confused on this.

Quoted 7 lines
Quoted 1 line

I understood as inner header. But worth to describe it. May be there. Need to

read v8 patch.

Hmm. I just realized that there's a security problem with hashing just the inner
header: it allow users inside the tunnel control queueing outside. By observing packet loss some information leaks between tunnels.

Quoted 5 lines

I likely didn't understand. Can you please explain?

Queuing is always done on the inner header with/without encapsulation. Hash is always reported for inner header.
It is only adding the ability to hash even when outer header exists.

If hashing just on outer header (currently the only option) then a given tunnel all lands in a given queue.
Just keep that queue separate and users of this tunnel can not learn whether other queues are overflowing, and can not overflow other queues.

If you hash inner header then user can flood device with
packets of a given connection and the same connection in a different tunnel hashes to the same queue. Now one tunnel can
- cause DoS for another tunnel
- cause packet loss or latency triggering possible security bugs within guest
- detect that another tunnel is using the connection by
detecting its own packet loss or increased latency

Quoted 3 lines

If queuing to be decided based on outer header (hash), then that is different. Hashing both inner and outer in a flat q structure unlikely works, right? Because both hashes can result in different q selection.

That's the point.

Is there any precedent in OSes for configuring things like this that we can look at?

Quoted 11 lines
Quoted 11 lines

Ideas for solving this they all involve hashing both inner and outer header:
1- report two sets of hashes. overkill?
2- hash both headers together
2- add salt. can come from driver or device itself

More ideas?

--
MST

#41 lore raw
Quoted 19 lines

From: Michael S. Tsirkin <[email protected]>
Sent: Wednesday, February 8, 2023 9:09 AM

Quoted 8 lines
Quoted 3 lines

header: it allow users inside the tunnel control queueing outside. By observing packet loss some information leaks between tunnels.

I likely didn't understand. Can you please explain?

Queuing is always done on the inner header with/without encapsulation. Hash is always reported for inner header.
It is only adding the ability to hash even when outer header exists.

If hashing just on outer header (currently the only option) then a given tunnel all lands in a given queue.
Just keep that queue separate and users of this tunnel can not learn whether other queues are overflowing, and can not overflow other queues.

Quoted 8 lines

If you hash inner header then user can flood device with packets of a given connection and the same connection in a different tunnel hashes to the same queue. Now one tunnel can
- cause DoS for another tunnel
- cause packet loss or latency triggering possible security bugs within guest
- detect that another tunnel is using the connection by
detecting its own packet loss or increased latency

Yes. It can lead to above issues.
Steering on inner is on best effort based sw implementations running on top of net device.
To avoid above issues, a hierarchical model is needed.
I am not aware of any.
To my knowledge, usually who care for above issues end up using a different net device for each VNI and achieve the desired hierarchy.

Quoted 11 lines
Quoted 4 lines

If queuing to be decided based on outer header (hash), then that is different. Hashing both inner and outer in a flat q structure unlikely works, right? Because both hashes can result in different q selection.

That's the point.

Is there any precedent in OSes for configuring things like this that we can look at?

ethtool -N (not yet part of virtio) is the closest match that can steer based on inner and outer both, but it is not hierarchical, and it is orthogonal to this feature.

Quoted 12 lines
Quoted 12 lines
Quoted 12 lines

Ideas for solving this they all involve hashing both inner and outer header:
1- report two sets of hashes. overkill?
2- hash both headers together
2- add salt. can come from driver or device itself

More ideas?

--
MST

author
#42 lore raw
In reply to Michael S. Tsirkin
Quoted 78 lines · Michael S. Tsirkin
Quoted 70 lines · Parav Pandit
Quoted 63 lines

From: Michael S. Tsirkin <[email protected]>
Sent: Wednesday, February 8, 2023 8:52 AM

Quoted 34 lines · Parav Pandit
Quoted 34 lines

From: Michael S. Tsirkin <[email protected]>
Sent: Wednesday, February 8, 2023 8:32 AM

Quoted 29 lines · Parav Pandit
Quoted 2 lines

From: Heng Qi <[email protected]>
Sent: Tuesday, February 7, 2023 10:25 PM

[..]

Quoted 24 lines
Quoted 7 lines
Quoted 2 lines

Do you think we need both hash_types and hash_tunnel_types?

In struct virtio_net_config we need two fields. a. supported_hash_types (already exists) b. supported_hash_tunnel_type
-> bitmap indicating for which outer headers, inner hash -> calculation is

supported.

Thanks for the suggestion, we seem to have reached an agreement.

Quoted 4 lines

In struct virtio_net_hdr we need two fields.
a. hash_report (already exists) b. hash_tunnel_type 8 bits -> absolute value indicating which outer header

exists when inner header hash calculated.

Quoted 3 lines

You already have it in your patch named as hash_report_tunnel. May be better to name as hash_report_tunnel_type to make it clearer that its

type.

Sure.

Thanks for your reply.

Quoted 7 lines
Quoted 6 lines
Quoted 1 line

I had one last question. Why do we need to inform the

hash_report_tunnel_type of the outer header in the virtio_net_hdr?

Quoted 1 line

Is this for debug? Or is there a use case that will process this value?

Well we have hash_report which is kind of similar (and also kind of pointless but I think it's there because WHQL wants it).

Hash_report is useful. It tells hash_value is in which namespace (ipv4-tcp/ipv4

udp etc).

Quoted 6 lines

OS can use this value to find tcp connection in a given namespace.

Quoted 3 lines

Maybe we can steal some bits
from there instead of a new field?

I do not have problem adding extra bits. I just don't find that just telling that

its vxlan or nvgre to the OS is useful.

Quoted 1 line

If OS needs to know about outer header details, it needs to know the VNI

information than just telling vxlan.

This does make sense.

Quoted 4 lines
Quoted 4 lines

I have a follow up question though: are we only hashing the inner header or both inner and outer header? Somewhat confused on this.

Quoted 7 lines
Quoted 1 line

I understood as inner header. But worth to describe it. May be there. Need to

read v8 patch.

Hmm. I just realized that there's a security problem with hashing just the inner
header: it allow users inside the tunnel control queueing outside. By observing packet loss some information leaks between tunnels.

Quoted 5 lines

I likely didn't understand. Can you please explain?

Queuing is always done on the inner header with/without encapsulation. Hash is always reported for inner header.
It is only adding the ability to hash even when outer header exists.

Quoted 6 lines

If hashing just on outer header (currently the only option) then a given tunnel all lands in a given queue.
Just keep that queue separate and users of this tunnel can not learn whether other queues are overflowing, and can not overflow other queues.

This is not a problem with the inner hash, it is a general problem with the outer hash.
I communicated with our people who are doing cloud security (they are also one of the demanders of inner hash), and it is a common problem for one tunnel to attack another tunnel, which has nothing to do with inner hash or outer hash.

For example, there is a tunnel t1; a tunnel t2; a tunnel endpoint VTEP0, and the vni id of t1 is id1, and the vni id of v2 is id2; a VM.

-----------
Tunnel t1 | |
VTEP_1 <=======================> (vni id1) |

                                  |         |
								  |   VTEP0 | ======================> VM
               Tunnel t2          |         |
VTEP_2 <=======================> (vni id2)  |
								  -----------

At this time, regardless of the inner hash or the outer hash, the traffic of tunnel t1 and tunnel t2 will reach the VM through VTEP0 (whether it is a single queue or multiple queues), and may be placed on the same queue to cause queue overflow. Some current forwarding tools such as DPDK have good forwarding performance, and it is difficult to fill up the queue; or switch the attack traffic to the attack clusters; or connect the traffic of different tunnels to different network card ports or network devices.

Thanks.

Quoted 43 lines

If you hash inner header then user can flood device with
packets of a given connection and the same connection in a different tunnel hashes to the same queue. Now one tunnel can
- cause DoS for another tunnel
- cause packet loss or latency triggering possible security bugs within guest
- detect that another tunnel is using the connection by
detecting its own packet loss or increased latency

Quoted 3 lines

If queuing to be decided based on outer header (hash), then that is different. Hashing both inner and outer in a flat q structure unlikely works, right? Because both hashes can result in different q selection.

That's the point.

Is there any precedent in OSes for configuring things like this that we can look at?

Quoted 11 lines
Quoted 11 lines

Ideas for solving this they all involve hashing both inner and outer header:
1- report two sets of hashes. overkill?
2- hash both headers together
2- add salt. can come from driver or device itself

More ideas?

--
MST

This publicly archived list offers a means to provide input to the OASIS Virtual I/O Device (VIRTIO) TC.

In order to verify user consent to the Feedback License terms and to minimize spam in the list archive, subscription is required before posting.

Quoted 8 lines

Subscribe: [email protected]
Unsubscribe: [email protected]
List help: [email protected]
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/

This publicly archived list offers a means to provide input to the OASIS Virtual I/O Device (VIRTIO) TC.

In order to verify user consent to the Feedback License terms and to minimize spam in the list archive, subscription is required before posting.

Subscribe: [email protected]
Unsubscribe: [email protected]
List help: [email protected]
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/

author
#43 lore raw
Quoted 93 lines · Heng Qi
Quoted 78 lines · Michael S. Tsirkin
Quoted 70 lines · Parav Pandit
Quoted 63 lines

From: Michael S. Tsirkin <[email protected]>
Sent: Wednesday, February 8, 2023 8:52 AM

Quoted 34 lines · Parav Pandit
Quoted 34 lines

From: Michael S. Tsirkin <[email protected]>
Sent: Wednesday, February 8, 2023 8:32 AM

Quoted 29 lines · Parav Pandit
Quoted 2 lines

From: Heng Qi <[email protected]>
Sent: Tuesday, February 7, 2023 10:25 PM

[..]

Quoted 24 lines
Quoted 7 lines
Quoted 2 lines

Do you think we need both hash_types and hash_tunnel_types?

In struct virtio_net_config we need two fields. a. supported_hash_types (already exists) b. supported_hash_tunnel_type
-> bitmap indicating for which outer headers, inner hash -> calculation is

supported.

Thanks for the suggestion, we seem to have reached an agreement.

Quoted 4 lines

In struct virtio_net_hdr we need two fields.
a. hash_report (already exists) b. hash_tunnel_type 8 bits -> absolute value indicating which outer header

exists when inner header hash calculated.

Quoted 3 lines

You already have it in your patch named as hash_report_tunnel. May be better to name as hash_report_tunnel_type to make it clearer that its

type.

Sure.

Thanks for your reply.

Quoted 7 lines
Quoted 6 lines
Quoted 1 line

I had one last question. Why do we need to inform the

hash_report_tunnel_type of the outer header in the virtio_net_hdr?

Quoted 1 line

Is this for debug? Or is there a use case that will process this value?

Well we have hash_report which is kind of similar (and also kind of pointless but I think it's there because WHQL wants it).

Hash_report is useful. It tells hash_value is in which namespace (ipv4-tcp/ipv4

udp etc).

Quoted 6 lines

OS can use this value to find tcp connection in a given namespace.

Quoted 3 lines

Maybe we can steal some bits
from there instead of a new field?

I do not have problem adding extra bits. I just don't find that just telling that

its vxlan or nvgre to the OS is useful.

Quoted 1 line

If OS needs to know about outer header details, it needs to know the VNI

information than just telling vxlan.

This does make sense.

Quoted 4 lines
Quoted 4 lines

I have a follow up question though: are we only hashing the inner header or both inner and outer header? Somewhat confused on this.

Quoted 7 lines
Quoted 1 line

I understood as inner header. But worth to describe it. May be there. Need to

read v8 patch.

Hmm. I just realized that there's a security problem with hashing just the inner
header: it allow users inside the tunnel control queueing outside. By observing packet loss some information leaks between tunnels.

Quoted 5 lines

I likely didn't understand. Can you please explain?

Queuing is always done on the inner header with/without encapsulation. Hash is always reported for inner header.
It is only adding the ability to hash even when outer header exists.

Quoted 6 lines

If hashing just on outer header (currently the only option) then a given tunnel all lands in a given queue.
Just keep that queue separate and users of this tunnel can not learn whether other queues are overflowing, and can not overflow other queues.

This is not a problem with the inner hash, it is a general problem with the outer hash.
I communicated with our people who are doing cloud security (they are also one of the demanders of inner hash), and it is a common problem for one tunnel to attack another tunnel, which has nothing to do with inner hash or outer hash.

For example, there is a tunnel t1; a tunnel t2; a tunnel endpoint VTEP0, and the vni id of t1 is id1, and the vni id of v2 is id2; a VM.

Quoted 8 lines

-----------
Tunnel t1 | |
VTEP_1 <=======================> (vni id1) |

                                  |         |
								  |   VTEP0 | ======================> VM
               Tunnel t2          |         |
VTEP_2 <=======================> (vni id2)  |
								  -----------

The description graph should look like this:

-----------
Tunnel t1 | |

VTEP_1 <=======================> (vni id1)  |
                                  |         |
                                  |   VTEP  | ======================> VM
               Tunnel t2          |         |
VTEP_2 <=======================> (vni id2)  |
                                  -----------
Quoted 63 lines

At this time, regardless of the inner hash or the outer hash, the traffic of tunnel t1 and tunnel t2 will reach the VM through VTEP0 (whether it is a single queue or multiple queues), and may be placed on the same queue to cause queue overflow. Some current forwarding tools such as DPDK have good forwarding performance, and it is difficult to fill up the queue; or switch the attack traffic to the attack clusters; or connect the traffic of different tunnels to different network card ports or network devices.

Thanks.

Quoted 43 lines

If you hash inner header then user can flood device with
packets of a given connection and the same connection in a different tunnel hashes to the same queue. Now one tunnel can
- cause DoS for another tunnel
- cause packet loss or latency triggering possible security bugs within guest
- detect that another tunnel is using the connection by
detecting its own packet loss or increased latency

Quoted 3 lines

If queuing to be decided based on outer header (hash), then that is different. Hashing both inner and outer in a flat q structure unlikely works, right? Because both hashes can result in different q selection.

That's the point.

Is there any precedent in OSes for configuring things like this that we can look at?

Quoted 11 lines
Quoted 11 lines

Ideas for solving this they all involve hashing both inner and outer header:
1- report two sets of hashes. overkill?
2- hash both headers together
2- add salt. can come from driver or device itself

More ideas?

--
MST

This publicly archived list offers a means to provide input to the OASIS Virtual I/O Device (VIRTIO) TC.

In order to verify user consent to the Feedback License terms and to minimize spam in the list archive, subscription is required before posting.

Quoted 8 lines

Subscribe: [email protected]
Unsubscribe: [email protected]
List help: [email protected]
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/

---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]

#44 lore raw
In reply to Michael S. Tsirkin
Quoted 63 lines

From: Michael S. Tsirkin <[email protected]>
Sent: Wednesday, February 8, 2023 8:52 AM

Quoted 34 lines · Parav Pandit
Quoted 34 lines

From: Michael S. Tsirkin <[email protected]>
Sent: Wednesday, February 8, 2023 8:32 AM

Quoted 29 lines · Parav Pandit
Quoted 2 lines

From: Heng Qi <[email protected]>
Sent: Tuesday, February 7, 2023 10:25 PM

[..]

Quoted 24 lines
Quoted 7 lines
Quoted 2 lines

Do you think we need both hash_types and hash_tunnel_types?

In struct virtio_net_config we need two fields. a. supported_hash_types (already exists) b. supported_hash_tunnel_type
-> bitmap indicating for which outer headers, inner hash -> calculation is

supported.

Thanks for the suggestion, we seem to have reached an agreement.

Quoted 4 lines

In struct virtio_net_hdr we need two fields.
a. hash_report (already exists) b. hash_tunnel_type 8 bits -> absolute value indicating which outer header

exists when inner header hash calculated.

Quoted 3 lines

You already have it in your patch named as hash_report_tunnel. May be better to name as hash_report_tunnel_type to make it clearer that its

type.

Sure.

Thanks for your reply.

Quoted 7 lines
Quoted 6 lines
Quoted 1 line

I had one last question. Why do we need to inform the

hash_report_tunnel_type of the outer header in the virtio_net_hdr?

Quoted 1 line

Is this for debug? Or is there a use case that will process this value?

Well we have hash_report which is kind of similar (and also kind of pointless but I think it's there because WHQL wants it).

Hash_report is useful. It tells hash_value is in which namespace (ipv4-tcp/ipv4

udp etc).

Quoted 6 lines

OS can use this value to find tcp connection in a given namespace.

Quoted 3 lines

Maybe we can steal some bits
from there instead of a new field?

I do not have problem adding extra bits. I just don't find that just telling that

its vxlan or nvgre to the OS is useful.

Quoted 1 line

If OS needs to know about outer header details, it needs to know the VNI

information than just telling vxlan.

This does make sense.

Quoted 4 lines
Quoted 4 lines

I have a follow up question though: are we only hashing the inner header or both inner and outer header? Somewhat confused on this.

Quoted 7 lines
Quoted 1 line

I understood as inner header. But worth to describe it. May be there. Need to

read v8 patch.

Hmm. I just realized that there's a security problem with hashing just the inner
header: it allow users inside the tunnel control queueing outside. By observing packet loss some information leaks between tunnels.

Ah I know now.
We are leaking outer header information inside the virtio net hdr, and outer header might be already stripped off by a different entity.

I think the use case here is it's the same sw entity that owns the virtio net device does the encap/decap too.

#45 lore raw
Quoted 66 lines · Parav Pandit
Quoted 66 lines

From: Michael S. Tsirkin <[email protected]>
Sent: Wednesday, February 8, 2023 8:52 AM

Quoted 34 lines · Parav Pandit
Quoted 34 lines

From: Michael S. Tsirkin <[email protected]>
Sent: Wednesday, February 8, 2023 8:32 AM

Quoted 29 lines · Parav Pandit
Quoted 2 lines

From: Heng Qi <[email protected]>
Sent: Tuesday, February 7, 2023 10:25 PM

[..]

Quoted 24 lines
Quoted 7 lines
Quoted 2 lines

Do you think we need both hash_types and hash_tunnel_types?

In struct virtio_net_config we need two fields. a. supported_hash_types (already exists) b. supported_hash_tunnel_type
-> bitmap indicating for which outer headers, inner hash -> calculation is

supported.

Thanks for the suggestion, we seem to have reached an agreement.

Quoted 4 lines

In struct virtio_net_hdr we need two fields.
a. hash_report (already exists) b. hash_tunnel_type 8 bits -> absolute value indicating which outer header

exists when inner header hash calculated.

Quoted 3 lines

You already have it in your patch named as hash_report_tunnel. May be better to name as hash_report_tunnel_type to make it clearer that its

type.

Sure.

Thanks for your reply.

Quoted 7 lines
Quoted 6 lines
Quoted 1 line

I had one last question. Why do we need to inform the

hash_report_tunnel_type of the outer header in the virtio_net_hdr?

Quoted 1 line

Is this for debug? Or is there a use case that will process this value?

Well we have hash_report which is kind of similar (and also kind of pointless but I think it's there because WHQL wants it).

Hash_report is useful. It tells hash_value is in which namespace (ipv4-tcp/ipv4

udp etc).

Quoted 6 lines

OS can use this value to find tcp connection in a given namespace.

Quoted 3 lines

Maybe we can steal some bits
from there instead of a new field?

I do not have problem adding extra bits. I just don't find that just telling that

its vxlan or nvgre to the OS is useful.

Quoted 1 line

If OS needs to know about outer header details, it needs to know the VNI

information than just telling vxlan.

This does make sense.

Quoted 5 lines
Quoted 4 lines

I have a follow up question though: are we only hashing the inner header or both inner and outer header? Somewhat confused on this.

I understood as inner header. But worth to describe it. May be there. Need to

read v8 patch.

Quoted 8 lines
Quoted 4 lines

Hmm. I just realized that there's a security problem with hashing just the inner
header: it allow users inside the tunnel control queueing outside. By observing packet loss some information leaks between tunnels.

Ah I know now.
We are leaking outer header information inside the virtio net hdr, and outer header might be already stripped off by a different entity.

I think the use case here is it's the same sw entity that owns the virtio net device does the encap/decap too.

No not exactly, we are leaking info between encap tunnels.

Signature · 1 line
MST
#46 lore raw
In reply to Heng Qi

content.tex \subsection{Feature bits}\label{sec:Device Types / Network Device / Feature bits · line 3089

Quoted 12 lines

From: [email protected] <[email protected]>
Sent: Wednesday, January 4, 2023 2:14 AM

If the tunnel is used to encapsulate the packets, the hash calculated using the outer header of the receive packets is always fixed for the same flow packets, i.e. they will be steered to the same receive queue.

+\item[VIRTIO_NET_F_HASH_TUNNEL(52)] Device supports inner

content.tex +2 -0
+    header hash for GRE, VXLAN and GENEVE tunnel-encapsulated packets.
+

A device may not support all 3 at the same time.
Please remove mentioning tunneling protocols description from here. Just say device support inner header hash ...

An additional bit map somewhere else should say supported hash over different tunneling types.

[...]

Quoted 112 lines
(quoted diff) +4 -0
+The device calculates the hash on the inner IPv4 packet of an
+encapsulated packet according to 'Enabled hash types' bitmask as follows:
+\begin{itemize}
+  \item If VIRTIO_NET_HASH_TYPE_TCPv4 is set and the encapsulated packet

has an inner

content.tex +8 -0
+	TCPv4 header, the hash is calculated over the following fields:
+        \begin{itemsize}
+          \item inner Source IP address
+          \item inner Destination IP address
+          \item inner Source TCP port
+          \item inner Destination TCP port
+        \end{itemsize}
+  \item Else if VIRTIO_NET_HASH_TYPE_UDPv4 is set and the encapsulated

packet has an

content.tex +8 -0
+	inner UDPv4 header, the hash is calculated over the following fields:
+        \begin{itemsize}
+          \item inner Source IP address
+          \item inner Destination IP address
+          \item inner Source UDP port
+          \item inner Destination UDP port
+        \end{itemize}
+  \item Else if VIRTIO_NET_HASH_TYPE_IPv4 is set, the hash is calculated over

the

content.tex +9 -0
+	following fields:
+        \begin{itemsize}
+          \item inner Source IP address
+          \item inner Destination IP address
+        \end{itemsize}
+  \item Else the device does not calculate the hash \end{itemize}
+
+The device calculates the hash on the inner IPv6 packet without an
+extension header of an encapsulated packet according to 'Enabled hash types'

bitmask as follows:
+\begin{itemize}
+ \item If VIRTIO_NET_HASH_TYPE_TCPv6 is set and the encapsulated packet has an inner

content.tex +8 -0
+	TCPv6 header, the hash is calculated over the following fields:
+        \begin{itemsize}
+          \item inner Source IPv6 address
+          \item inner Destination IPv6 address
+          \item inner Source TCP port
+          \item inner Destination TCP port
+        \end{itemsize}
+  \item Else if VIRTIO_NET_HASH_TYPE_UDPv6 is set and the encapsulated

packet has an

content.tex +8 -0
+	inner UDPv6 header, the hash is calculated over the following fields:
+        \begin{itemsize}
+          \item inner Source IPv6 address
+          \item inner Destination IPv6 address
+          \item inner Source UDP port
+          \item inner Destination UDP port
+        \end{itemize}
+  \item Else if VIRTIO_NET_HASH_TYPE_IPv6 is set, the hash is calculated over

the

content.tex +9 -0
+	following fields:
+        \begin{itemsize}
+          \item inner Source IPv6 address
+          \item inner Destination IPv6 address
+        \end{itemsize}
+  \item Else the device does not calculate the hash \end{itemize}
+
+The device calculates the hash on the inner IPv6 packet with an
+extension header of an encapsulated packet according to 'Enabled hash types'

bitmask as follows:
+\begin{itemsize}
+ \item If VIRTIO_NET_HASH_TYPE_TCP_EX is set and the encapsulated packet has an inner

content.tex +3 -0
+	TCPv6 header, the hash is calculated over the following fields:
+        \begin{itemize}
+          \item Home address from the home address option in the inner IPv6

destination

content.tex +3 -0
+              options header. If the inner extension header is not present, use the
+              inner Source IPv6 address.
+          \item IPv6 address that is contained in the Routing-Header-Type-2 from

the

content.tex +6 -0
+              associated inner extension header. If the inner extension header is not
+              present, use the inner Destination IPv6 address.
+          \item inner Source TCP port
+          \item inner Destination TCP port
+        \end{itemize}
+  \item Else if VIRTIO_NET_HASH_TYPE_UDP_EX is set and the encapsulated

packet has an inner

content.tex +3 -0
+	UDPv6 header, the hash is calculated over the following fields:
+        \begin{itemsize}
+          \item Home address from the home address option in the inner IPv6

destination

content.tex +3 -0
+              options header. If the inner extension header is not present, use the
+              inner Source IPv6 address.
+          \item IPv6 address that is contained in the Routing-Header-Type-2 from

the

content.tex +6 -0
+              associated inner extension header. If the inner extension header is not
+              present, use the inner Destination IPv6 address.
+          \item inner Source UDP port
+          \item inner Destination UDP port
+        \end{itemize}
+  \item Else if VIRTIO_NET_HASH_TYPE_IP_EX is set, the hash is calculated

over the

content.tex +3 -0
+	following fields:
+        \begin{itemsize}
+          \item Home address from the home address option in the inner IPv6

destination

content.tex +3 -0
+              options header. If the inner extension header is not present, use the
+              inner Source IPv6 address.
+          \item IPv6 address that is contained in the Routing-Header-Type-2 from

the

content.tex +4 -0
+              associated inner extension header. If the inner extension header is not
+              present, use the inner Destination IPv6 address.
+        \end{itemize}
+  \item Else skip the inner IPv6 extension header and calculate the inner

header hash as

(quoted diff) +1 -0
+	defined for an encapsulated packet whose inner payload is an IPv6

packet without

content.tex \subsubsection{Processing of Incoming Packets}\label{sec:Device Types / Network · line 4134

Quoted 3 lines
content.tex +3 -0
+	an extension header.
+\end{itemsize}
+

With the inclusion of tunnel outer header, it doesn't need to redefine the hashing for inner packets which is exactly same without the tunnel. hash tunnel capability only indicates that hashing is done on the inner packets as_before.

author
#47 lore raw
Quoted 9 lines · Parav Pandit
Quoted 8 lines

From: [email protected] <[email protected]>
Sent: Wednesday, January 4, 2023 2:14 AM

If the tunnel is used to encapsulate the packets, the hash calculated using the outer header of the receive packets is always fixed for the same flow packets, i.e. they will be steered to the same receive queue.

Quoted 6 lines

content.tex \subsection{Feature bits}\label{sec:Device Types / Network Device / Feature bits · line 3089

Quoted 3 lines

+\item[VIRTIO_NET_F_HASH_TUNNEL(52)] Device supports inner

content.tex +2 -0
+    header hash for GRE, VXLAN and GENEVE tunnel-encapsulated packets.
+

A device may not support all 3 at the same time.
Please remove mentioning tunneling protocols description from here. Just say device support inner header hash ...

Sorry for the late reply due to vacation.

Good idea, Michael suggested doing the same. But we also discussed this issue:
Early, we used a feature bit to force devices to support GRE and VXLAN (see
https://lists.oasis-open.org/archives/virtio-dev/202211/msg00183.html ).
Later Jason suggested to use VIRTIO_NET_F_HASH_TUNNEL instead of VIRTIO_NET_F_HASH_GRE_VXLAN_GENEVE_INNER (see
https://lists.oasis-open.org/archives/virtio-dev/202212/msg00014.html ). Now Michael proposes to remove this list (see
https://lists.oasis-open.org/archives/virtio-dev/202301/msg00079.html ), because the migration is based on the feature bit and hash types to determine whether the live migration is successful.

Quoted 3 lines

An additional bit map somewhere else should say supported hash over different tunneling types.

Yes, we use \field{supported_hash_types} to declare supported hash types.

Quoted 112 lines

[...]

Quoted 110 lines
(quoted diff) +4 -0
+The device calculates the hash on the inner IPv4 packet of an
+encapsulated packet according to 'Enabled hash types' bitmask as follows:
+\begin{itemize}
+  \item If VIRTIO_NET_HASH_TYPE_TCPv4 is set and the encapsulated packet

has an inner

content.tex +8 -0
+	TCPv4 header, the hash is calculated over the following fields:
+        \begin{itemsize}
+          \item inner Source IP address
+          \item inner Destination IP address
+          \item inner Source TCP port
+          \item inner Destination TCP port
+        \end{itemsize}
+  \item Else if VIRTIO_NET_HASH_TYPE_UDPv4 is set and the encapsulated

packet has an

content.tex +8 -0
+	inner UDPv4 header, the hash is calculated over the following fields:
+        \begin{itemsize}
+          \item inner Source IP address
+          \item inner Destination IP address
+          \item inner Source UDP port
+          \item inner Destination UDP port
+        \end{itemize}
+  \item Else if VIRTIO_NET_HASH_TYPE_IPv4 is set, the hash is calculated over

the

content.tex +9 -0
+	following fields:
+        \begin{itemsize}
+          \item inner Source IP address
+          \item inner Destination IP address
+        \end{itemsize}
+  \item Else the device does not calculate the hash \end{itemize}
+
+The device calculates the hash on the inner IPv6 packet without an
+extension header of an encapsulated packet according to 'Enabled hash types'

bitmask as follows:
+\begin{itemize}
+ \item If VIRTIO_NET_HASH_TYPE_TCPv6 is set and the encapsulated packet has an inner

content.tex +8 -0
+	TCPv6 header, the hash is calculated over the following fields:
+        \begin{itemsize}
+          \item inner Source IPv6 address
+          \item inner Destination IPv6 address
+          \item inner Source TCP port
+          \item inner Destination TCP port
+        \end{itemsize}
+  \item Else if VIRTIO_NET_HASH_TYPE_UDPv6 is set and the encapsulated

packet has an

content.tex +8 -0
+	inner UDPv6 header, the hash is calculated over the following fields:
+        \begin{itemsize}
+          \item inner Source IPv6 address
+          \item inner Destination IPv6 address
+          \item inner Source UDP port
+          \item inner Destination UDP port
+        \end{itemize}
+  \item Else if VIRTIO_NET_HASH_TYPE_IPv6 is set, the hash is calculated over

the

content.tex +9 -0
+	following fields:
+        \begin{itemsize}
+          \item inner Source IPv6 address
+          \item inner Destination IPv6 address
+        \end{itemsize}
+  \item Else the device does not calculate the hash \end{itemize}
+
+The device calculates the hash on the inner IPv6 packet with an
+extension header of an encapsulated packet according to 'Enabled hash types'

bitmask as follows:
+\begin{itemsize}
+ \item If VIRTIO_NET_HASH_TYPE_TCP_EX is set and the encapsulated packet has an inner

content.tex +3 -0
+	TCPv6 header, the hash is calculated over the following fields:
+        \begin{itemize}
+          \item Home address from the home address option in the inner IPv6

destination

content.tex +3 -0
+              options header. If the inner extension header is not present, use the
+              inner Source IPv6 address.
+          \item IPv6 address that is contained in the Routing-Header-Type-2 from

the

content.tex +6 -0
+              associated inner extension header. If the inner extension header is not
+              present, use the inner Destination IPv6 address.
+          \item inner Source TCP port
+          \item inner Destination TCP port
+        \end{itemize}
+  \item Else if VIRTIO_NET_HASH_TYPE_UDP_EX is set and the encapsulated

packet has an inner

content.tex +3 -0
+	UDPv6 header, the hash is calculated over the following fields:
+        \begin{itemsize}
+          \item Home address from the home address option in the inner IPv6

destination

content.tex +3 -0
+              options header. If the inner extension header is not present, use the
+              inner Source IPv6 address.
+          \item IPv6 address that is contained in the Routing-Header-Type-2 from

the

content.tex +6 -0
+              associated inner extension header. If the inner extension header is not
+              present, use the inner Destination IPv6 address.
+          \item inner Source UDP port
+          \item inner Destination UDP port
+        \end{itemize}
+  \item Else if VIRTIO_NET_HASH_TYPE_IP_EX is set, the hash is calculated

over the

content.tex +3 -0
+	following fields:
+        \begin{itemsize}
+          \item Home address from the home address option in the inner IPv6

destination

content.tex +3 -0
+              options header. If the inner extension header is not present, use the
+              inner Source IPv6 address.
+          \item IPv6 address that is contained in the Routing-Header-Type-2 from

the

content.tex +4 -0
+              associated inner extension header. If the inner extension header is not
+              present, use the inner Destination IPv6 address.
+        \end{itemize}
+  \item Else skip the inner IPv6 extension header and calculate the inner

header hash as

Quoted 9 lines

content.tex \subsubsection{Processing of Incoming Packets}\label{sec:Device Types / Network · line 4134

Quoted 5 lines
(quoted diff) +1 -0
+	defined for an encapsulated packet whose inner payload is an IPv6

packet without

content.tex +3 -0
+	an extension header.
+\end{itemsize}
+

With the inclusion of tunnel outer header, it doesn't need to redefine the hashing for inner packets which is exactly same without the tunnel. hash tunnel capability only indicates that hashing is done on the inner packets as_before.

This seems like a trade-off, and I can get rid of this if the simple statement "computing the hash for the inner header of the tunnel packet is the same as without tunnel" is also clear to the reader.
What do you think? Cc Jason and Michael.

Thanks.

Quoted 15 lines

This publicly archived list offers a means to provide input to the OASIS Virtual I/O Device (VIRTIO) TC.

In order to verify user consent to the Feedback License terms and to minimize spam in the list archive, subscription is required before posting.

Subscribe: [email protected]
Unsubscribe: [email protected]
List help: [email protected]
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/

#48 lore raw
Quoted 20 lines

From: Heng Qi <[email protected]>
Sent: Tuesday, January 31, 2023 12:57 AM

Quoted 9 lines · Parav Pandit
Quoted 9 lines

From: [email protected]
<[email protected]>
Sent: Wednesday, January 4, 2023 2:14 AM

If the tunnel is used to encapsulate the packets, the hash calculated using the outer header of the receive packets is always fixed for the same flow packets, i.e. they will be steered to the same

receive queue.

Quoted 4 lines
Quoted 4 lines

+\item[VIRTIO_NET_F_HASH_TUNNEL(52)] Device supports inner

content.tex +2 -0
+    header hash for GRE, VXLAN and GENEVE tunnel-encapsulated packets.
+
Quoted 8 lines
Quoted 3 lines

A device may not support all 3 at the same time.
Please remove mentioning tunneling protocols description from here. Just say device support inner header hash ...

Sorry for the late reply due to vacation.

Good idea, Michael suggested doing the same. But we also discussed this issue:
Early, we used a feature bit to force devices to support GRE and VXLAN

..
There is no need to force the device.
LM will work as it will use both he values - the feature bit and the supported/negotiated hash type in the config space.

Quoted 5 lines
Quoted 1 line

An additional bit map somewhere else should say supported hash over

different tunneling types.

Quoted 1 line

Yes, we use \field{supported_hash_types} to declare supported hash types.

Nice. So yes, just remove the description from the feature bits. [..]

Quoted 4 lines
Quoted 2 lines

With the inclusion of tunnel outer header, it doesn't need to redefine the

hashing for inner packets which is exactly same without the tunnel.

Quoted 1 line

hash tunnel capability only indicates that hashing is done on the inner packets

Quoted 8 lines

as_before.

Quoted 1 line

This seems like a trade-off, and I can get rid of this if the simple statement "computing the hash for the inner header of the tunnel packet is the same as without tunnel"
is also clear to the reader.
What do you think? Cc Jason and Michael.

Not sure I follow your comment.
Inclusion of outer header has zero change in inner header hash.

The data structure etc is just fine in the patch you proposed.
So, there is no need to redefine all over it again.
Only description need to be simplified as below.
Please just change the text wording to indicate that when tunnel feature is negotiated, and if the hashing is performed by the device on the inner header, hash_report_tunnel will contain the valid outer tunnel type information.

Also, this patch is adding two functionalities.
1. Inner header hash calculation of existing already defined hash types
2. outer header hash for new type for GRE,VXLAN,GENEVE.
#1 should be in 1st patch.
#2 should be in 2nd patch.
This is better to review.

Overall useful features. Thanks for adding.

This publicly archived list offers a means to provide input to the OASIS Virtual I/O Device (VIRTIO) TC.

In order to verify user consent to the Feedback License terms and to minimize spam in the list archive, subscription is required before posting.

Subscribe: [email protected]
Unsubscribe: [email protected]
List help: [email protected]
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/

#49 lore raw
Quoted 59 lines · Parav Pandit
Quoted 20 lines

From: Heng Qi <[email protected]>
Sent: Tuesday, January 31, 2023 12:57 AM

Quoted 9 lines · Parav Pandit
Quoted 9 lines

From: [email protected]
<[email protected]>
Sent: Wednesday, January 4, 2023 2:14 AM

If the tunnel is used to encapsulate the packets, the hash calculated using the outer header of the receive packets is always fixed for the same flow packets, i.e. they will be steered to the same

receive queue.

Quoted 4 lines
Quoted 4 lines

+\item[VIRTIO_NET_F_HASH_TUNNEL(52)] Device supports inner

content.tex +2 -0
+    header hash for GRE, VXLAN and GENEVE tunnel-encapsulated packets.
+
Quoted 8 lines
Quoted 3 lines

A device may not support all 3 at the same time.
Please remove mentioning tunneling protocols description from here. Just say device support inner header hash ...

Sorry for the late reply due to vacation.

Good idea, Michael suggested doing the same. But we also discussed this issue:
Early, we used a feature bit to force devices to support GRE and VXLAN

..
There is no need to force the device.
LM will work as it will use both he values - the feature bit and the supported/negotiated hash type in the config space.

Quoted 5 lines
Quoted 1 line

An additional bit map somewhere else should say supported hash over

different tunneling types.

Quoted 1 line

Yes, we use \field{supported_hash_types} to declare supported hash types.

Nice. So yes, just remove the description from the feature bits. [..]

Quoted 4 lines
Quoted 2 lines

With the inclusion of tunnel outer header, it doesn't need to redefine the

hashing for inner packets which is exactly same without the tunnel.

Quoted 1 line

hash tunnel capability only indicates that hashing is done on the inner packets

Quoted 8 lines

as_before.

Quoted 1 line

This seems like a trade-off, and I can get rid of this if the simple statement "computing the hash for the inner header of the tunnel packet is the same as without tunnel"
is also clear to the reader.
What do you think? Cc Jason and Michael.

Not sure I follow your comment.
Inclusion of outer header has zero change in inner header hash.

The data structure etc is just fine in the patch you proposed.
So, there is no need to redefine all over it again.
Only description need to be simplified as below.
Please just change the text wording to indicate that when tunnel feature is negotiated, and if the hashing is performed by the device on the inner header, hash_report_tunnel will contain the valid outer tunnel type information.

Quoted 6 lines

Also, this patch is adding two functionalities.
1. Inner header hash calculation of existing already defined hash types
2. outer header hash for new type for GRE,VXLAN,GENEVE.
#1 should be in 1st patch.
#2 should be in 2nd patch.
This is better to review.

Oh, that's an interesting take. Do you feel we need more separate hash type bits then?

Quoted 1 line

Overall useful features. Thanks for adding.

#50 lore raw
In reply to Parav Pandit
Quoted 59 lines · Parav Pandit
Quoted 20 lines

From: Heng Qi <[email protected]>
Sent: Tuesday, January 31, 2023 12:57 AM

Quoted 9 lines · Parav Pandit
Quoted 9 lines

From: [email protected]
<[email protected]>
Sent: Wednesday, January 4, 2023 2:14 AM

If the tunnel is used to encapsulate the packets, the hash calculated using the outer header of the receive packets is always fixed for the same flow packets, i.e. they will be steered to the same

receive queue.

Quoted 4 lines
Quoted 4 lines

+\item[VIRTIO_NET_F_HASH_TUNNEL(52)] Device supports inner

content.tex +2 -0
+    header hash for GRE, VXLAN and GENEVE tunnel-encapsulated packets.
+
Quoted 8 lines
Quoted 3 lines

A device may not support all 3 at the same time.
Please remove mentioning tunneling protocols description from here. Just say device support inner header hash ...

Sorry for the late reply due to vacation.

Good idea, Michael suggested doing the same. But we also discussed this issue:
Early, we used a feature bit to force devices to support GRE and VXLAN

..
There is no need to force the device.
LM will work as it will use both he values - the feature bit and the supported/negotiated hash type in the config space.

Quoted 5 lines
Quoted 1 line

An additional bit map somewhere else should say supported hash over

different tunneling types.

Quoted 1 line

Yes, we use \field{supported_hash_types} to declare supported hash types.

Nice. So yes, just remove the description from the feature bits. [..]

Quoted 4 lines
Quoted 2 lines

With the inclusion of tunnel outer header, it doesn't need to redefine the

hashing for inner packets which is exactly same without the tunnel.

Quoted 1 line

hash tunnel capability only indicates that hashing is done on the inner packets

Quoted 8 lines

as_before.

Quoted 1 line

This seems like a trade-off, and I can get rid of this if the simple statement "computing the hash for the inner header of the tunnel packet is the same as without tunnel"
is also clear to the reader.
What do you think? Cc Jason and Michael.

Not sure I follow your comment.
Inclusion of outer header has zero change in inner header hash.

The data structure etc is just fine in the patch you proposed.
So, there is no need to redefine all over it again.
Only description need to be simplified as below.
Please just change the text wording to indicate that when tunnel feature is negotiated, and if the hashing is performed by the device on the inner header, hash_report_tunnel will contain the valid outer tunnel type information.

Quoted 6 lines

Also, this patch is adding two functionalities.
1. Inner header hash calculation of existing already defined hash types
2. outer header hash for new type for GRE,VXLAN,GENEVE.
#1 should be in 1st patch.
#2 should be in 2nd patch.
This is better to review.

Parav, you come to this discussion pretty late. Asking to split up the patch when it's v1/v2 is ok. Asking after others have already reviewed v6 is not you are making review easier for yourself but re-review harder for others who already have a mind map of the patch.

In this case unless we really want to enable these separately (and frankly I don't see a good reason to) then splitting it up makes review more confusing.

Quoted 1 line

Overall useful features. Thanks for adding.

#51 lore raw
Quoted 12 lines

From: [email protected] <[email protected] open.org> On Behalf Of Michael S. Tsirkin
Sent: Wednesday, February 1, 2023 1:57 AM

Quoted 6 lines

Also, this patch is adding two functionalities.
1. Inner header hash calculation of existing already defined hash types 2. outer header hash for new type for GRE,VXLAN,GENEVE. #1 should be in 1st patch.
#2 should be in 2nd patch.
This is better to review.

Quoted 8 lines

Parav, you come to this discussion pretty late. Asking to split up the patch when it's v1/v2 is ok. Asking after others have already reviewed v6 is not you are making review easier for yourself but re-review harder for others who already have a mind map of the patch.

In this case unless we really want to enable these separately (and frankly I don't see a good reason to) then splitting it up makes review more confusing.

No. There is no need to enable it separately.
It was hard to parse new inner type decoding addition which has close to zero relation to outer headers.

As you say, it has some history, I don't have strong opinion to split.
But going forward in subsequent work, it is better to see logical changes in multiple patches.

author
#52 lore raw
Quoted 14 lines · Parav Pandit
Quoted 14 lines

From: [email protected] <[email protected] open.org> On Behalf Of Michael S. Tsirkin
Sent: Wednesday, February 1, 2023 1:57 AM

Quoted 6 lines

Also, this patch is adding two functionalities.
1. Inner header hash calculation of existing already defined hash types 2. outer header hash for new type for GRE,VXLAN,GENEVE. #1 should be in 1st patch.
#2 should be in 2nd patch.
This is better to review.

Parav, you come to this discussion pretty late. Asking to split up the patch when it's v1/v2 is ok. Asking after others have already reviewed v6 is not you are making review easier for yourself but re-review harder for others who already have a mind map of the patch.

Quoted 8 lines
Quoted 3 lines

In this case unless we really want to enable these separately (and frankly I don't see a good reason to) then splitting it up makes review more confusing.

No. There is no need to enable it separately.
It was hard to parse new inner type decoding addition which has close to zero relation to outer headers.

As you say, it has some history, I don't have strong opinion to split.
But going forward in subsequent work, it is better to see logical changes in multiple patches.

It seems that we don't need to emphasize the outer header hash, which is the same behavior as usual when the inner header hash is not raised.

Thanks.

This publicly archived list offers a means to provide input to the OASIS Virtual I/O Device (VIRTIO) TC.

In order to verify user consent to the Feedback License terms and to minimize spam in the list archive, subscription is required before posting.

Subscribe: [email protected]
Unsubscribe: [email protected]
List help: [email protected]
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/

author
#53 lore raw
In reply to Parav Pandit
Quoted 24 lines · Parav Pandit
Quoted 24 lines

From: Heng Qi <[email protected]>
Sent: Tuesday, January 31, 2023 12:57 AM

Quoted 9 lines · Parav Pandit
Quoted 9 lines

From: [email protected]
<[email protected]>
Sent: Wednesday, January 4, 2023 2:14 AM

If the tunnel is used to encapsulate the packets, the hash calculated using the outer header of the receive packets is always fixed for the same flow packets, i.e. they will be steered to the same

receive queue.

Quoted 7 lines

content.tex \subsection{Feature bits}\label{sec:Device Types / Network Device / Feature bits · line 3089

Quoted 4 lines

+\item[VIRTIO_NET_F_HASH_TUNNEL(52)] Device supports inner

content.tex +2 -0
+    header hash for GRE, VXLAN and GENEVE tunnel-encapsulated packets.
+

A device may not support all 3 at the same time.
Please remove mentioning tunneling protocols description from here. Just say device support inner header hash ...

Quoted 7 lines
Quoted 4 lines

Sorry for the late reply due to vacation.

Good idea, Michael suggested doing the same. But we also discussed this issue:
Early, we used a feature bit to force devices to support GRE and VXLAN

..
There is no need to force the device.
LM will work as it will use both he values - the feature bit and the supported/negotiated hash type in the config space.

Yes, that's exactly what we do now.

Quoted 7 lines
Quoted 6 lines
Quoted 2 lines

An additional bit map somewhere else should say supported hash over

different tunneling types.

Quoted 1 line

Yes, we use \field{supported_hash_types} to declare supported hash types.

Nice. So yes, just remove the description from the feature bits.

The next version will see this modification.

Quoted 13 lines

[..]

Quoted 4 lines
Quoted 2 lines

With the inclusion of tunnel outer header, it doesn't need to redefine the

hashing for inner packets which is exactly same without the tunnel.

Quoted 1 line

hash tunnel capability only indicates that hashing is done on the inner packets

Quoted 8 lines

as_before.

Quoted 1 line

This seems like a trade-off, and I can get rid of this if the simple statement "computing the hash for the inner header of the tunnel packet is the same as without tunnel"
is also clear to the reader.
What do you think? Cc Jason and Michael.

Quoted 8 lines

Not sure I follow your comment.
Inclusion of outer header has zero change in inner header hash.

The data structure etc is just fine in the patch you proposed.
So, there is no need to redefine all over it again.
Only description need to be simplified as below.
Please just change the text wording to indicate that when tunnel feature is negotiated, and if the hashing is performed by the device on the inner header, hash_report_tunnel will contain the valid outer tunnel type information.

OK, I'll take your advice and make appropriate changes.

Thanks.

Quoted 8 lines

Also, this patch is adding two functionalities.
1. Inner header hash calculation of existing already defined hash types
2. outer header hash for new type for GRE,VXLAN,GENEVE.
#1 should be in 1st patch.
#2 should be in 2nd patch.
This is better to review.

Overall useful features. Thanks for adding.

This publicly archived list offers a means to provide input to the OASIS Virtual I/O Device (VIRTIO) TC.

In order to verify user consent to the Feedback License terms and to minimize spam in the list archive, subscription is required before posting.

Subscribe: [email protected]
Unsubscribe: [email protected]
List help: [email protected]
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/