Skip to contentSkip to conversation

Introducing RSS virtio-net

PATCH Superseded Sameeh Jubran 3 participants 15 messages Apr 16, 2018 – Apr 18, 2018

v2 of 5 Superseded Latest is v5 →

lore
Share & tools
lore

Sameeh Jubran

author
#0 lore raw

v2 cover Introducing RSS virtio-net

From: Sameeh Jubran <[email protected]>

Hi all,

This patch series introduces an infrastructure into virtio-net for adding steering modes while introducing RSS as one.

Please take a look and share your thoughts.

Difference from v1:

* generic steering mode introduced and RSS is defined as part of the steering feature by Jason request

v1:
https://lists.oasis-open.org/archives/virtio-dev/201801/msg00177.html

Sameeh Jubran (2):
content: net: Add VIRTIO_NET_F_CTRL_STEERING_MODE feature
content: net: steering mode: Add RSS

 content.tex | 173 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 173 insertions(+)
Signature · 6 lines
2.13.6


---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
author
#1 lore raw

PATCH v2 1/2 content: net: Add VIRTIO_NET_F_CTRL_STEERING_MODE feature

From: Sameeh Jubran <[email protected]>

This commit introduces steering mode into network device. Steering mode is a general infrastructure for various steering modes that can be implemented on top of it such as Automatic and RSS.

Signed-off-by Sameeh Jubran <[email protected]>
---
 content.tex | 59 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 59 insertions(+)
content.tex +59 -0
index c840588..3d538e8 100644
@@ -3115,6 +3115,9 @@ features.
 
 \item[VIRTIO_NET_F_CTRL_MAC_ADDR(23)] Set MAC address through control
     channel.
+
+\item[VIRTIO_NET_F_CTRL_STEERING_MODE(27)] Device supports configurable steering
+    mode.
 \end{description}
 
 \subsubsection{Feature bit requirements}\label{sec:Device Types / Network Device / Feature bits / Feature bit requirements}

@@ -3317,6 +3320,8 @@ struct virtio_net_hdr {
         le16 csum_start;
         le16 csum_offset;
         le16 num_buffers;
+// Only if VIRTIO_NET_F_CTRL_STEERING_MODE has been negotiated
+        le64 hash;
 };
 \end{lstlisting}
 

@@ -4007,6 +4012,60 @@ VIRTIO_NET_CTRL_MQ_VQ_PAIRS_SET command.
 The device MUST NOT queue packets on receive queues greater than
 \field{virtqueue_pairs} once it has placed the VIRTIO_NET_CTRL_MQ_VQ_PAIRS_SET command in the used ring.
 
+\paragraph{Steering mode}\label{sec:Device Types / Network Device / Device Operation / Control Virtqueue / Steering mode}
+
+\begin{lstlisting}
+// steering mode flags
+#define STEERING_MODE_AUTO          1
+
+struct virtio_net_steering_modes {
+le32 steering_modes;
+};
+
+struct virtio_net_steering_mode {
+le32 steering_mode;
+le32 command;
+
+    union {
+    }
+};
+
+#define VIRTIO_NET_F_CTRL_STEERING_MODE     27
+
+#define VIRTIO_NET_CTRL_STEERING_MODE                7
+#define VIRTIO_NET_CTRL_SM_GET_SUPPORTED_MODES        0
+#define VIRTIO_NET_CTRL_SM_CONTROL                    1
+\end{lstlisting}
+
+If the VIRTIO_NET_F_CTRL_STEERING_MODE is negotiated the driver can send control commands for the
+configuring the steering mode. There are multiple steering modes and they can be configured using
+the VIRTIO_NET_CTRL_SM_CONTROL command. Each mode has it's own set of commands.
+
+The auto steering mode is the default mode if nothing else has been configured by the driver
+and the VIRTIO_NET_F_CTRL_STEERING_MODE feature is acked by the driver.
+
+The class VIRTIO_NET_CTRL_STEERING_MODE has two commands:
+
+\begin{enumerate}
+
+\item VIRTIO_NET_CTRL_SM_GET_SUPPORTED_MODES is used for getting the supported steering modes by the device.
+	The command should be executed by the driver before attempting to configure the steering mode. Once the device
+	receives this command it should fill the virtio_net_steering_modes structure with the supported steering mode
+	flags.
+
+\item The command VIRTIO_NET_CTRL_SM_CONTROL is used for controlling the different steering modes. Each steering mode
+	has its own set of commands. When executing this command the structure virtio_net_steering_mode should be used as follows:
+	the \field{steering_mode} should be filled with one of the steering mode flags along with an appropriate command from the chosen
+	steering mode. The union of virtio_net_steering_mode should be used and filled as the mode's command suggests.
+\end{enumerate}
+
+If this feature has been negotiated, the virtio header has an additional
+\field{hash} field attached to it.
+
+\subparagraph{Automatic Receive Steering}{Device Types / Network Device / Device Operation / Control Virtqueue / Steering mode / Automatic Receive Steering}
+
+This is the default steering mode, please refer to the "Automatic receive steering in multiqueue" section.
+
 \subparagraph{Legacy Interface: Automatic receive steering in multiqueue mode}\label{sec:Device Types / Network Device / Device Operation / Control Virtqueue / Automatic receive steering in multiqueue mode / Legacy Interface: Automatic receive steering in multiqueue mode}
 When using the legacy interface, transitional devices and drivers
 MUST format \field{virtqueue_pairs}
Signature · 6 lines
2.13.6


---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
#2 lore raw

Re: [virtio-dev] [PATCH v2 1/2] content: net: Add VIRTIO_NET_F_CTRL_STEERING_MODE feature

Quoted 47 lines · Sameeh Jubran

From: Sameeh Jubran <[email protected]>

This commit introduces steering mode into network device. Steering mode is a general infrastructure for various steering modes that can be implemented on top of it such as Automatic and RSS.

Signed-off-by Sameeh Jubran <[email protected]>
---
 content.tex | 59 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 59 insertions(+)
content.tex +14 -0
index c840588..3d538e8 100644
@@ -3115,6 +3115,9 @@ features.
 
 \item[VIRTIO_NET_F_CTRL_MAC_ADDR(23)] Set MAC address through control
     channel.
+
+\item[VIRTIO_NET_F_CTRL_STEERING_MODE(27)] Device supports configurable steering
+    mode.
 \end{description}
 
 \subsubsection{Feature bit requirements}\label{sec:Device Types / Network Device / Feature bits / Feature bit requirements}

@@ -3317,6 +3320,8 @@ struct virtio_net_hdr {
         le16 csum_start;
         le16 csum_offset;
         le16 num_buffers;
+// Only if VIRTIO_NET_F_CTRL_STEERING_MODE has been negotiated
+        le64 hash;
 };
 \end{lstlisting}
 

@@ -4007,6 +4012,60 @@ VIRTIO_NET_CTRL_MQ_VQ_PAIRS_SET command.
 The device MUST NOT queue packets on receive queues greater than
 \field{virtqueue_pairs} once it has placed the VIRTIO_NET_CTRL_MQ_VQ_PAIRS_SET command in the used ring.
 
+\paragraph{Steering mode}\label{sec:Device Types / Network Device / Device Operation / Control Virtqueue / Steering mode}
+
+\begin{lstlisting}
+// steering mode flags
+#define STEERING_MODE_AUTO          1
+
+struct virtio_net_steering_modes {
+le32 steering_modes;
+};

content.tex VIRTIO_NET_CTRL_MQ_VQ_PAIRS_SET command. · line 4033

Quoted 10 lines
content.tex +10 -0
+
+struct virtio_net_steering_mode {
+le32 steering_mode;
+le32 command;
+
+    union {
+    }
+};
+
+#define VIRTIO_NET_F_CTRL_STEERING_MODE     27

Please use a high number (63 and down). We are running out of low numbers it is best to keep them for things we must backport to old virtio - such as mtu - which are required for the device working properly.

Pls coordinate with Vlad (cc'd) who is adding a new feature too.

Quoted 31 lines
content.tex +29 -0
+
+#define VIRTIO_NET_CTRL_STEERING_MODE                7
+#define VIRTIO_NET_CTRL_SM_GET_SUPPORTED_MODES        0
+#define VIRTIO_NET_CTRL_SM_CONTROL                    1
+\end{lstlisting}
+
+If the VIRTIO_NET_F_CTRL_STEERING_MODE is negotiated the driver can send control commands for the
+configuring the steering mode. There are multiple steering modes and they can be configured using
+the VIRTIO_NET_CTRL_SM_CONTROL command. Each mode has it's own set of commands.
+
+The auto steering mode is the default mode if nothing else has been configured by the driver
+and the VIRTIO_NET_F_CTRL_STEERING_MODE feature is acked by the driver.
+
+The class VIRTIO_NET_CTRL_STEERING_MODE has two commands:
+
+\begin{enumerate}
+
+\item VIRTIO_NET_CTRL_SM_GET_SUPPORTED_MODES is used for getting the supported steering modes by the device.
+	The command should be executed by the driver before attempting to configure the steering mode. Once the device
+	receives this command it should fill the virtio_net_steering_modes structure with the supported steering mode
+	flags.
+
+\item The command VIRTIO_NET_CTRL_SM_CONTROL is used for controlling the different steering modes. Each steering mode
+	has its own set of commands. When executing this command the structure virtio_net_steering_mode should be used as follows:
+	the \field{steering_mode} should be filled with one of the steering mode flags along with an appropriate command from the chosen
+	steering mode. The union of virtio_net_steering_mode should be used and filled as the mode's command suggests.
+\end{enumerate}
+
+If this feature has been negotiated, the virtio header

has an additional
+\field{hash} field attached to it.

content.tex VIRTIO_NET_CTRL_MQ_VQ_PAIRS_SET command. · line 4071

Quoted 8 lines
content.tex +5 -0
+
+\subparagraph{Automatic Receive Steering}{Device Types / Network Device / Device Operation / Control Virtqueue / Steering mode / Automatic Receive Steering}
+
+This is the default steering mode, please refer to the "Automatic receive steering in multiqueue" section.
+
 \subparagraph{Legacy Interface: Automatic receive steering in multiqueue mode}\label{sec:Device Types / Network Device / Device Operation / Control Virtqueue / Automatic receive steering in multiqueue mode / Legacy Interface: Automatic receive steering in multiqueue mode}
 When using the legacy interface, transitional devices and drivers
 MUST format \field{virtqueue_pairs}

Please add each part where it belongs, e.g. a feature bit with other feature bits, header field with other header fields, etc.

I think Vlad tested extending the header and found it adds non-negligeable performance overhead. Is it worth splitting out hash calculation? Is hash used for RX path only?

Quoted 7 lines
Signature · 6 lines
2.13.6


---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]

author
#3 lore raw

Re: [virtio-dev] [PATCH v2 1/2] content: net: Add VIRTIO_NET_F_CTRL_STEERING_MODE feature

Quoted 60 lines · Michael S. Tsirkin
Quoted 9 lines · Sameeh Jubran

From: Sameeh Jubran <[email protected]>

This commit introduces steering mode into network device. Steering mode is a general infrastructure for various steering modes that can be implemented on top of it such as Automatic and RSS.

Signed-off-by Sameeh Jubran <[email protected]>
---
 content.tex | 59 ++++++++++++++++++++++++++++++

+++++++++++++++++++++++++++++

content.tex features. · line 3119

Quoted 12 lines
 1 file changed, 59 insertions(+)
content.tex +2 -0
index c840588..3d538e8 100644
@@ -3115,6 +3115,9 @@ features.
 
 \item[VIRTIO_NET_F_CTRL_MAC_ADDR(23)] Set MAC address through control
     channel.
+
+\item[VIRTIO_NET_F_CTRL_STEERING_MODE(27)] Device supports

configurable steering

content.tex features.

Quoted 4 lines
content.tex +1 -0
+    mode.
 \end{description}
 
 \subsubsection{Feature bit requirements}\label{sec:Device Types /

Network Device / Feature bits / Feature bit requirements}

content.tex VIRTIO_NET_CTRL_MQ_VQ_PAIRS_SET command. · line 4013

Quoted 12 lines
content.tex +2 -0
@@ -3317,6 +3320,8 @@ struct virtio_net_hdr {
         le16 csum_start;
         le16 csum_offset;
         le16 num_buffers;
+// Only if VIRTIO_NET_F_CTRL_STEERING_MODE has been negotiated
+        le64 hash;
 };
 \end{lstlisting}
 

@@ -4007,6 +4012,60 @@ VIRTIO_NET_CTRL_MQ_VQ_PAIRS_SET command.
 The device MUST NOT queue packets on receive queues greater than
 \field{virtqueue_pairs} once it has placed the

VIRTIO_NET_CTRL_MQ_VQ_PAIRS_SET command in the used ring.

content.tex VIRTIO_NET_CTRL_MQ_VQ_PAIRS_SET command. · line 4014

Quoted 2 lines
content.tex +1 -0
+\paragraph{Steering mode}\label{sec:Device Types / Network Device /

Device Operation / Control Virtqueue / Steering mode}

Quoted 15 lines
content.tex +15 -0
+
+\begin{lstlisting}
+// steering mode flags
+#define STEERING_MODE_AUTO          1
+
+struct virtio_net_steering_modes {
+le32 steering_modes;
+};
+
+struct virtio_net_steering_mode {
+le32 steering_mode;
+le32 command;
+
+    union {
+    }
Quoted 9 lines

content.tex VIRTIO_NET_CTRL_MQ_VQ_PAIRS_SET command. · line 4033

Quoted 3 lines
content.tex +3 -0
+};
+
+#define VIRTIO_NET_F_CTRL_STEERING_MODE     27

Please use a high number (63 and down). We are running out of low numbers it is best to keep them for things we must backport to old virtio - such as mtu - which are required for the device working properly.

Will do.

Quoted 54 lines

Pls coordinate with Vlad (cc'd) who is adding a new feature too.

content.tex VIRTIO_NET_CTRL_MQ_VQ_PAIRS_SET command.

Quoted 7 lines
content.tex +7 -0
+
+#define VIRTIO_NET_CTRL_STEERING_MODE                7
+#define VIRTIO_NET_CTRL_SM_GET_SUPPORTED_MODES        0
+#define VIRTIO_NET_CTRL_SM_CONTROL                    1
+\end{lstlisting}
+
+If the VIRTIO_NET_F_CTRL_STEERING_MODE is negotiated the driver can

send control commands for the

content.tex VIRTIO_NET_CTRL_MQ_VQ_PAIRS_SET command. · line 4040

Quoted 1 line
content.tex +1 -0
+configuring the steering mode. There are multiple steering modes and

they can be configured using

content.tex VIRTIO_NET_CTRL_MQ_VQ_PAIRS_SET command. · line 4041

Quoted 1 line
content.tex +1 -0
+the VIRTIO_NET_CTRL_SM_CONTROL command. Each mode has it's own set of

commands.

content.tex VIRTIO_NET_CTRL_MQ_VQ_PAIRS_SET command. · line 4043

Quoted 2 lines
content.tex +2 -0
+
+The auto steering mode is the default mode if nothing else has been

configured by the driver

content.tex VIRTIO_NET_CTRL_MQ_VQ_PAIRS_SET command.

Quoted 7 lines
content.tex +7 -0
+and the VIRTIO_NET_F_CTRL_STEERING_MODE feature is acked by the driver.
+
+The class VIRTIO_NET_CTRL_STEERING_MODE has two commands:
+
+\begin{enumerate}
+
+\item VIRTIO_NET_CTRL_SM_GET_SUPPORTED_MODES is used for getting the

supported steering modes by the device.

content.tex VIRTIO_NET_CTRL_MQ_VQ_PAIRS_SET command. · line 4051

Quoted 1 line
content.tex +1 -0
+     The command should be executed by the driver before attempting to

configure the steering mode. Once the device

content.tex VIRTIO_NET_CTRL_MQ_VQ_PAIRS_SET command. · line 4052

Quoted 1 line
content.tex +1 -0
+     receives this command it should fill the virtio_net_steering_modes

structure with the supported steering mode

content.tex VIRTIO_NET_CTRL_MQ_VQ_PAIRS_SET command.

Quoted 3 lines
content.tex +3 -0
+     flags.
+
+\item The command VIRTIO_NET_CTRL_SM_CONTROL is used for controlling

the different steering modes. Each steering mode

content.tex VIRTIO_NET_CTRL_MQ_VQ_PAIRS_SET command. · line 4056

Quoted 1 line
content.tex +1 -0
+     has its own set of commands. When executing this command the

structure virtio_net_steering_mode should be used as follows:

content.tex VIRTIO_NET_CTRL_MQ_VQ_PAIRS_SET command. · line 4057

Quoted 1 line
content.tex +1 -0
+     the \field{steering_mode} should be filled with one of the

steering mode flags along with an appropriate command from the chosen

content.tex VIRTIO_NET_CTRL_MQ_VQ_PAIRS_SET command. · line 4058

Quoted 1 line
content.tex +1 -0
+     steering mode. The union of virtio_net_steering_mode should be

used and filled as the mode's command suggests.

Quoted 7 lines
content.tex +3 -0
+\end{enumerate}
+
+If this feature has been negotiated, the virtio header

has an additional
+\field{hash} field attached to it.
+
+\subparagraph{Automatic Receive Steering}{Device Types / Network Device

/ Device Operation / Control Virtqueue / Steering mode / Automatic Receive Steering}

content.tex

Quoted 2 lines
content.tex +2 -0
+
+This is the default steering mode, please refer to the "Automatic

receive steering in multiqueue" section.

Quoted 2 lines

+
\subparagraph{Legacy Interface: Automatic receive steering in

Quoted 9 lines

multiqueue mode}\label{sec:Device Types / Network Device / Device Operation / Control Virtqueue / Automatic receive steering in multiqueue mode / Legacy Interface: Automatic receive steering in multiqueue mode}

Quoted 2 lines

When using the legacy interface, transitional devices and drivers MUST format \field{virtqueue_pairs}

Please add each part where it belongs, e.g. a feature bit with other feature bits, header field with other header fields, etc.

What are you referring to specifically? Can you please elaborate?

Quoted 5 lines

I think Vlad tested extending the header and found it adds non-negligeable performance overhead. Is it worth splitting out hash calculation? Is hash used for RX path only?

Hash is used for RX only and I can't think this is not avoidable. We need to inform the driver what is the hash value for the packet, it's a Windows requirement. What do you mean by splitting out the hash calculation?
We are already extending the virtio-header for RSC feature which didn't make it into spec somehow.

Quoted 10 lines
Quoted 9 lines

--
2.13.6

---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]

Signature · 4 lines
Respectfully,
*Sameeh Jubran*
*Linkedin <https://il.linkedin.com/pub/sameeh-jubran/87/747/a8a>*
*Software Engineer @ Daynix <http://www.daynix.com>.*
author
#4 lore raw

Re: [virtio-dev] [PATCH v2 1/2] content: net: Add VIRTIO_NET_F_CTRL_STEERING_MODE feature

Quoted 143 lines · Sameeh Jubran
Quoted 60 lines · Michael S. Tsirkin
Quoted 9 lines · Sameeh Jubran

From: Sameeh Jubran <[email protected]>

This commit introduces steering mode into network device. Steering mode is a general infrastructure for various steering modes that can be implemented on top of it such as Automatic and RSS.

Signed-off-by Sameeh Jubran <[email protected]>
---
 content.tex | 59 ++++++++++++++++++++++++++++++

+++++++++++++++++++++++++++++

content.tex features. · line 3119

Quoted 12 lines
 1 file changed, 59 insertions(+)
content.tex +2 -0
index c840588..3d538e8 100644
@@ -3115,6 +3115,9 @@ features.
 
 \item[VIRTIO_NET_F_CTRL_MAC_ADDR(23)] Set MAC address through control
     channel.
+
+\item[VIRTIO_NET_F_CTRL_STEERING_MODE(27)] Device supports

configurable steering

content.tex features.

Quoted 4 lines
content.tex +1 -0
+    mode.
 \end{description}
 
 \subsubsection{Feature bit requirements}\label{sec:Device Types /

Network Device / Feature bits / Feature bit requirements}

content.tex VIRTIO_NET_CTRL_MQ_VQ_PAIRS_SET command. · line 4013

Quoted 12 lines
content.tex +2 -0
@@ -3317,6 +3320,8 @@ struct virtio_net_hdr {
         le16 csum_start;
         le16 csum_offset;
         le16 num_buffers;
+// Only if VIRTIO_NET_F_CTRL_STEERING_MODE has been negotiated
+        le64 hash;
 };
 \end{lstlisting}
 

@@ -4007,6 +4012,60 @@ VIRTIO_NET_CTRL_MQ_VQ_PAIRS_SET command.
 The device MUST NOT queue packets on receive queues greater than
 \field{virtqueue_pairs} once it has placed the

VIRTIO_NET_CTRL_MQ_VQ_PAIRS_SET command in the used ring.

content.tex VIRTIO_NET_CTRL_MQ_VQ_PAIRS_SET command. · line 4014

Quoted 2 lines
content.tex +1 -0
+\paragraph{Steering mode}\label{sec:Device Types / Network Device /

Device Operation / Control Virtqueue / Steering mode}

Quoted 15 lines
content.tex +15 -0
+
+\begin{lstlisting}
+// steering mode flags
+#define STEERING_MODE_AUTO          1
+
+struct virtio_net_steering_modes {
+le32 steering_modes;
+};
+
+struct virtio_net_steering_mode {
+le32 steering_mode;
+le32 command;
+
+    union {
+    }
Quoted 9 lines

content.tex VIRTIO_NET_CTRL_MQ_VQ_PAIRS_SET command. · line 4033

Quoted 3 lines
content.tex +3 -0
+};
+
+#define VIRTIO_NET_F_CTRL_STEERING_MODE     27

Please use a high number (63 and down). We are running out of low numbers it is best to keep them for things we must backport to old virtio - such as mtu - which are required for the device working properly.

Will do.

Quoted 55 lines

Pls coordinate with Vlad (cc'd) who is adding a new feature too.

content.tex VIRTIO_NET_CTRL_MQ_VQ_PAIRS_SET command.

Quoted 7 lines
content.tex +7 -0
+
+#define VIRTIO_NET_CTRL_STEERING_MODE                7
+#define VIRTIO_NET_CTRL_SM_GET_SUPPORTED_MODES        0
+#define VIRTIO_NET_CTRL_SM_CONTROL                    1
+\end{lstlisting}
+
+If the VIRTIO_NET_F_CTRL_STEERING_MODE is negotiated the driver can

send control commands for the

content.tex VIRTIO_NET_CTRL_MQ_VQ_PAIRS_SET command. · line 4040

Quoted 1 line
content.tex +1 -0
+configuring the steering mode. There are multiple steering modes and

they can be configured using

content.tex VIRTIO_NET_CTRL_MQ_VQ_PAIRS_SET command. · line 4041

Quoted 1 line
content.tex +1 -0
+the VIRTIO_NET_CTRL_SM_CONTROL command. Each mode has it's own set of

commands.

content.tex VIRTIO_NET_CTRL_MQ_VQ_PAIRS_SET command. · line 4043

Quoted 2 lines
content.tex +2 -0
+
+The auto steering mode is the default mode if nothing else has been

configured by the driver

content.tex VIRTIO_NET_CTRL_MQ_VQ_PAIRS_SET command. · line 4044

Quoted 1 line
content.tex +1 -0
+and the VIRTIO_NET_F_CTRL_STEERING_MODE feature is acked by the

driver.

content.tex VIRTIO_NET_CTRL_MQ_VQ_PAIRS_SET command.

Quoted 6 lines
content.tex +6 -0
+
+The class VIRTIO_NET_CTRL_STEERING_MODE has two commands:
+
+\begin{enumerate}
+
+\item VIRTIO_NET_CTRL_SM_GET_SUPPORTED_MODES is used for getting the

supported steering modes by the device.

content.tex VIRTIO_NET_CTRL_MQ_VQ_PAIRS_SET command. · line 4051

Quoted 1 line
content.tex +1 -0
+     The command should be executed by the driver before attempting to

configure the steering mode. Once the device

content.tex VIRTIO_NET_CTRL_MQ_VQ_PAIRS_SET command. · line 4052

Quoted 1 line
content.tex +1 -0
+     receives this command it should fill the

virtio_net_steering_modes structure with the supported steering mode

content.tex VIRTIO_NET_CTRL_MQ_VQ_PAIRS_SET command.

Quoted 3 lines
content.tex +3 -0
+     flags.
+
+\item The command VIRTIO_NET_CTRL_SM_CONTROL is used for controlling

the different steering modes. Each steering mode

content.tex VIRTIO_NET_CTRL_MQ_VQ_PAIRS_SET command. · line 4056

Quoted 1 line
content.tex +1 -0
+     has its own set of commands. When executing this command the

structure virtio_net_steering_mode should be used as follows:

content.tex VIRTIO_NET_CTRL_MQ_VQ_PAIRS_SET command. · line 4057

Quoted 1 line
content.tex +1 -0
+     the \field{steering_mode} should be filled with one of the

steering mode flags along with an appropriate command from the chosen

content.tex VIRTIO_NET_CTRL_MQ_VQ_PAIRS_SET command. · line 4058

Quoted 1 line
content.tex +1 -0
+     steering mode. The union of virtio_net_steering_mode should be

used and filled as the mode's command suggests.

Quoted 7 lines
content.tex +3 -0
+\end{enumerate}
+
+If this feature has been negotiated, the virtio header

has an additional
+\field{hash} field attached to it.
+
+\subparagraph{Automatic Receive Steering}{Device Types / Network

Device / Device Operation / Control Virtqueue / Steering mode / Automatic Receive Steering}

content.tex

Quoted 2 lines
content.tex +2 -0
+
+This is the default steering mode, please refer to the "Automatic

receive steering in multiqueue" section.

Quoted 2 lines

+
\subparagraph{Legacy Interface: Automatic receive steering in

Quoted 9 lines

multiqueue mode}\label{sec:Device Types / Network Device / Device Operation / Control Virtqueue / Automatic receive steering in multiqueue mode / Legacy Interface: Automatic receive steering in multiqueue mode}

Quoted 2 lines

When using the legacy interface, transitional devices and drivers MUST format \field{virtqueue_pairs}

Please add each part where it belongs, e.g. a feature bit with other feature bits, header field with other header fields, etc.

What are you referring to specifically? Can you please elaborate?

Quoted 1 line
Quoted 7 lines
Quoted 4 lines

I think Vlad tested extending the header and found it adds non-negligeable performance overhead. Is it worth splitting out hash calculation? Is hash used for RX path only?

Hash is used for RX only and I can't think this is not avoidable. We need to inform the driver what is the hash value for the packet,

Just to clarify, Windows does provide us with hash for the packets so we can forward them to the device using the hash field in the virtio header. This value can be used to perform optimizations by the backend.

Quoted 24 lines

it's a Windows requirement. What do you mean by splitting out the hash calculation?
We are already extending the virtio-header for RSC feature which didn't make it into spec somehow.

Quoted 10 lines
Quoted 9 lines

--
2.13.6

---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]

--
Respectfully,
*Sameeh Jubran*
*Linkedin <https://il.linkedin.com/pub/sameeh-jubran/87/747/a8a>*
*Software Engineer @ Daynix <http://www.daynix.com>.*

Signature · 4 lines
Respectfully,
*Sameeh Jubran*
*Linkedin <https://il.linkedin.com/pub/sameeh-jubran/87/747/a8a>*
*Software Engineer @ Daynix <http://www.daynix.com>.*
#5 lore raw

[virtio-dev] Re: [PATCH v2 1/2] content: net: Add VIRTIO_NET_F_CTRL_STEERING_MODE feature

In reply to Sameeh Jubran
Quoted 35 lines · Sameeh Jubran

From: Sameeh Jubran <[email protected]>

This commit introduces steering mode into network device. Steering mode is a general infrastructure for various steering modes that can be implemented on top of it such as Automatic and RSS.

Signed-off-by Sameeh Jubran <[email protected]>
---
  content.tex | 59 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
  1 file changed, 59 insertions(+)
content.tex +5 -0
index c840588..3d538e8 100644
@@ -3115,6 +3115,9 @@ features.
 
  \item[VIRTIO_NET_F_CTRL_MAC_ADDR(23)] Set MAC address through control
      channel.
+
+\item[VIRTIO_NET_F_CTRL_STEERING_MODE(27)] Device supports configurable steering
+    mode.
  \end{description}
 
  \subsubsection{Feature bit requirements}\label{sec:Device Types / Network Device / Feature bits / Feature bit requirements}

@@ -3317,6 +3320,8 @@ struct virtio_net_hdr {
          le16 csum_start;
          le16 csum_offset;
          le16 num_buffers;
+// Only if VIRTIO_NET_F_CTRL_STEERING_MODE has been negotiated
+        le64 hash;
  };
  \end{lstlisting}

content.tex VIRTIO_NET_CTRL_MQ_VQ_PAIRS_SET command. · line 4019

Quoted 8 lines
content.tex +5 -0
@@ -4007,6 +4012,60 @@ VIRTIO_NET_CTRL_MQ_VQ_PAIRS_SET command.
  The device MUST NOT queue packets on receive queues greater than
  \field{virtqueue_pairs} once it has placed the VIRTIO_NET_CTRL_MQ_VQ_PAIRS_SET command in the used ring.
 
+\paragraph{Steering mode}\label{sec:Device Types / Network Device / Device Operation / Control Virtqueue / Steering mode}
+
+\begin{lstlisting}
+// steering mode flags
+#define STEERING_MODE_AUTO          1

As steering mode flags is a bit field, should it be 0x1?

Quoted 34 lines
content.tex +34 -0
+
+struct virtio_net_steering_modes {
+le32 steering_modes;
+};
+
+struct virtio_net_steering_mode {
+le32 steering_mode;
+le32 command;
+
+    union {
+    }
+};
+
+#define VIRTIO_NET_F_CTRL_STEERING_MODE     27
+
+#define VIRTIO_NET_CTRL_STEERING_MODE                7
+#define VIRTIO_NET_CTRL_SM_GET_SUPPORTED_MODES        0
+#define VIRTIO_NET_CTRL_SM_CONTROL                    1
+\end{lstlisting}
+
+If the VIRTIO_NET_F_CTRL_STEERING_MODE is negotiated the driver can send control commands for the
+configuring the steering mode. There are multiple steering modes and they can be configured using
+the VIRTIO_NET_CTRL_SM_CONTROL command. Each mode has it's own set of commands.
+
+The auto steering mode is the default mode if nothing else has been configured by the driver
+and the VIRTIO_NET_F_CTRL_STEERING_MODE feature is acked by the driver.
+
+The class VIRTIO_NET_CTRL_STEERING_MODE has two commands:
+
+\begin{enumerate}
+
+\item VIRTIO_NET_CTRL_SM_GET_SUPPORTED_MODES is used for getting the supported steering modes by the device.
+	The command should be executed by the driver before attempting to configure the steering mode. Once the device
+	receives this command it should fill the virtio_net_steering_modes structure with the supported steering mode

content.tex VIRTIO_NET_CTRL_MQ_VQ_PAIRS_SET command. · line 4063

Quoted 10 lines
content.tex +10 -0
+	flags.
+
+\item The command VIRTIO_NET_CTRL_SM_CONTROL is used for controlling the different steering modes. Each steering mode
+	has its own set of commands. When executing this command the structure virtio_net_steering_mode should be used as follows:
+	the \field{steering_mode} should be filled with one of the steering mode flags along with an appropriate command from the chosen
+	steering mode. The union of virtio_net_steering_mode should be used and filled as the mode's command suggests.
+\end{enumerate}
+
+If this feature has been negotiated, the virtio header has an additional
+\field{hash} field attached to it.

may be say: ..virtio net header has additional field{hash}

content.tex VIRTIO_NET_CTRL_MQ_VQ_PAIRS_SET command. · line 4071

Quoted 8 lines
content.tex +5 -0
+
+\subparagraph{Automatic Receive Steering}{Device Types / Network Device / Device Operation / Control Virtqueue / Steering mode / Automatic Receive Steering}
+
+This is the default steering mode, please refer to the "Automatic receive steering in multiqueue" section.
+
  \subparagraph{Legacy Interface: Automatic receive steering in multiqueue mode}\label{sec:Device Types / Network Device / Device Operation / Control Virtqueue / Automatic receive steering in multiqueue mode / Legacy Interface: Automatic receive steering in multiqueue mode}
  When using the legacy interface, transitional devices and drivers
  MUST format \field{virtqueue_pairs}

---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]

author
#6 lore raw

[virtio-dev] Re: [PATCH v2 1/2] content: net: Add VIRTIO_NET_F_CTRL_STEERING_MODE feature

Quoted 40 lines · Vijayabhaskar Balakrishna

On 4/16/2018 6:05 AM, Sameeh Jubran wrote:

From: Sameeh Jubran <[email protected]>

Quoted 37 lines

This commit introduces steering mode into network device. Steering mode is a general infrastructure for various steering modes that can be implemented on top of it such as Automatic and RSS.

Signed-off-by Sameeh Jubran <[email protected]>
---
  content.tex | 59 ++++++++++++++++++++++++++++++
+++++++++++++++++++++++++++++
  1 file changed, 59 insertions(+)
content.tex +2 -0
index c840588..3d538e8 100644
@@ -3115,6 +3115,9 @@ features.
    \item[VIRTIO_NET_F_CTRL_MAC_ADDR(23)] Set MAC address through control
      channel.
+
+\item[VIRTIO_NET_F_CTRL_STEERING_MODE(27)] Device supports configurable

steering
+ mode.
\end{description}
\subsubsection{Feature bit requirements}\label{sec:Device Types /
Network Device / Feature bits / Feature bit requirements}

content.tex +2 -0
@@ -3317,6 +3320,8 @@ struct virtio_net_hdr {
          le16 csum_start;
          le16 csum_offset;
          le16 num_buffers;
+// Only if VIRTIO_NET_F_CTRL_STEERING_MODE has been negotiated
+        le64 hash;
  };
  \end{lstlisting}
  @@ -4007,6 +4012,60 @@ VIRTIO_NET_CTRL_MQ_VQ_PAIRS_SET command.

The device MUST NOT queue packets on receive queues greater than \field{virtqueue_pairs} once it has placed the
VIRTIO_NET_CTRL_MQ_VQ_PAIRS_SET command in the used ring.

Quoted 8 lines

content.tex VIRTIO_NET_CTRL_MQ_VQ_PAIRS_SET command. · line 4019

Quoted 7 lines

+\paragraph{Steering mode}\label{sec:Device Types / Network Device /
Device Operation / Control Virtqueue / Steering mode}

content.tex +4 -0
+
+\begin{lstlisting}
+// steering mode flags
+#define STEERING_MODE_AUTO          1

As steering mode flags is a bit field, should it be 0x1?

They are indeed bit fields, will dd 0x in order to clarify that.

Quoted 77 lines

+

Quoted 52 lines
content.tex +20 -0
+struct virtio_net_steering_modes {
+le32 steering_modes;
+};
+
+struct virtio_net_steering_mode {
+le32 steering_mode;
+le32 command;
+
+    union {
+    }
+};
+
+#define VIRTIO_NET_F_CTRL_STEERING_MODE     27
+
+#define VIRTIO_NET_CTRL_STEERING_MODE                7
+#define VIRTIO_NET_CTRL_SM_GET_SUPPORTED_MODES        0
+#define VIRTIO_NET_CTRL_SM_CONTROL                    1
+\end{lstlisting}
+
+If the VIRTIO_NET_F_CTRL_STEERING_MODE is negotiated the driver can

send control commands for the
+configuring the steering mode. There are multiple steering modes and they can be configured using
+the VIRTIO_NET_CTRL_SM_CONTROL command. Each mode has it's own set of commands.

content.tex +2 -0
+
+The auto steering mode is the default mode if nothing else has been

configured by the driver

content.tex +7 -0
+and the VIRTIO_NET_F_CTRL_STEERING_MODE feature is acked by the driver.
+
+The class VIRTIO_NET_CTRL_STEERING_MODE has two commands:
+
+\begin{enumerate}
+
+\item VIRTIO_NET_CTRL_SM_GET_SUPPORTED_MODES is used for getting the

supported steering modes by the device.
+ The command should be executed by the driver before attempting to
configure the steering mode. Once the device
+ receives this command it should fill the
virtio_net_steering_modes structure with the supported steering mode

content.tex +3 -0
+       flags.
+
+\item The command VIRTIO_NET_CTRL_SM_CONTROL is used for controlling the

different steering modes. Each steering mode
+ has its own set of commands. When executing this command the
structure virtio_net_steering_mode should be used as follows:
+ the \field{steering_mode} should be filled with one of the
steering mode flags along with an appropriate command from the chosen
+ steering mode. The union of virtio_net_steering_mode should be
used and filled as the mode's command suggests. +\end{enumerate}

content.tex VIRTIO_NET_CTRL_MQ_VQ_PAIRS_SET command. · line 4063

Quoted 3 lines
content.tex +3 -0
+
+If this feature has been negotiated, the virtio header has an additional
+\field{hash} field attached to it.

may be say: ..virtio net header has additional field{hash}

+

Quoted 14 lines

+\subparagraph{Automatic Receive Steering}{Device Types / Network Device / Device Operation / Control Virtqueue / Steering mode / Automatic Receive Steering}

content.tex +2 -0
+
+This is the default steering mode, please refer to the "Automatic

receive steering in multiqueue" section.
+
\subparagraph{Legacy Interface: Automatic receive steering in
multiqueue mode}\label{sec:Device Types / Network Device / Device Operation / Control Virtqueue / Automatic receive steering in multiqueue mode / Legacy Interface: Automatic receive steering in multiqueue mode}
When using the legacy interface, transitional devices and drivers MUST format \field{virtqueue_pairs}

Signature · 4 lines
Respectfully,
*Sameeh Jubran*
*Linkedin <https://il.linkedin.com/pub/sameeh-jubran/87/747/a8a>*
*Software Engineer @ Daynix <http://www.daynix.com>.*
author
#7 lore raw

PATCH v2 2/2 content: net: steering mode: Add RSS

In reply to Sameeh Jubran

From: Sameeh Jubran <[email protected]>

This commit introduces the RSS feature into virtio-net. It is introduced as a sub mode for a general command which configures the steering mode.

Most modern high end network devices today support configurable hash functions, this commit introduces RSS - Receive Side Scaling - [1] to virtio net device.

The RSS is a technology from Microsoft that boosts network device performance by efficiently distributing the traffic among the CPUs in a multiprocessor system.

This feature is supported in most of the modern network cards as well as most modern OSes including Linux and Windows. It is worth mentioning that both DPDK and Hyper-v support RSS too.

[1] https://docs.microsoft.com/en-us/windows-hardware/drivers/network/ndis-receive-side-scaling2

Signed-off-by Sameeh Jubran <[email protected]>
---
 content.tex | 114 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 114 insertions(+)
content.tex +114 -0
index 3d538e8..8076147 100644
@@ -4017,6 +4017,7 @@ The device MUST NOT queue packets on receive queues greater than
 \begin{lstlisting}
 // steering mode flags
 #define STEERING_MODE_AUTO          1
+#define STEERING_MODE_RSS           2
 
 struct virtio_net_steering_modes {
 le32 steering_modes;

@@ -4027,6 +4028,7 @@ le32 steering_mode;
 le32 command;
 
     union {
+    struct virtio_net_rss rss_conf;
     }
 };
 

@@ -4066,6 +4068,118 @@ If this feature has been negotiated, the virtio header has an additional
 
 This is the default steering mode, please refer to the "Automatic receive steering in multiqueue" section.
 
+\subparagraph{Receive Side Scaling}{Device Types / Network Device / Device Operation / Control Virtqueue / Steering mode / Receive Side Scaling}
+
+\begin{lstlisting}
+#define RSS_HASH_FUNCTION_NONE      1
+#define RSS_HASH_FUNCTION_TOEPLITZ  2
+#define RSS_HASH_FUNCTION_SYMMETRIC 3
+
+// Hash function fields
+#define RSS_HASH_FIELDS_IPV4          0x00000100
+#define RSS_HASH_FIELDS_TCP_IPV4      0x00000200
+#define RSS_HASH_FIELDS_IPV6          0x00000400
+#define RSS_HASH_FIELDS_IPV6_EX       0x00000800
+#define RSS_HASH_FIELDS_TCP_IPV6      0x00001000
+#define RSS_HASH_FIELDS_TCP_IPV6_EX   0x00002000
+
+struct virtio_net_rss_supported_hash{
+le32 hash_function;
+}
+
+struct virtio_net_rss {
+le32 hash_function;
+le32 hash_function_flags;
+le32 hash_key_length;
+le32 indirection_table_length;
+	struct {
+		le32 hash_key[hash_key_length];
+		le32 indirection_table[indirection_table_length];
+	}
+};
+
+#define VIRTIO_NET_SM_CTRL_RSS_GET_SUPPORTED_FUNCTIONS   0
+#define VIRTIO_NET_SM_CTRL_RSS_SET                       1
+\end{lstlisting}
+
+If the VIRTIO_NET_F_CTRL_STEERING_MODE is negotiated the driver can send control
+commands for the RSS configuration. For configuring RSS the virtio_net_steering_mode
+should be filled. The \field{steering_mode} field should be filled with the STEERING_MODE_RSS
+flag along with one of the VIRTIO_NET_SM_CTRL_RSS commands in the \field{command} field. The
+\field{rss_conf} field should be used.
+
+The class VIRTIO_NET_CTRL_RSS has two commands:
+
+\begin{enumerate}
+\item VIRTIO_NET_SM_CTRL_RSS_GET_SUPPORTED_FUNCTIONS returns the hash functions
+	supported by the device to the driver.
+\item VIRTIO_NET_SM_CTRL_RSS_SET applies the new RSS configuration. The command is
+	used by the driver for setting RSS hash function, hash key and
+	indirection table in the device.
+\end{enumerate}
+
+\devicenormative{\subparagraph}{Receive Side Scaling}{Device Types / Network Device / Device Operation / Control Virtqueue / Steering mode / Receive Side Scaling}
+
+The device MUST fill the virtio_net_rss_supported_hash structure with the hash
+functions it supports and return the structure to the driver. Zero or more
+flags of the RSS_HASH_FUNCTION flags MUST be used to fill the \field{hash_function}
+field.
+
+The device MUST drop all previous RSS configuration upon receiving
+VIRTIO_NET_SM_CTRL_RSS_SET command.
+
+The device MUST set the RSS configuration according to the settings provided as
+follows, once the configuration process is completed the device SHOULD apply
+the hash function to each of the incoming packets and distribute the packets
+through the virqueues using the calculated hash and the indirection table
+that were earlier provided by the driver.
+
+Setting RSS configuration
+\begin{enumerate}
+\item The driver fills all of the fields and passes them through the control
+	queue to the device.
+
+\item The device sets the RSS configuration as provided by the driver.
+
+\item If the device successfully applied the configuration, on each packet
+	received the device MUST calculate the hashing for the packet and
+	store it in the virtio-net header in the \field{hash} field and the
+	hash fields used in the calculation in rss_hash_type.
+\end{enumerate}
+
+In case of any unexpected values/ unsupported hash function the driver
+MUST return VIRTIO_NET_ERR in the \field{ack} field.
+
+\drivernormative{\subparagraph}{Receive Side Scaling}{Device Types / Network Device / Device Operation / Control Virtqueue / Steering mode / Receive Side Scaling}
+
+If the driver wants to set RSS hash it should fill the RSS structure fields
+as follows:
+
+\begin{itemize}
+\item The driver SHOULD choose the hash function that SHOULD be used and fill
+	it in the \field{hash_function} field along with the appropriate flags
+	in the \field{hash_function_flags} field. These flags indicate to the
+	device which packet fields MUST be used in the calculation process of
+	the hash.
+\item Once the hash function has been chosen a suitable hash key should be set
+	in the \field{hash_key} field, the length of the key should be stored
+	in the \field{hash_key_length} field.
+\item Lastly the driver should fill the indirection table array in the
+	\field{indirection_table} field while setting the array length in
+	\field{indirection_table_length}. This structure is used by the device
+	for determining in which RX virt queue the packet should be placed.
+\end{itemize}
+Once the configuration phase is over successfully, the packets SHOULD have the
+\field{hash} field with the hash value that was calculated by the device.
+
+Whenever the driver wants to discard the current RSS settings, it can send an
+VIRTIO_NET_SM_CTRL_RSS_SET command along with rss structure that has
+RSS_HASH_FUNCTION_NONE the \field{hash_function} field.
+
+The driver MUST check the \field{ack} field value provided by the device, in
+case the value is not VIRTIO_NET_OK then the driver MUST handle failure and
+retry another hash function or else give up.
+
 \subparagraph{Legacy Interface: Automatic receive steering in multiqueue mode}\label{sec:Device Types / Network Device / Device Operation / Control Virtqueue / Automatic receive steering in multiqueue mode / Legacy Interface: Automatic receive steering in multiqueue mode}
 When using the legacy interface, transitional devices and drivers
 MUST format \field{virtqueue_pairs}
Signature · 6 lines
2.13.6


---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
#8 lore raw

[virtio-dev] Re: [PATCH v2 2/2] content: net: steering mode: Add RSS

Quoted 44 lines · Sameeh Jubran

From: Sameeh Jubran <[email protected]>

This commit introduces the RSS feature into virtio-net. It is introduced as a sub mode for a general command which configures the steering mode.

Most modern high end network devices today support configurable hash functions, this commit introduces RSS - Receive Side Scaling - [1] to virtio net device.

The RSS is a technology from Microsoft that boosts network device performance by efficiently distributing the traffic among the CPUs in a multiprocessor system.

This feature is supported in most of the modern network cards as well as most modern OSes including Linux and Windows. It is worth mentioning that both DPDK and Hyper-v support RSS too.

[1] https://docs.microsoft.com/en-us/windows-hardware/drivers/network/ndis-receive-side-scaling2

Signed-off-by Sameeh Jubran <[email protected]>
---
  content.tex | 114 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
  1 file changed, 114 insertions(+)
content.tex +2 -0
index 3d538e8..8076147 100644
@@ -4017,6 +4017,7 @@ The device MUST NOT queue packets on receive queues greater than
  \begin{lstlisting}
  // steering mode flags
  #define STEERING_MODE_AUTO          1
+#define STEERING_MODE_RSS           2
 
  struct virtio_net_steering_modes {
  le32 steering_modes;

@@ -4027,6 +4028,7 @@ le32 steering_mode;
  le32 command;
 
      union {
+    struct virtio_net_rss rss_conf;
      }
  };

content.tex If this feature has been negotiated, the virtio header has an additional · line 4076

Quoted 9 lines
content.tex +6 -0
@@ -4066,6 +4068,118 @@ If this feature has been negotiated, the virtio header has an additional
 
  This is the default steering mode, please refer to the "Automatic receive steering in multiqueue" section.
 
+\subparagraph{Receive Side Scaling}{Device Types / Network Device / Device Operation / Control Virtqueue / Steering mode / Receive Side Scaling}
+
+\begin{lstlisting}
+#define RSS_HASH_FUNCTION_NONE      1
+#define RSS_HASH_FUNCTION_TOEPLITZ  2
+#define RSS_HASH_FUNCTION_SYMMETRIC 3

Above are bit fields?  Should it be 0x1, 0x2, 0x4 etc.,

Quoted 59 lines
content.tex +59 -0
+
+// Hash function fields
+#define RSS_HASH_FIELDS_IPV4          0x00000100
+#define RSS_HASH_FIELDS_TCP_IPV4      0x00000200
+#define RSS_HASH_FIELDS_IPV6          0x00000400
+#define RSS_HASH_FIELDS_IPV6_EX       0x00000800
+#define RSS_HASH_FIELDS_TCP_IPV6      0x00001000
+#define RSS_HASH_FIELDS_TCP_IPV6_EX   0x00002000
+
+struct virtio_net_rss_supported_hash{
+le32 hash_function;
+}
+
+struct virtio_net_rss {
+le32 hash_function;
+le32 hash_function_flags;
+le32 hash_key_length;
+le32 indirection_table_length;
+	struct {
+		le32 hash_key[hash_key_length];
+		le32 indirection_table[indirection_table_length];
+	}
+};
+
+#define VIRTIO_NET_SM_CTRL_RSS_GET_SUPPORTED_FUNCTIONS   0
+#define VIRTIO_NET_SM_CTRL_RSS_SET                       1
+\end{lstlisting}
+
+If the VIRTIO_NET_F_CTRL_STEERING_MODE is negotiated the driver can send control
+commands for the RSS configuration. For configuring RSS the virtio_net_steering_mode
+should be filled. The \field{steering_mode} field should be filled with the STEERING_MODE_RSS
+flag along with one of the VIRTIO_NET_SM_CTRL_RSS commands in the \field{command} field. The
+\field{rss_conf} field should be used.
+
+The class VIRTIO_NET_CTRL_RSS has two commands:
+
+\begin{enumerate}
+\item VIRTIO_NET_SM_CTRL_RSS_GET_SUPPORTED_FUNCTIONS returns the hash functions
+	supported by the device to the driver.
+\item VIRTIO_NET_SM_CTRL_RSS_SET applies the new RSS configuration. The command is
+	used by the driver for setting RSS hash function, hash key and
+	indirection table in the device.
+\end{enumerate}
+
+\devicenormative{\subparagraph}{Receive Side Scaling}{Device Types / Network Device / Device Operation / Control Virtqueue / Steering mode / Receive Side Scaling}
+
+The device MUST fill the virtio_net_rss_supported_hash structure with the hash
+functions it supports and return the structure to the driver. Zero or more
+flags of the RSS_HASH_FUNCTION flags MUST be used to fill the \field{hash_function}
+field.
+
+The device MUST drop all previous RSS configuration upon receiving
+VIRTIO_NET_SM_CTRL_RSS_SET command.
+
+The device MUST set the RSS configuration according to the settings provided as
+follows, once the configuration process is completed the device SHOULD apply
+the hash function to each of the incoming packets and distribute the packets
+through the virqueues using the calculated hash and the indirection table
+that were earlier provided by the driver.

content.tex If this feature has been negotiated, the virtio header has an additional · line 4145

Quoted 10 lines
content.tex +10 -0
+
+Setting RSS configuration
+\begin{enumerate}
+\item The driver fills all of the fields and passes them through the control
+	queue to the device.
+
+\item The device sets the RSS configuration as provided by the driver.
+
+\item If the device successfully applied the configuration, on each packet
+	received the device MUST calculate the hashing for the packet and

calculate the hash for the packet..

content.tex If this feature has been negotiated, the virtio header has an additional · line 4147

Quoted 2 lines
content.tex +2 -0
+	store it in the virtio-net header in the \field{hash} field and the
+	hash fields used in the calculation in rss_hash_type.

"rss_hash_type" carryover from previous (v1) patch?  May be should say:

MUST calculate the hash for the packet using fields specified in hash_function_flags

Quoted 28 lines
content.tex +28 -0
+\end{enumerate}
+
+In case of any unexpected values/ unsupported hash function the driver
+MUST return VIRTIO_NET_ERR in the \field{ack} field.
+
+\drivernormative{\subparagraph}{Receive Side Scaling}{Device Types / Network Device / Device Operation / Control Virtqueue / Steering mode / Receive Side Scaling}
+
+If the driver wants to set RSS hash it should fill the RSS structure fields
+as follows:
+
+\begin{itemize}
+\item The driver SHOULD choose the hash function that SHOULD be used and fill
+	it in the \field{hash_function} field along with the appropriate flags
+	in the \field{hash_function_flags} field. These flags indicate to the
+	device which packet fields MUST be used in the calculation process of
+	the hash.
+\item Once the hash function has been chosen a suitable hash key should be set
+	in the \field{hash_key} field, the length of the key should be stored
+	in the \field{hash_key_length} field.
+\item Lastly the driver should fill the indirection table array in the
+	\field{indirection_table} field while setting the array length in
+	\field{indirection_table_length}. This structure is used by the device
+	for determining in which RX virt queue the packet should be placed.
+\end{itemize}
+Once the configuration phase is over successfully, the packets SHOULD have the
+\field{hash} field with the hash value that was calculated by the device.
+
+Whenever the driver wants to discard the current RSS settings, it can send an

content.tex If this feature has been negotiated, the virtio header has an additional · line 4185

Quoted 10 lines
content.tex +7 -0
+VIRTIO_NET_SM_CTRL_RSS_SET command along with rss structure that has
+RSS_HASH_FUNCTION_NONE the \field{hash_function} field.
+
+The driver MUST check the \field{ack} field value provided by the device, in
+case the value is not VIRTIO_NET_OK then the driver MUST handle failure and
+retry another hash function or else give up.
+
  \subparagraph{Legacy Interface: Automatic receive steering in multiqueue mode}\label{sec:Device Types / Network Device / Device Operation / Control Virtqueue / Automatic receive steering in multiqueue mode / Legacy Interface: Automatic receive steering in multiqueue mode}
  When using the legacy interface, transitional devices and drivers
  MUST format \field{virtqueue_pairs}

---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]

author
#9 lore raw

[virtio-dev] Re: [PATCH v2 2/2] content: net: steering mode: Add RSS

Quoted 57 lines · Vijayabhaskar Balakrishna
Quoted 53 lines · Sameeh Jubran

From: Sameeh Jubran <[email protected]>

This commit introduces the RSS feature into virtio-net. It is introduced as a sub mode for a general command which configures the steering mode.

Most modern high end network devices today support configurable hash functions,
this commit introduces RSS - Receive Side Scaling - [1] to virtio net device.

The RSS is a technology from Microsoft that boosts network device performance
by efficiently distributing the traffic among the CPUs in a multiprocessor system.

This feature is supported in most of the modern network cards as well as most
modern OSes including Linux and Windows. It is worth mentioning that both DPDK
and Hyper-v support RSS too.

[1] https://docs.microsoft.com/en-us/windows-hardware/drivers/ne
twork/ndis-receive-side-scaling2

Signed-off-by Sameeh Jubran <[email protected]>
---
  content.tex | 114 ++++++++++++++++++++++++++++++
++++++++++++++++++++++++++++++
  1 file changed, 114 insertions(+)
content.tex +0 -0
index 3d538e8..8076147 100644
@@ -4017,6 +4017,7 @@ The device MUST NOT queue packets on receive queues

greater than
\begin{lstlisting}
// steering mode flags
#define STEERING_MODE_AUTO 1

content.tex +2 -0
+#define STEERING_MODE_RSS           2
    struct virtio_net_steering_modes {
  le32 steering_modes;

@@ -4027,6 +4028,7 @@ le32 steering_mode;
  le32 command;
        union {
+    struct virtio_net_rss rss_conf;
      }
  };
  @@ -4066,6 +4068,118 @@ If this feature has been negotiated, the virtio

header has an additional
This is the default steering mode, please refer to the "Automatic
receive steering in multiqueue" section.
+\subparagraph{Receive Side Scaling}{Device Types / Network Device /

Quoted 8 lines

content.tex If this feature has been negotiated, the virtio header has an additional · line 4076

Quoted 7 lines

Device Operation / Control Virtqueue / Steering mode / Receive Side Scaling}

content.tex +5 -0
+
+\begin{lstlisting}
+#define RSS_HASH_FUNCTION_NONE      1
+#define RSS_HASH_FUNCTION_TOEPLITZ  2
+#define RSS_HASH_FUNCTION_SYMMETRIC 3

Above are bit fields? Should it be 0x1, 0x2, 0x4 etc.,

Will do.

Quoted 89 lines

+

Quoted 82 lines
content.tex +28 -0
+// Hash function fields
+#define RSS_HASH_FIELDS_IPV4          0x00000100
+#define RSS_HASH_FIELDS_TCP_IPV4      0x00000200
+#define RSS_HASH_FIELDS_IPV6          0x00000400
+#define RSS_HASH_FIELDS_IPV6_EX       0x00000800
+#define RSS_HASH_FIELDS_TCP_IPV6      0x00001000
+#define RSS_HASH_FIELDS_TCP_IPV6_EX   0x00002000
+
+struct virtio_net_rss_supported_hash{
+le32 hash_function;
+}
+
+struct virtio_net_rss {
+le32 hash_function;
+le32 hash_function_flags;
+le32 hash_key_length;
+le32 indirection_table_length;
+       struct {
+               le32 hash_key[hash_key_length];
+               le32 indirection_table[indirection_table_length];
+       }
+};
+
+#define VIRTIO_NET_SM_CTRL_RSS_GET_SUPPORTED_FUNCTIONS   0
+#define VIRTIO_NET_SM_CTRL_RSS_SET                       1
+\end{lstlisting}
+
+If the VIRTIO_NET_F_CTRL_STEERING_MODE is negotiated the driver can

send control
+commands for the RSS configuration. For configuring RSS the virtio_net_steering_mode
+should be filled. The \field{steering_mode} field should be filled with the STEERING_MODE_RSS
+flag along with one of the VIRTIO_NET_SM_CTRL_RSS commands in the \field{command} field. The
+\field{rss_conf} field should be used.

content.tex +5 -0
+
+The class VIRTIO_NET_CTRL_RSS has two commands:
+
+\begin{enumerate}
+\item VIRTIO_NET_SM_CTRL_RSS_GET_SUPPORTED_FUNCTIONS returns the hash

functions
+ supported by the device to the driver.
+\item VIRTIO_NET_SM_CTRL_RSS_SET applies the new RSS configuration. The command is

content.tex +5 -0
+       used by the driver for setting RSS hash function, hash key and
+       indirection table in the device.
+\end{enumerate}
+
+\devicenormative{\subparagraph}{Receive Side Scaling}{Device Types /

Network Device / Device Operation / Control Virtqueue / Steering mode / Receive Side Scaling}

content.tex +2 -0
+
+The device MUST fill the virtio_net_rss_supported_hash structure with

the hash
+functions it supports and return the structure to the driver. Zero or more
+flags of the RSS_HASH_FUNCTION flags MUST be used to fill the \field{hash_function}

content.tex +6 -0
+field.
+
+The device MUST drop all previous RSS configuration upon receiving
+VIRTIO_NET_SM_CTRL_RSS_SET command.
+
+The device MUST set the RSS configuration according to the settings

provided as
+follows, once the configuration process is completed the device SHOULD apply
+the hash function to each of the incoming packets and distribute the packets

content.tex +6 -0
+through the virqueues using the calculated hash and the indirection table
+that were earlier provided by the driver.
+
+Setting RSS configuration
+\begin{enumerate}
+\item The driver fills all of the fields and passes them through the

control

content.tex +5 -0
+       queue to the device.
+
+\item The device sets the RSS configuration as provided by the driver.
+
+\item If the device successfully applied the configuration, on each
Quoted 3 lines

packet
+ received the device MUST calculate the hashing for the packet and

calculate the hash for the packet..

Quoted 9 lines
Quoted 4 lines

+ store it in the virtio-net header in the \field{hash} field and
the
+ hash fields used in the calculation in rss_hash_type.

"rss_hash_type" carryover from previous (v1) patch? May be should say:

MUST calculate the hash for the packet using fields specified in
hash_function_flags

Will do thanks :)

Quoted 62 lines

+\end{enumerate}

Quoted 56 lines
content.tex +5 -0
+
+In case of any unexpected values/ unsupported hash function the driver
+MUST return VIRTIO_NET_ERR in the \field{ack} field.
+
+\drivernormative{\subparagraph}{Receive Side Scaling}{Device Types /

Network Device / Device Operation / Control Virtqueue / Steering mode / Receive Side Scaling}

content.tex +2 -0
+
+If the driver wants to set RSS hash it should fill the RSS structure

fields

content.tex +4 -0
+as follows:
+
+\begin{itemize}
+\item The driver SHOULD choose the hash function that SHOULD be used and

fill
+ it in the \field{hash_function} field along with the appropriate
flags
+ in the \field{hash_function_flags} field. These flags indicate to
the
+ device which packet fields MUST be used in the calculation
process of
+ the hash.
+\item Once the hash function has been chosen a suitable hash key should be set
+ in the \field{hash_key} field, the length of the key should be
stored

content.tex +4 -0
+       in the \field{hash_key_length} field.
+\item Lastly the driver should fill the indirection table array in the
+       \field{indirection_table} field while setting the array length in
+       \field{indirection_table_length}. This structure is used by the

device
+ for determining in which RX virt queue the packet should be
placed.
+\end{itemize}
+Once the configuration phase is over successfully, the packets SHOULD have the
+\field{hash} field with the hash value that was calculated by the device.

content.tex +2 -0
+
+Whenever the driver wants to discard the current RSS settings, it can

send an

content.tex +4 -0
+VIRTIO_NET_SM_CTRL_RSS_SET command along with rss structure that has
+RSS_HASH_FUNCTION_NONE the \field{hash_function} field.
+
+The driver MUST check the \field{ack} field value provided by the

device, in
+case the value is not VIRTIO_NET_OK then the driver MUST handle failure and

content.tex +2 -0
+retry another hash function or else give up.
+
  \subparagraph{Legacy Interface: Automatic receive steering in

multiqueue mode}\label{sec:Device Types / Network Device / Device Operation / Control Virtqueue / Automatic receive steering in multiqueue mode / Legacy Interface: Automatic receive steering in multiqueue mode}
When using the legacy interface, transitional devices and drivers MUST format \field{virtqueue_pairs}

Signature · 4 lines
Respectfully,
*Sameeh Jubran*
*Linkedin <https://il.linkedin.com/pub/sameeh-jubran/87/747/a8a>*
*Software Engineer @ Daynix <http://www.daynix.com>.*
#10 lore raw

Re: [virtio-dev] [PATCH v2 2/2] content: net: steering mode: Add RSS

In reply to Sameeh Jubran
Quoted 69 lines · Sameeh Jubran

From: Sameeh Jubran <[email protected]>

This commit introduces the RSS feature into virtio-net. It is introduced as a sub mode for a general command which configures the steering mode.

Most modern high end network devices today support configurable hash functions, this commit introduces RSS - Receive Side Scaling - [1] to virtio net device.

The RSS is a technology from Microsoft that boosts network device performance by efficiently distributing the traffic among the CPUs in a multiprocessor system.

This feature is supported in most of the modern network cards as well as most modern OSes including Linux and Windows. It is worth mentioning that both DPDK and Hyper-v support RSS too.

[1] https://docs.microsoft.com/en-us/windows-hardware/drivers/network/ndis-receive-side-scaling2

Signed-off-by Sameeh Jubran <[email protected]>
---
 content.tex | 114 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 114 insertions(+)
content.tex +24 -0
index 3d538e8..8076147 100644
@@ -4017,6 +4017,7 @@ The device MUST NOT queue packets on receive queues greater than
 \begin{lstlisting}
 // steering mode flags
 #define STEERING_MODE_AUTO          1
+#define STEERING_MODE_RSS           2
 
 struct virtio_net_steering_modes {
 le32 steering_modes;

@@ -4027,6 +4028,7 @@ le32 steering_mode;
 le32 command;
 
     union {
+    struct virtio_net_rss rss_conf;
     }
 };
 

@@ -4066,6 +4068,118 @@ If this feature has been negotiated, the virtio header has an additional
 
 This is the default steering mode, please refer to the "Automatic receive steering in multiqueue" section.
 
+\subparagraph{Receive Side Scaling}{Device Types / Network Device / Device Operation / Control Virtqueue / Steering mode / Receive Side Scaling}
+
+\begin{lstlisting}
+#define RSS_HASH_FUNCTION_NONE      1
+#define RSS_HASH_FUNCTION_TOEPLITZ  2
+#define RSS_HASH_FUNCTION_SYMMETRIC 3
+
+// Hash function fields
+#define RSS_HASH_FIELDS_IPV4          0x00000100
+#define RSS_HASH_FIELDS_TCP_IPV4      0x00000200
+#define RSS_HASH_FIELDS_IPV6          0x00000400
+#define RSS_HASH_FIELDS_IPV6_EX       0x00000800
+#define RSS_HASH_FIELDS_TCP_IPV6      0x00001000
+#define RSS_HASH_FIELDS_TCP_IPV6_EX   0x00002000
+
+struct virtio_net_rss_supported_hash{
+le32 hash_function;
+}
+
+struct virtio_net_rss {
+le32 hash_function;
+le32 hash_function_flags;

content.tex If this feature has been negotiated, the virtio header has an additional · line 4102

Quoted 10 lines
content.tex +10 -0
+le32 hash_key_length;
+le32 indirection_table_length;
+	struct {
+		le32 hash_key[hash_key_length];
+		le32 indirection_table[indirection_table_length];
+	}
+};
+
+#define VIRTIO_NET_SM_CTRL_RSS_GET_SUPPORTED_FUNCTIONS   0
+#define VIRTIO_NET_SM_CTRL_RSS_SET                       1

Please add comments in the code as well.

content.tex If this feature has been negotiated, the virtio header has an additional

Quoted 4 lines
content.tex +4 -0
+\end{lstlisting}
+
+If the VIRTIO_NET_F_CTRL_STEERING_MODE is negotiated the driver can send control
+commands for the RSS configuration.

Confused. Does VIRTIO_NET_F_CTRL_STEERING_MODE imply steering mode generally or RSS specifically?
How does a device with streering mode ctrl but without RSS look?

Quoted 36 lines

For configuring RSS the virtio_net_steering_mode

content.tex +35 -0
+should be filled. The \field{steering_mode} field should be filled with the STEERING_MODE_RSS
+flag along with one of the VIRTIO_NET_SM_CTRL_RSS commands in the \field{command} field. The
+\field{rss_conf} field should be used.
+
+The class VIRTIO_NET_CTRL_RSS has two commands:
+
+\begin{enumerate}
+\item VIRTIO_NET_SM_CTRL_RSS_GET_SUPPORTED_FUNCTIONS returns the hash functions
+	supported by the device to the driver.
+\item VIRTIO_NET_SM_CTRL_RSS_SET applies the new RSS configuration. The command is
+	used by the driver for setting RSS hash function, hash key and
+	indirection table in the device.
+\end{enumerate}
+
+\devicenormative{\subparagraph}{Receive Side Scaling}{Device Types / Network Device / Device Operation / Control Virtqueue / Steering mode / Receive Side Scaling}
+
+The device MUST fill the virtio_net_rss_supported_hash structure with the hash
+functions it supports and return the structure to the driver. Zero or more
+flags of the RSS_HASH_FUNCTION flags MUST be used to fill the \field{hash_function}
+field.
+
+The device MUST drop all previous RSS configuration upon receiving
+VIRTIO_NET_SM_CTRL_RSS_SET command.
+
+The device MUST set the RSS configuration according to the settings provided as
+follows, once the configuration process is completed the device SHOULD apply
+the hash function to each of the incoming packets and distribute the packets
+through the virqueues using the calculated hash and the indirection table
+that were earlier provided by the driver.
+
+Setting RSS configuration
+\begin{enumerate}
+\item The driver fills all of the fields and passes them through the control
+	queue to the device.
+

content.tex If this feature has been negotiated, the virtio header has an additional · line 4151

Quoted 10 lines
content.tex +10 -0
+\item The device sets the RSS configuration as provided by the driver.
+
+\item If the device successfully applied the configuration, on each packet
+	received the device MUST calculate the hashing for the packet and
+	store it in the virtio-net header in the \field{hash} field and the
+	hash fields used in the calculation in rss_hash_type.
+\end{enumerate}
+
+In case of any unexpected values/ unsupported hash function the driver
+MUST return VIRTIO_NET_ERR in the \field{ack} field.

driver or the device?

All MUST statements belon in correct normative sections.

Generally what is the text trying to say here?

Quoted 4 lines
content.tex +4 -0
+
+\drivernormative{\subparagraph}{Receive Side Scaling}{Device Types / Network Device / Device Operation / Control Virtqueue / Steering mode / Receive Side Scaling}
+
+If the driver wants to set RSS hash it should fill the RSS structure fields

content.tex If this feature has been negotiated, the virtio header has an additional

Quoted 10 lines
content.tex +10 -0
+as follows:
+
+\begin{itemize}
+\item The driver SHOULD choose the hash function that SHOULD be used and fill
+	it in the \field{hash_function} field along with the appropriate flags
+	in the \field{hash_function_flags} field. These flags indicate to the
+	device which packet fields MUST be used in the calculation process of
+	the hash.
+\item Once the hash function has been chosen a suitable hash key should be set
+	in the \field{hash_key} field,

which key is suitable?

content.tex If this feature has been negotiated, the virtio header has an additional · line 4166

Quoted 2 lines

the length of the key should be stored

content.tex +1 -0
+	in the \field{hash_key_length} field.

in 4 byte units?
are there limits on the length?

Quoted 1 line

+\item Lastly the driver should fill the indirection table array

is an indirection table required? why not have a function produce the queue number?

content.tex

Quoted 3 lines

in the

content.tex +2 -0
+	\field{indirection_table} field while setting the array length in
+	\field{indirection_table_length}.

any limits on the length here?

content.tex If this feature has been negotiated, the virtio header has an additional · line 4170

Quoted 2 lines

This structure is used by the device

content.tex +1 -0
+	for determining in which RX virt queue the packet should be placed.

How?

Quoted 3 lines

+\end{itemize}
+Once the configuration phase is over successfully, the packets SHOULD have the +\field{hash} field with the hash value that was calculated by the device.

Why not make this optional? I suspect it's not really useful e.g. for dpdk or xdp which do not generally use hardware offloads.

content.tex If this feature has been negotiated, the virtio header has an additional · line 4177

Quoted 4 lines
content.tex +4 -0
+
+Whenever the driver wants to discard the current RSS settings, it can send an
+VIRTIO_NET_SM_CTRL_RSS_SET command along with rss structure that has
+RSS_HASH_FUNCTION_NONE the \field{hash_function} field.

And then what happens?

content.tex If this feature has been negotiated, the virtio header has an additional · line 4181

Quoted 4 lines
content.tex +4 -0
+
+The driver MUST check the \field{ack} field value provided by the device, in
+case the value is not VIRTIO_NET_OK then the driver MUST handle failure and
+retry another hash function or else give up.

Is there anything special here?

Quoted 4 lines

+
\subparagraph{Legacy Interface: Automatic receive steering in multiqueue mode}\label{sec:Device Types / Network Device / Device Operation / Control Virtqueue / Automatic receive steering in multiqueue mode / Legacy Interface: Automatic receive steering in multiqueue mode} When using the legacy interface, transitional devices and drivers
MUST format \field{virtqueue_pairs}

conformance statements will need to be updated.

Quoted 7 lines
Signature · 6 lines
2.13.6


---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]

author
#11 lore raw

Re: [virtio-dev] [PATCH v2 2/2] content: net: steering mode: Add RSS

Quoted 87 lines · Michael S. Tsirkin
Quoted 6 lines · Sameeh Jubran

From: Sameeh Jubran <[email protected]>

This commit introduces the RSS feature into virtio-net. It is introduced as a sub mode for a general command which configures the steering mode.

Most modern high end network devices today support configurable hash

functions,

Quoted 1 line

this commit introduces RSS - Receive Side Scaling - [1] to virtio net

device.

Quoted 2 lines

The RSS is a technology from Microsoft that boosts network device

performance

Quoted 1 line

by efficiently distributing the traffic among the CPUs in a

multiprocessor

Quoted 3 lines

system.

This feature is supported in most of the modern network cards as well as

most

Quoted 1 line

modern OSes including Linux and Windows. It is worth mentioning that

both DPDK

Quoted 3 lines

and Hyper-v support RSS too.

[1] https://docs.microsoft.com/en-us/windows-hardware/drivers/

network/ndis-receive-side-scaling2

Quoted 4 lines
Signed-off-by Sameeh Jubran <[email protected]>
---
 content.tex | 114 ++++++++++++++++++++++++++++++

++++++++++++++++++++++++++++++

content.tex The device MUST NOT queue packets on receive

Quoted 7 lines
 1 file changed, 114 insertions(+)
content.tex +0 -0
index 3d538e8..8076147 100644
@@ -4017,6 +4017,7 @@ The device MUST NOT queue packets on receive

queues greater than

content.tex If this feature has been negotiated, the virtio

Quoted 16 lines
content.tex +2 -0
 \begin{lstlisting}
 // steering mode flags
 #define STEERING_MODE_AUTO          1
+#define STEERING_MODE_RSS           2
 
 struct virtio_net_steering_modes {
 le32 steering_modes;

@@ -4027,6 +4028,7 @@ le32 steering_mode;
 le32 command;
 
     union {
+    struct virtio_net_rss rss_conf;
     }
 };
 

@@ -4066,6 +4068,118 @@ If this feature has been negotiated, the virtio

header has an additional

content.tex If this feature has been negotiated, the virtio · line 4068

Quoted 2 lines
content.tex +0 -0
 This is the default steering mode, please refer to the "Automatic

receive steering in multiqueue" section.

content.tex If this feature has been negotiated, the virtio · line 4069

Quoted 2 lines
content.tex +1 -0
+\subparagraph{Receive Side Scaling}{Device Types / Network Device /

Device Operation / Control Virtqueue / Steering mode / Receive Side Scaling}

Quoted 26 lines
content.tex +26 -0
+
+\begin{lstlisting}
+#define RSS_HASH_FUNCTION_NONE      1
+#define RSS_HASH_FUNCTION_TOEPLITZ  2
+#define RSS_HASH_FUNCTION_SYMMETRIC 3
+
+// Hash function fields
+#define RSS_HASH_FIELDS_IPV4          0x00000100
+#define RSS_HASH_FIELDS_TCP_IPV4      0x00000200
+#define RSS_HASH_FIELDS_IPV6          0x00000400
+#define RSS_HASH_FIELDS_IPV6_EX       0x00000800
+#define RSS_HASH_FIELDS_TCP_IPV6      0x00001000
+#define RSS_HASH_FIELDS_TCP_IPV6_EX   0x00002000
+
+struct virtio_net_rss_supported_hash{
+le32 hash_function;
+}
+
+struct virtio_net_rss {
+le32 hash_function;
+le32 hash_function_flags;
+le32 hash_key_length;
+le32 indirection_table_length;
+     struct {
+             le32 hash_key[hash_key_length];
+             le32 indirection_table[indirection_table_length];
Quoted 8 lines

content.tex If this feature has been negotiated, the virtio header has an additional · line 4102

Quoted 5 lines
content.tex +5 -0
+     }
+};
+
+#define VIRTIO_NET_SM_CTRL_RSS_GET_SUPPORTED_FUNCTIONS   0
+#define VIRTIO_NET_SM_CTRL_RSS_SET                       1

Please add comments in the code as well.

Do you mean that I should add more comments to the code above which explains the structures and defines?

Quoted 7 lines

content.tex If this feature has been negotiated, the virtio header has an additional

Quoted 6 lines
content.tex +3 -0
+\end{lstlisting}
+
+If the VIRTIO_NET_F_CTRL_STEERING_MODE is negotiated the driver can

send control

Quoted 5 lines

content.tex If this feature has been negotiated, the virtio · line 4104

Quoted 1 line
content.tex +1 -0
+commands for the RSS configuration.

Confused. Does VIRTIO_NET_F_CTRL_STEERING_MODE imply steering mode generally or RSS specifically?
How does a device with streering mode ctrl but without RSS look?

The steering mode provides an infrastructure to multiple modes and it is not specific to RSS. The default mode of the steering mode is the Automatic mode which is the default mode when MQ is enabled. I though this would be the best option to provide backward compatibility and makes sense as we already have "Automatic steering mode" in the spec. VIRTIO_NET_F_CTRL_STEERING_MODE gives each mode it's own subset of commands as is the case in RSS.

Quoted 67 lines
Quoted 4 lines

For configuring RSS the virtio_net_steering_mode
+should be filled. The \field{steering_mode} field should be filled with

the STEERING_MODE_RSS

Quoted 1 line

+flag along with one of the VIRTIO_NET_SM_CTRL_RSS commands in the

\field{command} field. The

content.tex If this feature has been negotiated, the virtio header has an additional

Quoted 6 lines

+\field{rss_conf} field should be used.

content.tex +5 -0
+
+The class VIRTIO_NET_CTRL_RSS has two commands:
+
+\begin{enumerate}
+\item VIRTIO_NET_SM_CTRL_RSS_GET_SUPPORTED_FUNCTIONS returns the hash

functions

Quoted 2 lines

+ supported by the device to the driver.
+\item VIRTIO_NET_SM_CTRL_RSS_SET applies the new RSS configuration. The

command is

content.tex If this feature has been negotiated, the virtio header has an additional

Quoted 5 lines
content.tex +5 -0
+     used by the driver for setting RSS hash function, hash key and
+     indirection table in the device.
+\end{enumerate}
+
+\devicenormative{\subparagraph}{Receive Side Scaling}{Device Types /

Network Device / Device Operation / Control Virtqueue / Steering mode / Receive Side Scaling}

content.tex

Quoted 2 lines
content.tex +2 -0
+
+The device MUST fill the virtio_net_rss_supported_hash structure with

the hash

Quoted 1 line

+functions it supports and return the structure to the driver. Zero or

more

Quoted 1 line

+flags of the RSS_HASH_FUNCTION flags MUST be used to fill the

\field{hash_function}

content.tex If this feature has been negotiated, the virtio header has an additional

Quoted 6 lines
content.tex +6 -0
+field.
+
+The device MUST drop all previous RSS configuration upon receiving
+VIRTIO_NET_SM_CTRL_RSS_SET command.
+
+The device MUST set the RSS configuration according to the settings

provided as

Quoted 1 line

+follows, once the configuration process is completed the device SHOULD

apply

Quoted 1 line

+the hash function to each of the incoming packets and distribute the

packets

Quoted 1 line

+through the virqueues using the calculated hash and the indirection

table

content.tex If this feature has been negotiated, the virtio header has an additional

Quoted 5 lines
content.tex +5 -0
+that were earlier provided by the driver.
+
+Setting RSS configuration
+\begin{enumerate}
+\item The driver fills all of the fields and passes them through the

control

content.tex If this feature has been negotiated, the virtio header has an additional

Quoted 5 lines
content.tex +5 -0
+     queue to the device.
+
+\item The device sets the RSS configuration as provided by the driver.
+
+\item If the device successfully applied the configuration, on each

packet

content.tex If this feature has been negotiated, the virtio header has an additional · line 4151

Quoted 7 lines
content.tex +7 -0
+     received the device MUST calculate the hashing for the packet and
+     store it in the virtio-net header in the \field{hash} field and the
+     hash fields used in the calculation in rss_hash_type.
+\end{enumerate}
+
+In case of any unexpected values/ unsupported hash function the driver
+MUST return VIRTIO_NET_ERR in the \field{ack} field.

driver or the device?

Quoted 2 lines

All MUST statements belon in correct normative sections.

True, this is obviously a mistake, it should be the device instead of the driver.

Quoted 3 lines

Generally what is the text trying to say here?

That the device should handle errors and inform the drivers if there are any errors during the execution of the commands.

Quoted 17 lines
Quoted 3 lines

+
+\drivernormative{\subparagraph}{Receive Side Scaling}{Device Types /

Network Device / Device Operation / Control Virtqueue / Steering mode / Receive Side Scaling}

content.tex

Quoted 2 lines
content.tex +2 -0
+
+If the driver wants to set RSS hash it should fill the RSS structure

fields

content.tex If this feature has been negotiated, the virtio header has an additional

Quoted 4 lines
content.tex +4 -0
+as follows:
+
+\begin{itemize}
+\item The driver SHOULD choose the hash function that SHOULD be used

and fill

Quoted 1 line

+ it in the \field{hash_function} field along with the appropriate

flags

Quoted 1 line

+ in the \field{hash_function_flags} field. These flags indicate to

the

Quoted 9 lines
Quoted 1 line

+ device which packet fields MUST be used in the calculation process

of

Quoted 2 lines

+ the hash.
+\item Once the hash function has been chosen a suitable hash key should

be set

Quoted 1 line

+ in the \field{hash_key} field,

which key is suitable?

depends on the configuration, in Windows case the OS provide us with the key, however, in general it could be anything.

Quoted 7 lines
Quoted 3 lines

the length of the key should be stored
+ in the \field{hash_key_length} field.

in 4 byte units?
are there limits on the length?

I think one byte can be used as well as now it is 40 bytes in Windows. What is the best size that should be chosen in order to keep it future proof?

Quoted 6 lines
Quoted 2 lines

+\item Lastly the driver should fill the indirection table array

is an indirection table required? why not have a function produce the queue number?

Yes it is required as in Windows it is given by the OS. the indirection table is basically a map between the hash and cpu ids which correspond to queue ids.

Quoted 7 lines

content.tex

Quoted 4 lines

in the

content.tex +2 -0
+     \field{indirection_table} field while setting the array length in
+     \field{indirection_table_length}.

any limits on the length here?

currently the indirection table is 128 bytes in Windows. We can change the sizes to fit that case, but what about future compatibility? What do you think should be the optimal size of the fields in order to keep this future proof while keeping the overhead minimal.

Quoted 6 lines
Quoted 3 lines

This structure is used by the device
+ for determining in which RX virt queue the packet should be placed.

How?

By applying the hash function on the packet while using the provided key in order to determine the hash for the packet and then use the indirection table to place the packet inside the suitable queue. This is hash function dependent implementation, but more elaboration can be added indeed.

Quoted 10 lines
Quoted 3 lines

+\end{itemize}
+Once the configuration phase is over successfully, the packets SHOULD

have the

Quoted 1 line

+\field{hash} field with the hash value that was calculated by the

device.

Why not make this optional? I suspect it's not really useful e.g. for dpdk or xdp which do not generally use hardware offloads.

Good point, will make it optional.

Quoted 9 lines

content.tex

Quoted 3 lines
content.tex +2 -0
+
+Whenever the driver wants to discard the current RSS settings, it can

send an

content.tex If this feature has been negotiated, the virtio header has an additional · line 4177

Quoted 2 lines
content.tex +2 -0
+VIRTIO_NET_SM_CTRL_RSS_SET command along with rss structure that has
+RSS_HASH_FUNCTION_NONE the \field{hash_function} field.

And then what happens?

and then the device is configured with RSS settings and MUST start distributing the packets into the queues accordingly while attaching the calculated hash to the header.

Quoted 10 lines

content.tex

Quoted 3 lines
content.tex +2 -0
+
+The driver MUST check the \field{ack} field value provided by the

device, in

Quoted 1 line

+case the value is not VIRTIO_NET_OK then the driver MUST handle failure

and

Quoted 1 line

+retry another hash function or else give up.

Is there anything special here?

Not really just describing the error handling in detail.

Quoted 11 lines
Quoted 3 lines

+
\subparagraph{Legacy Interface: Automatic receive steering in

multiqueue mode}\label{sec:Device Types / Network Device / Device Operation / Control Virtqueue / Automatic receive steering in multiqueue mode / Legacy Interface: Automatic receive steering in multiqueue mode}

Quoted 2 lines

When using the legacy interface, transitional devices and drivers MUST format \field{virtqueue_pairs}

conformance statements will need to be updated.

which statements for example?

Quoted 9 lines
Quoted 8 lines

--
2.13.6

---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]

Signature · 4 lines
Respectfully,
*Sameeh Jubran*
*Linkedin <https://il.linkedin.com/pub/sameeh-jubran/87/747/a8a>*
*Software Engineer @ Daynix <http://www.daynix.com>.*
#12 lore raw

Re: [virtio-dev] [PATCH v2 2/2] content: net: steering mode: Add RSS

Quoted 101 lines · Sameeh Jubran

On Tue, Apr 17, 2018 at 4:50 AM, Michael S. Tsirkin <[email protected]> wrote:

    On Mon, Apr 16, 2018 at 04:05:26PM +0300, Sameeh Jubran wrote:
    > From: Sameeh Jubran <[email protected]>
    >
    > This commit introduces the RSS feature into virtio-net. It is introduced
    > as a sub mode for a general command which configures the steering mode.
    >
    > Most modern high end network devices today support configurable hash
    functions,
    > this commit introduces RSS - Receive Side Scaling - [1] to virtio net
    device.
    >
    > The RSS is a technology from Microsoft that boosts network device
    performance
    > by efficiently distributing the traffic among the CPUs in a
    multiprocessor
    > system.
    >
    > This feature is supported in most of the modern network cards as well as
    most
    > modern OSes including Linux and Windows. It is worth mentioning that both
    DPDK
    > and Hyper-v support RSS too.
    >
    > [1] https://docs.microsoft.com/en-us/windows-hardware/drivers/network/
    ndis-receive-side-scaling2
    >
    > Signed-off-by: Sameeh Jubran <[email protected]>
    > ---
    >  content.tex | 114 ++++++++++++++++++++++++++++++
    ++++++++++++++++++++++++++++++
    >  1 file changed, 114 insertions(+)
    >
    > diff --git a/content.tex b/content.tex
    > index 3d538e8..8076147 100644
    > --- a/content.tex
    > +++ b/content.tex
    > @@ -4017,6 +4017,7 @@ The device MUST NOT queue packets on receive queues
    greater than
    >  \begin{lstlisting}
    >  // steering mode flags
    >  #define STEERING_MODE_AUTO          1
    > +#define STEERING_MODE_RSS           2
    > 
    >  struct virtio_net_steering_modes {
    >  le32 steering_modes;
    > @@ -4027,6 +4028,7 @@ le32 steering_mode;
    >  le32 command;
    > 
    >      union {
    > +    struct virtio_net_rss rss_conf;
    >      }
    >  };
    > 
    > @@ -4066,6 +4068,118 @@ If this feature has been negotiated, the virtio
    header has an additional
    > 
    >  This is the default steering mode, please refer to the "Automatic
    receive steering in multiqueue" section.
    > 
    > +\subparagraph{Receive Side Scaling}{Device Types / Network Device /
    Device Operation / Control Virtqueue / Steering mode / Receive Side
    Scaling}
    > +
    > +\begin{lstlisting}
    > +#define RSS_HASH_FUNCTION_NONE      1
    > +#define RSS_HASH_FUNCTION_TOEPLITZ  2
    > +#define RSS_HASH_FUNCTION_SYMMETRIC 3
    > +
    > +// Hash function fields
    > +#define RSS_HASH_FIELDS_IPV4          0x00000100
    > +#define RSS_HASH_FIELDS_TCP_IPV4      0x00000200
    > +#define RSS_HASH_FIELDS_IPV6          0x00000400
    > +#define RSS_HASH_FIELDS_IPV6_EX       0x00000800
    > +#define RSS_HASH_FIELDS_TCP_IPV6      0x00001000
    > +#define RSS_HASH_FIELDS_TCP_IPV6_EX   0x00002000
    > +
    > +struct virtio_net_rss_supported_hash{
    > +le32 hash_function;
    > +}
    > +
    > +struct virtio_net_rss {
    > +le32 hash_function;
    > +le32 hash_function_flags;
    > +le32 hash_key_length;
    > +le32 indirection_table_length;
    > +     struct {
    > +             le32 hash_key[hash_key_length];
    > +             le32 indirection_table[indirection_table_length];
    > +     }
    > +};
    > +
    > +#define VIRTIO_NET_SM_CTRL_RSS_GET_SUPPORTED_FUNCTIONS   0
    > +#define VIRTIO_NET_SM_CTRL_RSS_SET                       1


    Please add comments in the code as well.
Quoted 2 lines

Do you mean that I should add more comments to the code above which explains the structures and defines?

Exactly.

Quoted 9 lines

> +\end{lstlisting}
> +
> +If the VIRTIO_NET_F_CTRL_STEERING_MODE is negotiated the driver can send control
> +commands for the RSS configuration.

Quoted 6 lines

Confused.  Does VIRTIO_NET_F_CTRL_STEERING_MODE imply steering mode generally or RSS specifically?
How does a device with streering mode ctrl but without RSS look?

The steering mode provides an infrastructure to multiple modes and it is not specific to RSS.

Then maybe this sentence should not imply that VIRTIO_NET_F_CTRL_STEERING_MODE allows RSS.

Quoted 109 lines

The default mode of the steering mode is the Automatic mode
which is the default mode when MQ is enabled. I though this would be the best option to provide backward compatibility and makes sense as we already have "Automatic steering mode" in the spec. VIRTIO_NET_F_CTRL_STEERING_MODE  gives each mode it's own subset of commands as is the case in RSS.

Quoted 2 lines

For configuring RSS the virtio_net_steering_mode
+should be filled. The \field{steering_mode} field should be filled with

the STEERING_MODE_RSS
> +flag along with one of the VIRTIO_NET_SM_CTRL_RSS commands in the \field {command} field. The
> +\field{rss_conf} field should be used.
> +
> +The class VIRTIO_NET_CTRL_RSS has two commands:
> +
> +\begin{enumerate}
> +\item VIRTIO_NET_SM_CTRL_RSS_GET_SUPPORTED_FUNCTIONS returns the hash functions
> +     supported by the device to the driver.
> +\item VIRTIO_NET_SM_CTRL_RSS_SET applies the new RSS configuration. The command is
> +     used by the driver for setting RSS hash function, hash key and > +     indirection table in the device.
> +\end{enumerate}
> +
> +\devicenormative{\subparagraph}{Receive Side Scaling}{Device Types / Network Device / Device Operation / Control Virtqueue / Steering mode / Receive Side Scaling}
> +
> +The device MUST fill the virtio_net_rss_supported_hash structure with the hash
> +functions it supports and return the structure to the driver. Zero or more
> +flags of the RSS_HASH_FUNCTION flags MUST be used to fill the \field {hash_function}
> +field.
> +
> +The device MUST drop all previous RSS configuration upon receiving > +VIRTIO_NET_SM_CTRL_RSS_SET command.
> +
> +The device MUST set the RSS configuration according to the settings provided as
> +follows, once the configuration process is completed the device SHOULD apply
> +the hash function to each of the incoming packets and distribute the packets
> +through the virqueues using the calculated hash and the indirection table
> +that were earlier provided by the driver.
> +
> +Setting RSS configuration
> +\begin{enumerate}
> +\item The driver fills all of the fields and passes them through the control
> +     queue to the device.
> +
> +\item The device sets the RSS configuration as provided by the driver. > +
> +\item If the device successfully applied the configuration, on each packet
> +     received the device MUST calculate the hashing for the packet and > +     store it in the virtio-net header in the \field{hash} field and the > +     hash fields used in the calculation in rss_hash_type. > +\end{enumerate}
> +
> +In case of any unexpected values/ unsupported hash function the driver > +MUST return VIRTIO_NET_ERR in the \field{ack} field.

driver or the device?

All MUST statements belon in correct normative sections.

True, this is obviously a mistake, it should be the device instead of the driver.

Generally what is the text trying to say here?

That the device should handle errors and inform the drivers if there are any errors during the execution of the commands.

Quoted 2 lines

+
+\drivernormative{\subparagraph}{Receive Side Scaling}{Device Types /

Network Device / Device Operation / Control Virtqueue / Steering mode / Receive Side Scaling}
> +
> +If the driver wants to set RSS hash it should fill the RSS structure fields
> +as follows:
> +
> +\begin{itemize}
> +\item The driver SHOULD choose the hash function that SHOULD be used and fill
> +     it in the \field{hash_function} field along with the appropriate flags
> +     in the \field{hash_function_flags} field. These flags indicate to the
> +     device which packet fields MUST be used in the calculation process of
> +     the hash.
> +\item Once the hash function has been chosen a suitable hash key should be set
> +     in the \field{hash_key} field,

Quoted 5 lines

which key is suitable?

depends on the configuration, in Windows case the OS provide us with the key, however, in general it could be anything.

We need to somehow describe it in the spec :)

Quoted 8 lines
Quoted 2 lines

the length of the key should be stored
+     in the \field{hash_key_length} field.

in 4 byte units?
are there limits on the length?

I think one byte can be used as well as now it is 40 bytes in Windows. What is the best size that should be chosen in order to keep it future proof?

As a 1st step, could you add spec text explaining what it is?

Quoted 13 lines
Quoted 1 line

+\item Lastly the driver should fill the indirection table array

is an indirection table required? why not have a function produce the queue number?

Yes it is required as in Windows it is given by the OS. the indirection table is basically a map between the hash and cpu ids which correspond to queue ids.

Quoted 3 lines

in the
+     \field{indirection_table} field while setting the array length in +     \field{indirection_table_length}.

Quoted 6 lines

any limits on the length here?

currently the indirection table is 128 bytes in Windows. We can change the sizes to fit that case, but what about future compatibility? What do you think should be the optimal size of the fields in order to keep this future proof while keeping the overhead minimal.

As host must maintain this in memory, you can have host specify the maximum. guest will bail out if host does not support what it needs.

Quoted 10 lines
Quoted 3 lines

This structure is used by the device
+     for determining in which RX virt queue the packet should be placed.

How?

By applying the hash function on the packet while using the provided key in order to determine the hash for the packet and then use the indirection table to place the packet inside the suitable queue. This is hash function dependent implementation, but more elaboration can be added indeed.

I guess indirection table is independent?

Quoted 19 lines
Quoted 3 lines

+\end{itemize}
+Once the configuration phase is over successfully, the packets SHOULD

have the
> +\field{hash} field with the hash value that was calculated by the device.

Why not make this optional? I suspect it's not really useful e.g. for dpdk or xdp which do not generally use hardware offloads.

Good point, will make it optional.

Review

Quoted 2 lines
(quoted diff) +2 -0
+
+Whenever the driver wants to discard the current RSS settings, it can

send an
> +VIRTIO_NET_SM_CTRL_RSS_SET command along with rss structure that has > +RSS_HASH_FUNCTION_NONE the \field{hash_function} field.

Quoted 3 lines

And then what happens?

and then the device is configured with RSS settings

so NONE will enable RSS?

Quoted 12 lines

and MUST start distributing
the packets into the queues accordingly while attaching the calculated hash to the header.

Review

Quoted 2 lines
(quoted diff) +2 -0
+
+The driver MUST check the \field{ack} field value provided by the

device, in
> +case the value is not VIRTIO_NET_OK then the driver MUST handle failure and
> +retry another hash function or else give up.

Quoted 3 lines

Is there anything special here?

Not really just describing the error handling in detail.

If it applies to all commands maybe add it to generic code.

Quoted 9 lines
Quoted 3 lines

+
  \subparagraph{Legacy Interface: Automatic receive steering in multiqueue

mode}\label{sec:Device Types / Network Device / Device Operation / Control Virtqueue / Automatic receive steering in multiqueue mode / Legacy
Interface: Automatic receive steering in multiqueue mode}
>  When using the legacy interface, transitional devices and drivers >  MUST format \field{virtqueue_pairs}

Quoted 3 lines

conformance statements will need to be updated.

which statements for example?

I mean links are needed in conformance.tex

Quoted 12 lines
Quoted 8 lines

--
2.13.6

---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]

Quoted 5 lines

--
Respectfully,
Sameeh Jubran
Linkedin
Software Engineer @ Daynix.

---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]

author
#13 lore raw

Re: [virtio-dev] [PATCH v2 2/2] content: net: steering mode: Add RSS

Quoted 412 lines · Michael S. Tsirkin
Quoted 3 lines · Sameeh Jubran

On Tue, Apr 17, 2018 at 4:50 AM, Michael S. Tsirkin <[email protected]>

Quoted 5 lines · wrote:

On Mon, Apr 16, 2018 at 04:05:26PM +0300, Sameeh Jubran wrote:
> From: Sameeh Jubran <[email protected]>
>
> This commit introduces the RSS feature into virtio-net. It is

introduced

Quoted 1 line
Quoted 1 line

as a sub mode for a general command which configures the steering

mode.

Quoted 2 lines
Quoted 2 lines

Most modern high end network devices today support configurable

hash

Quoted 2 lines

functions,
> this commit introduces RSS - Receive Side Scaling - [1] to virtio

net

Quoted 9 lines

device.
>
> The RSS is a technology from Microsoft that boosts network device performance
> by efficiently distributing the traffic among the CPUs in a multiprocessor
> system.
>
> This feature is supported in most of the modern network cards as

well as

Quoted 2 lines

most
> modern OSes including Linux and Windows. It is worth mentioning

that both

Quoted 4 lines

DPDK
> and Hyper-v support RSS too.
>
> [1] https://docs.microsoft.com/en-us/windows-hardware/drivers/

network/

Quoted 5 lines
    ndis-receive-side-scaling2
    >
    > Signed-off-by: Sameeh Jubran <[email protected]>
    > ---
    >  content.tex | 114 ++++++++++++++++++++++++++++++
Quoted 8 lines
    ++++++++++++++++++++++++++++++
    >  1 file changed, 114 insertions(+)
    >
    > diff --git a/content.tex b/content.tex
    > index 3d538e8..8076147 100644
    > --- a/content.tex
    > +++ b/content.tex
    > @@ -4017,6 +4017,7 @@ The device MUST NOT queue packets on receive

queues

Quoted 9 lines
    greater than
    >  \begin{lstlisting}
    >  // steering mode flags
    >  #define STEERING_MODE_AUTO          1
    > +#define STEERING_MODE_RSS           2
    >
    >  struct virtio_net_steering_modes {
    >  le32 steering_modes;
    > @@ -4027,6 +4028,7 @@ le32 steering_mode;
Quoted 8 lines
    >  le32 command;
    >
    >      union {
    > +    struct virtio_net_rss rss_conf;
    >      }
    >  };
    >
    > @@ -4066,6 +4068,118 @@ If this feature has been negotiated, the

virtio

Quoted 6 lines

header has an additional
>
> This is the default steering mode, please refer to the "Automatic receive steering in multiqueue" section.
>
> +\subparagraph{Receive Side Scaling}{Device Types / Network Device

/

Quoted 37 lines
    Device Operation / Control Virtqueue / Steering mode / Receive Side
    Scaling}
    > +
    > +\begin{lstlisting}
    > +#define RSS_HASH_FUNCTION_NONE      1
    > +#define RSS_HASH_FUNCTION_TOEPLITZ  2
    > +#define RSS_HASH_FUNCTION_SYMMETRIC 3
    > +
    > +// Hash function fields
    > +#define RSS_HASH_FIELDS_IPV4          0x00000100
    > +#define RSS_HASH_FIELDS_TCP_IPV4      0x00000200
    > +#define RSS_HASH_FIELDS_IPV6          0x00000400
    > +#define RSS_HASH_FIELDS_IPV6_EX       0x00000800
    > +#define RSS_HASH_FIELDS_TCP_IPV6      0x00001000
    > +#define RSS_HASH_FIELDS_TCP_IPV6_EX   0x00002000
    > +
    > +struct virtio_net_rss_supported_hash{
    > +le32 hash_function;
    > +}
    > +
    > +struct virtio_net_rss {
    > +le32 hash_function;
    > +le32 hash_function_flags;
    > +le32 hash_key_length;
    > +le32 indirection_table_length;
    > +     struct {
    > +             le32 hash_key[hash_key_length];
    > +             le32 indirection_table[indirection_table_length];
    > +     }
    > +};
    > +
    > +#define VIRTIO_NET_SM_CTRL_RSS_GET_SUPPORTED_FUNCTIONS   0
    > +#define VIRTIO_NET_SM_CTRL_RSS_SET                       1


    Please add comments in the code as well.
Quoted 1 line

Do you mean that I should add more comments to the code above which

explains

Quoted 1 line

the structures and defines?

Exactly.

Quoted 6 lines

Review

Quoted 6 lines

+\end{lstlisting}

(quoted diff) +2 -0
+
+If the VIRTIO_NET_F_CTRL_STEERING_MODE is negotiated the driver

can send

Quoted 8 lines

control
> +commands for the RSS configuration.

Confused. Does VIRTIO_NET_F_CTRL_STEERING_MODE imply steering mode generally or RSS specifically?
How does a device with streering mode ctrl but without RSS look?

The steering mode provides an infrastructure to multiple modes and it is

not

Quoted 1 line

specific to RSS.

Then maybe this sentence should not imply that VIRTIO_NET_F_CTRL_STEERING_ MODE
allows RSS.

Quoted 2 lines

The default mode of the steering mode is the Automatic mode which is the default mode when MQ is enabled. I though this would be the

best

Quoted 1 line

option to provide backward compatibility and makes sense as we already

have

Quoted 2 lines

"Automatic steering mode" in the spec. VIRTIO_NET_F_CTRL_STEERING_MODE gives each mode it's own subset of

commands as

Quoted 6 lines

is the case in RSS.

Quoted 2 lines

For configuring RSS the virtio_net_steering_mode +should be filled. The \field{steering_mode} field should be

filled with

Quoted 2 lines

the STEERING_MODE_RSS
> +flag along with one of the VIRTIO_NET_SM_CTRL_RSS commands in the

\field

Quoted 7 lines

{command} field. The
> +\field{rss_conf} field should be used.
> +
> +The class VIRTIO_NET_CTRL_RSS has two commands:
> +
> +\begin{enumerate}
> +\item VIRTIO_NET_SM_CTRL_RSS_GET_SUPPORTED_FUNCTIONS returns the

hash

Quoted 3 lines

functions
> + supported by the device to the driver.
> +\item VIRTIO_NET_SM_CTRL_RSS_SET applies the new RSS

configuration. The

Quoted 2 lines

command is
> + used by the driver for setting RSS hash function, hash key

and

Quoted 4 lines

content.tex If this feature has been negotiated, the virtio header has an additional

Quoted 4 lines
content.tex +4 -0
+     indirection table in the device.
+\end{enumerate}
+
+\devicenormative{\subparagraph}{Receive Side Scaling}{Device

Types /

Quoted 1 line

Network Device / Device Operation / Control Virtqueue / Steering

mode /

Quoted 3 lines
    Receive Side Scaling}
    > +
    > +The device MUST fill the virtio_net_rss_supported_hash structure

with

Quoted 2 lines

the hash
> +functions it supports and return the structure to the driver.

Zero or

Quoted 2 lines

more
> +flags of the RSS_HASH_FUNCTION flags MUST be used to fill the

\field

Quoted 7 lines
    {hash_function}
    > +field.
    > +
    > +The device MUST drop all previous RSS configuration upon receiving
    > +VIRTIO_NET_SM_CTRL_RSS_SET command.
    > +
    > +The device MUST set the RSS configuration according to the

settings

Quoted 2 lines

provided as
> +follows, once the configuration process is completed the device

SHOULD

Quoted 2 lines

apply
> +the hash function to each of the incoming packets and distribute

the

Quoted 2 lines

packets
> +through the virqueues using the calculated hash and the

indirection

Quoted 6 lines

table
> +that were earlier provided by the driver.
> +
> +Setting RSS configuration
> +\begin{enumerate}
> +\item The driver fills all of the fields and passes them through

the

Quoted 4 lines

control
> + queue to the device.
> +
> +\item The device sets the RSS configuration as provided by the

driver.

Quoted 2 lines
Quoted 2 lines

+
+\item If the device successfully applied the configuration, on

each

Quoted 2 lines

packet
> + received the device MUST calculate the hashing for the

packet and

Quoted 1 line
Quoted 1 line

+ store it in the virtio-net header in the \field{hash} field

and the

Quoted 4 lines

content.tex If this feature has been negotiated, the virtio header has an additional

Quoted 4 lines
content.tex +4 -0
+     hash fields used in the calculation in rss_hash_type.
+\end{enumerate}
+
+In case of any unexpected values/ unsupported hash function the

driver

Quoted 7 lines
Quoted 1 line

+MUST return VIRTIO_NET_ERR in the \field{ack} field.

driver or the device?

All MUST statements belon in correct normative sections.

Quoted 7 lines

True, this is obviously a mistake, it should be the device instead of the driver.

Generally what is the text trying to say here?

That the device should handle errors and inform the drivers if there are

any

Quoted 5 lines

errors during the execution of the commands.

Quoted 2 lines

+
+\drivernormative{\subparagraph}{Receive Side Scaling}{Device

Types /

Quoted 1 line

Network Device / Device Operation / Control Virtqueue / Steering

mode /

Quoted 3 lines
    Receive Side Scaling}
    > +
    > +If the driver wants to set RSS hash it should fill the RSS

structure

Quoted 5 lines

fields
> +as follows:
> +
> +\begin{itemize}
> +\item The driver SHOULD choose the hash function that SHOULD be

used and

Quoted 2 lines

fill
> + it in the \field{hash_function} field along with the

appropriate

Quoted 2 lines

flags
> + in the \field{hash_function_flags} field. These flags

indicate to

Quoted 2 lines

the
> + device which packet fields MUST be used in the calculation

process

Quoted 3 lines

of
> + the hash.
> +\item Once the hash function has been chosen a suitable hash key

should

Quoted 6 lines

be set
> + in the \field{hash_key} field,

which key is suitable?

depends on the configuration, in Windows case the OS provide us with the

key,

Quoted 2 lines

however, in general it could be anything.

We need to somehow describe it in the spec :)

Quoted 7 lines
Quoted 2 lines

the length of the key should be stored
+ in the \field{hash_key_length} field.

in 4 byte units?
are there limits on the length?

I think one byte can be used as well as now it is 40 bytes in Windows.

What is

Quoted 1 line

the best size that should be chosen in order to keep it future proof?

As a 1st step, could you add spec text explaining what it is?

Quoted 6 lines
Quoted 1 line

+\item Lastly the driver should fill the indirection table array

is an indirection table required? why not have a function produce the queue number?

Yes it is required as in Windows it is given by the OS. the indirection

table

Quoted 1 line

is basically a map between the hash and cpu ids which correspond to

queue ids.

Quoted 4 lines
Quoted 4 lines

in the
+ \field{indirection_table} field while setting the array

length in

Quoted 5 lines
Quoted 1 line

+ \field{indirection_table_length}.

any limits on the length here?

currently the indirection table is 128 bytes in Windows. We can change

the

Quoted 1 line

sizes to fit that case, but what about future compatibility? What do you

think

Quoted 1 line

should be the optimal size of the fields in order to keep this future

proof

Quoted 1 line

while keeping the overhead minimal.

As host must maintain this in memory, you can have host specify the maximum. guest will bail out if host does not support what it needs.

Quoted 3 lines
Quoted 3 lines

This structure is used by the device
+ for determining in which RX virt queue the packet should be

placed.

Quoted 4 lines

How?

By applying the hash function on the packet while using the provided key

in

Quoted 1 line

order to determine the hash for the packet and then use the indirection

table

Quoted 1 line

to place the packet inside the suitable queue. This is hash function

dependent

Quoted 1 line

implementation, but more elaboration can be added indeed.

I guess indirection table is independent?

Quoted 3 lines
Quoted 3 lines

+\end{itemize}
+Once the configuration phase is over successfully, the packets

SHOULD

Quoted 4 lines

have the
> +\field{hash} field with the hash value that was calculated by the device.

Quoted 8 lines

Why not make this optional? I suspect it's not really useful e.g. for dpdk or xdp which do not generally use hardware offloads.

Good point, will make it optional.

Review

Quoted 2 lines
(quoted diff) +2 -0
+
+Whenever the driver wants to discard the current RSS settings, it

can

Quoted 2 lines

send an
> +VIRTIO_NET_SM_CTRL_RSS_SET command along with rss structure that

has

Quoted 5 lines
Quoted 1 line

+RSS_HASH_FUNCTION_NONE the \field{hash_function} field.

And then what happens?

and then the device is configured with RSS settings

so NONE will enable RSS?

Quoted 2 lines

and MUST start distributing
the packets into the queues accordingly while attaching the calculated

hash to

Quoted 7 lines

the header.

Review

Quoted 2 lines
(quoted diff) +2 -0
+
+The driver MUST check the \field{ack} field value provided by the

device, in
> +case the value is not VIRTIO_NET_OK then the driver MUST handle

failure

Quoted 6 lines

and
> +retry another hash function or else give up.

Is there anything special here?

Not really just describing the error handling in detail.

If it applies to all commands maybe add it to generic code.

Quoted 3 lines
Quoted 3 lines

+
\subparagraph{Legacy Interface: Automatic receive steering in

multiqueue

Quoted 1 line

mode}\label{sec:Device Types / Network Device / Device Operation /

Control

Quoted 5 lines

Virtqueue / Automatic receive steering in multiqueue mode / Legacy
Interface: Automatic receive steering in multiqueue mode} > When using the legacy interface, transitional devices and drivers > MUST format \field{virtqueue_pairs}

Quoted 6 lines
Quoted 3 lines

conformance statements will need to be updated.

which statements for example?

I mean links are needed in conformance.tex

Do you think that the Steering Mode should be in conformance?

Quoted 22 lines
Quoted 7 lines
Quoted 7 lines

--
2.13.6

------------------------------------------------------------

---------

Quoted 1 line
Quoted 1 line

To unsubscribe, e-mail: virtio-dev-unsubscribe@lists.

oasis-open.org

Quoted 1 line
Quoted 1 line

For additional commands, e-mail: [email protected]

open.org

Quoted 9 lines

--
Respectfully,
Sameeh Jubran
Linkedin
Software Engineer @ Daynix.

Signature · 4 lines
Respectfully,
*Sameeh Jubran*
*Linkedin <https://il.linkedin.com/pub/sameeh-jubran/87/747/a8a>*
*Software Engineer @ Daynix <http://www.daynix.com>.*
#14 lore raw

Re: [virtio-dev] [PATCH v2 2/2] content: net: steering mode: Add RSS

Quoted 1 line · Sameeh Jubran

Do you think that the Steering Mode should be in conformance?

Any conformance clauses must be linked to from the conformance chapter.

Signature · 5 lines
MST

---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]