virtio_net: support low and high rate of notification coalescing
PATCH v2 1/1 virtio_net: support low and high rate of notification coalescing
This patch adds a new feature, VIRTIO_NET_F_NOTF_COAL_LOW_HIGH, while clarifying/fixing existing coalescing concepts.
The new feature adds 4 new commands to VIRTIO_NET_CTRL_NOTF_COAL class:
- VIRTIO_NET_CTRL_NOTF_COAL_LOW_TX_SET - coalescing parameters to use when the packet rate is equal or lower then the low threshold for TX.
- VIRTIO_NET_CTRL_NOTF_COAL_LOW_RX_SET - coalescing parameters to use when the packet rate is equal or lower then the low threshold for RX.
- VIRTIO_NET_CTRL_NOTF_COAL_HIGH_TX_SET - coalescing parameters to use when the packet rate is equal or higher then the high threshold for TX.
- VIRTIO_NET_CTRL_NOTF_COAL_HIGH_RX_SET - coalescing parameters to use when the packet rate is equal or higher then the high threshold for RX.
On top of the new feature, this patch:
- Unifies the virtio_net_ctrl_coal structs, since the one for tx and the one for rx are identical.
- Clarifies that the coalescing commands are best-effort.
- Specifies coalescing in respect to delivering interrupts when config changes.
---
v2:
- Remove the "set of coalescing parameters" concept, use "coalescing parameters" instead.
- Unify struct virtio_net_ctrl_coal_rx and strcut virtio_net_ctrl_coal_tx to struct virtio_net_ctrl_coal.
- Separate the commands to tx and rx, so devices could have different low/high rate coalescing parameters for tx and rx.
- Unify struct virtio_net_ctrl_coal_high and struct virtio_net_ctrl_coal_low to struct virtio_net_ctrl_coal_threshold.
- Clarify that the packet rate is in packet per second units.
- Clarify that any notification coalescing command is best-effort.
- Specify coalescing in respect to delivering interrupts when config changes.
--- content.tex | 105 +++++++++++++++++++++++++++++++++++++++++++--------- 1 file changed, 87 insertions(+), 18 deletions(-)
content.tex +87 -18
@@ -3088,6 +3088,8 @@ \subsection{Feature bits}\label{sec:Device Types / Network Device / Feature bits \item[VIRTIO_NET_F_CTRL_MAC_ADDR(23)] Set MAC address through control channel. +\item[VIRTIO_NET_F_NOTF_COAL_LOW_HIGH(50)] Device supports notifications coalescing low rate and high rate sets. + \item[VIRTIO_NET_F_NOTF_COAL(53)] Device supports notifications coalescing. \item[VIRTIO_NET_F_GUEST_USO4 (54)] Driver can receive USOv4 packets.
@@ -3142,6 +3144,7 @@ \subsubsection{Feature bit requirements}\label{sec:Device Types / Network Device \item[VIRTIO_NET_F_MQ] Requires VIRTIO_NET_F_CTRL_VQ. \item[VIRTIO_NET_F_CTRL_MAC_ADDR] Requires VIRTIO_NET_F_CTRL_VQ. \item[VIRTIO_NET_F_NOTF_COAL] Requires VIRTIO_NET_F_CTRL_VQ. +\item[VIRTIO_NET_F_NOTF_COAL_LOW_HIGH] Requires VIRTIO_NET_F_NOTF_COAL. \item[VIRTIO_NET_F_RSC_EXT] Requires VIRTIO_NET_F_HOST_TSO4 or VIRTIO_NET_F_HOST_TSO6. \item[VIRTIO_NET_F_RSS] Requires VIRTIO_NET_F_CTRL_VQ. \end{description}
@@ -4493,43 +4496,62 @@ \subsubsection{Control Virtqueue}\label{sec:Device Types / Network Device / Devi If the VIRTIO_NET_F_NOTF_COAL feature is negotiated, the driver can send control commands for dynamically changing the coalescing parameters. +Note: In general, these commands are best-effort: A device could send a notification even if it is not supposed to. + \begin{lstlisting} -struct virtio_net_ctrl_coal_rx { - le32 rx_max_packets; - le32 rx_usecs; +struct virtio_net_ctrl_coal { + le32 max_packets; + le32 usecs; }; -struct virtio_net_ctrl_coal_tx { - le32 tx_max_packets; - le32 tx_usecs; +struct virtio_net_ctrl_coal_threshold { + le32 pkt_rate; + struct virtio_net_ctrl_coal params; }; #define VIRTIO_NET_CTRL_NOTF_COAL 6 #define VIRTIO_NET_CTRL_NOTF_COAL_TX_SET 0 #define VIRTIO_NET_CTRL_NOTF_COAL_RX_SET 1 + #define VIRTIO_NET_CTRL_NOTF_COAL_LOW_TX_SET 2 //Only if VIRTIO_NET_F_NOTF_COAL_LOW_HIGH negotiated + #define VIRTIO_NET_CTRL_NOTF_COAL_LOW_RX_SET 3 //Only if VIRTIO_NET_F_NOTF_COAL_LOW_HIGH negotiated + #define VIRTIO_NET_CTRL_NOTF_COAL_HIGH_TX_SET 4 //Only if VIRTIO_NET_F_NOTF_COAL_LOW_HIGH negotiated + #define VIRTIO_NET_CTRL_NOTF_COAL_HIGH_RX_SET 5 //Only if VIRTIO_NET_F_NOTF_COAL_LOW_HIGH negotiated \end{lstlisting} -Coalescing parameters: +TX Coalescing parameters: +\begin{itemize} +\item \field{usecs}: Maximum number of usecs to delay a TX notification. +\item \field{max_packets}: Maximum number of packets to send before a TX notification. +\end{itemize} + +RX Coalescing parameters: +\begin{itemize} +\item \field{usecs}: Maximum number of usecs to delay a RX notification. +\item \field{max_packets}: Maximum number of packets to receive before a RX notification. +\end{itemize} + +General Coalescing parameters: \begin{itemize} -\item \field{rx_usecs}: Maximum number of usecs to delay a RX notification. -\item \field{tx_usecs}: Maximum number of usecs to delay a TX notification. -\item \field{rx_max_packets}: Maximum number of packets to receive before a RX notification. -\item \field{tx_max_packets}: Maximum number of packets to send before a TX notification. +\item \field{pkt_rate}: Threshold for low/high packet rate coalescing parameters, in units of packets per second. \end{itemize} -The class VIRTIO_NET_CTRL_NOTF_COAL has 2 commands: +The class VIRTIO_NET_CTRL_NOTF_COAL has 6 commands: \begin{enumerate} -\item VIRTIO_NET_CTRL_NOTF_COAL_TX_SET: set the \field{tx_usecs} and \field{tx_max_packets} parameters. -\item VIRTIO_NET_CTRL_NOTF_COAL_RX_SET: set the \field{rx_usecs} and \field{rx_max_packets} parameters. +\item VIRTIO_NET_CTRL_NOTF_COAL_TX_SET: set the \field{usecs} and \field{max_packets} coalescing parameters for TX. +\item VIRTIO_NET_CTRL_NOTF_COAL_RX_SET: set the \field{usecs} and \field{max_packets} coalescing parameters for RX. +\item VIRTIO_NET_CTRL_NOTF_COAL_LOW_TX_SET: set the \field{usecs} and \field{max_packets} coalescing parameters for packet rate of \field{pkt_rate} or lower, for TX. +\item VIRTIO_NET_CTRL_NOTF_COAL_LOW_RX_SET: set the \field{usecs} and \field{max_packets} coalescing parameters for packet rate of \field{pkt_rate} or lower, for RX. +\item VIRTIO_NET_CTRL_NOTF_COAL_HIGH_TX_SET: set the \field{usecs} and \field{max_packets} coalescing parameters for packet rate of \field{pkt_rate} or higher, for TX. +\item VIRTIO_NET_CTRL_NOTF_COAL_HIGH_RX_SET: set the \field{usecs} and \field{max_packets} coalescing parameters for packet rate of \field{pkt_rate} or higher, for RX. \end{enumerate} \subparagraph{RX Notifications}\label{sec:Device Types / Network Device / Device Operation / Control Virtqueue / Notifications Coalescing / RX Notifications} If, for example: \begin{itemize} -\item \field{rx_usecs} = 10. -\item \field{rx_max_packets} = 15. +\item \field{usecs} = 10. +\item \field{max_packets} = 15. \end{itemize} The device will operate as follows:
@@ -4543,8 +4565,8 @@ \subsubsection{Control Virtqueue}\label{sec:Device Types / Network Device / Devi If, for example: \begin{itemize} -\item \field{tx_usecs} = 10. -\item \field{tx_max_packets} = 15. +\item \field{usecs} = 10. +\item \field{max_packets} = 15. \end{itemize} The device will operate as follows:
@@ -4554,18 +4576,65 @@ \subsubsection{Control Virtqueue}\label{sec:Device Types / Network Device / Devi \item If the notifications are not suppressed by the driver, the device will send an used buffer notification, otherwise, the device will not send an used buffer notification as long as the notifications are suppressed. \end{itemize} +\subparagraph{Notifications When Coalescing Parameters Change}\label{sec:Device Types / Network Device / Device Operation / Control Virtqueue / Notifications Coalescing / Notifications When Coalescing Parameters Change} + +When a device changes the coalescing parameters, the device needs to check if the new parameters are met and issue a notification if so. + +For example, \field{max_packets} = 15 for TX. + +If the device sends 10 packets, then it receives a VIRTIO_NET_CTRL_NOTF_COAL_TX_SET command with \field{max_packets} = 8, the device needs to immediately send a TX notification, if the notifications are not suppressed by the driver. + +\subparagraph{Low/High Rate Notifications}\label{sec:Device Types / Network Device / Device Operation / Control Virtqueue / Notifications Coalescing / Low/High Rate Notifications} + +If the VIRTIO_NET_F_NOTF_COAL_LOW_HIGH feature is negotiated, the driver can send the following low/high rate coalescing commands to the device: + +\begin{itemize} +\item VIRTIO_NET_CTRL_NOTF_COAL_LOW_TX_SET. +\item VIRTIO_NET_CTRL_NOTF_COAL_LOW_RX_SET. +\item VIRTIO_NET_CTRL_NOTF_COAL_HIGH_TX_SET. +\item VIRTIO_NET_CTRL_NOTF_COAL_HIGH_RX_SET. +\end{itemize} + +For VIRTIO_NET_CTRL_NOTF_COAL_LOW_TX_SET and VIRTIO_NET_CTRL_NOTF_COAL_LOW_RX_SET, struct virtio_net_ctrl_coal_threshold specifies the coalescing parameters to use when the packet rate is \field{pkt_rate} or lower. + +For VIRTIO_NET_CTRL_NOTF_COAL_HIGH_TX_SET and VIRTIO_NET_CTRL_NOTF_COAL_HIGH_RX_SET, struct virtio_net_ctrl_coal_threshold specifies the coalescing parameters to use when the packet rate is \field{pkt_rate} or higher. + \drivernormative{\subparagraph}{Notifications Coalescing}{Device Types / Network Device / Device Operation / Control Virtqueue / Notifications Coalescing} If the VIRTIO_NET_F_NOTF_COAL feature has not been negotiated, the driver MUST NOT issue VIRTIO_NET_CTRL_NOTF_COAL commands. +If the VIRTIO_NET_F_NOTF_COAL_LOW_HIGH feature has not been negotiated, the driver MUST NOT issue low/high rate coalescing commands. + +The driver SHOULD issue a low/high rate coalescing command with \field{pkt_rate} 0 in order to remove the low/high rate coalescing parameters. + +The driver MUST NOT issue a VIRTIO_NET_CTRL_NOTF_COAL_LOW_TX_SET command with \field{pkt_rate} equal or higher than a \field{pkt_rate} previously sent with a VIRTIO_NET_CTRL_NOTF_COAL_HIGH_TX_SET command. + +The driver MUST NOT issue a VIRTIO_NET_CTRL_NOTF_COAL_LOW_RX_SET command with \field{pkt_rate} equal or higher than a \field{pkt_rate} previously sent with a VIRTIO_NET_CTRL_NOTF_COAL_HIGH_RX_SET command. + +The driver MUST NOT issue a VIRTIO_NET_CTRL_NOTF_COAL_HIGH_TX_SET command with \field{pkt_rate} equal or lower than a \field{pkt_rate} previously sent with a VIRTIO_NET_CTRL_NOTF_COAL_LOW_TX_SET command. + +The driver MUST NOT issue a VIRTIO_NET_CTRL_NOTF_COAL_HIGH_RX_SET command with \field{pkt_rate} equal or lower than a \field{pkt_rate} previously sent with a VIRTIO_NET_CTRL_NOTF_COAL_LOW_RX_SET command. + \devicenormative{\subparagraph}{Notifications Coalescing}{Device Types / Network Device / Device Operation / Control Virtqueue / Notifications Coalescing} A device SHOULD respond to the VIRTIO_NET_CTRL_NOTF_COAL commands with VIRTIO_NET_ERR if it was not able to change the parameters. A device SHOULD NOT send used buffer notifications to the driver, if the notifications are suppressed as explained in \ref{sec:Basic Facilities of a Virtio Device / Virtqueues / Used Buffer Notification Suppression}, even if the coalescing counters expired. +A device MUST remove the low/high rate coalescing parameters, if a low/high rate coalescing command is received with \field{pkt_rate} 0. + +A device MUST respond with VIRTIO_NET_ERR to a VIRTIO_NET_CTRL_NOTF_COAL_LOW_TX_SET command with \field{pkt_rate} equal or higher than a \field{pkt_rate} previously received with a VIRTIO_NET_CTRL_NOTF_COAL_HIGH_TX_SET command. + +A device MUST respond with VIRTIO_NET_ERR to a VIRTIO_NET_CTRL_NOTF_COAL_LOW_RX_SET command with \field{pkt_rate} equal or higher than a \field{pkt_rate} previously received with a VIRTIO_NET_CTRL_NOTF_COAL_HIGH_RX_SET command. + +A device MUST respond with VIRTIO_NET_ERR to a VIRTIO_NET_CTRL_NOTF_COAL_HIGH_TX_SET command with \field{pkt_rate} equal or lower than a \field{pkt_rate} previously received with a VIRTIO_NET_CTRL_NOTF_COAL_LOW_TX_SET command. + +A device MUST respond with VIRTIO_NET_ERR to a VIRTIO_NET_CTRL_NOTF_COAL_HIGH_RX_SET command with \field{pkt_rate} equal or lower than a \field{pkt_rate} previously received with a VIRTIO_NET_CTRL_NOTF_COAL_LOW_RX_SET command. + Upon reset, a device MUST initialize all coalescing parameters to 0. +Upon reset, a device MUST not have a low/high rate coalescing parameters. + \subsubsection{Legacy Interface: Framing Requirements}\label{sec:Device Types / Network Device / Legacy Interface: Framing Requirements}
Signature · 6 lines
2.32.0 --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
After another inspection, I found some typos in the patch, will be fixed in the next version.
Quoted 8 lines
This patch adds a new feature,
VIRTIO_NET_F_NOTF_COAL_LOW_HIGH, while clarifying/fixing existing coalescing concepts.The new feature adds 4 new commands to
VIRTIO_NET_CTRL_NOTF_COALclass:
-VIRTIO_NET_CTRL_NOTF_COAL_LOW_TX_SET- coalescing parameters to use when the packet rate is equal or lower then the low threshold for TX.
s/then/than/
Quoted 3 lines
-
VIRTIO_NET_CTRL_NOTF_COAL_LOW_RX_SET- coalescing parameters to use when the packet rate is equal or lower then the low threshold for RX.
s/then/than/
Quoted 3 lines
-
VIRTIO_NET_CTRL_NOTF_COAL_HIGH_TX_SET- coalescing parameters to use when the packet rate is equal or higher then the high threshold for TX.
s/then/than/
Quoted 3 lines
-
VIRTIO_NET_CTRL_NOTF_COAL_HIGH_RX_SET- coalescing parameters to use when the packet rate is equal or higher then the high threshold for RX.
s/then/than/
Quoted 7 lines
On top of the new feature, this patch:
- Unifies thevirtio_net_ctrl_coalstructs, since the one for tx and the one for rx are identical.
- Clarifies that the coalescing commands are best-effort.
- Specifies coalescing in respect to delivering interrupts when config changes.
Quoted 6 lines
Signed-off-by Alvaro Karsz <[email protected]>---
v2:
- Remove the "set of coalescing parameters" concept, use "coalescing parameters" instead.
- Unify structvirtio_net_ctrl_coal_rxand strcutvirtio_net_ctrl_coal_tx
s/strcut/struct
Quoted 18 lines
to struct
virtio_net_ctrl_coal.
- Separate the commands to tx and rx, so devices could have different low/high rate coalescing parameters for tx and rx.
- Unify structvirtio_net_ctrl_coal_highand structvirtio_net_ctrl_coal_lowto structvirtio_net_ctrl_coal_threshold.
- Clarify that the packet rate is in packet per second units.
- Clarify that any notification coalescing command is best-effort.
- Specify coalescing in respect to delivering interrupts when config changes.--- content.tex | 105 +++++++++++++++++++++++++++++++++++++++++++--------- 1 file changed, 87 insertions(+), 18 deletions(-)content.tex +0
-0
content.tex \subsection{Feature bits}\label{sec:Device Types / Network Device / Feature bits · line 3092
Quoted 5 lines
content.tex +2
-0@@ -3088,6 +3088,8 @@ \subsection{Feature bits}\label{sec:Device Types / Network Device / Feature bits \item[VIRTIO_NET_F_CTRL_MAC_ADDR(23)] Set MAC address through control channel. +\item[VIRTIO_NET_F_NOTF_COAL_LOW_HIGH(50)] Device supports notifications coalescing low rate and high rate sets. +
Replace with: Device supports low/high packet rate coalescing parameters.
Quoted 168 lines
content.tex +85
-19\item[VIRTIO_NET_F_NOTF_COAL(53)] Device supports notifications coalescing. \item[VIRTIO_NET_F_GUEST_USO4 (54)] Driver can receive USOv4 packets.@@ -3142,6 +3144,7 @@ \subsubsection{Feature bit requirements}\label{sec:Device Types / Network Device \item[VIRTIO_NET_F_MQ] Requires VIRTIO_NET_F_CTRL_VQ. \item[VIRTIO_NET_F_CTRL_MAC_ADDR] Requires VIRTIO_NET_F_CTRL_VQ. \item[VIRTIO_NET_F_NOTF_COAL] Requires VIRTIO_NET_F_CTRL_VQ. +\item[VIRTIO_NET_F_NOTF_COAL_LOW_HIGH] Requires VIRTIO_NET_F_NOTF_COAL. \item[VIRTIO_NET_F_RSC_EXT] Requires VIRTIO_NET_F_HOST_TSO4 or VIRTIO_NET_F_HOST_TSO6. \item[VIRTIO_NET_F_RSS] Requires VIRTIO_NET_F_CTRL_VQ. \end{description}@@ -4493,43 +4496,62 @@ \subsubsection{Control Virtqueue}\label{sec:Device Types / Network Device / Devi If the VIRTIO_NET_F_NOTF_COAL feature is negotiated, the driver can send control commands for dynamically changing the coalescing parameters. +Note: In general, these commands are best-effort: A device could send a notification even if it is not supposed to. + \begin{lstlisting} -struct virtio_net_ctrl_coal_rx { - le32 rx_max_packets; - le32 rx_usecs; +struct virtio_net_ctrl_coal { + le32 max_packets; + le32 usecs; }; -struct virtio_net_ctrl_coal_tx { - le32 tx_max_packets; - le32 tx_usecs; +struct virtio_net_ctrl_coal_threshold { + le32 pkt_rate; + struct virtio_net_ctrl_coal params; }; #define VIRTIO_NET_CTRL_NOTF_COAL 6 #define VIRTIO_NET_CTRL_NOTF_COAL_TX_SET 0 #define VIRTIO_NET_CTRL_NOTF_COAL_RX_SET 1 + #define VIRTIO_NET_CTRL_NOTF_COAL_LOW_TX_SET 2 //Only if VIRTIO_NET_F_NOTF_COAL_LOW_HIGH negotiated + #define VIRTIO_NET_CTRL_NOTF_COAL_LOW_RX_SET 3 //Only if VIRTIO_NET_F_NOTF_COAL_LOW_HIGH negotiated + #define VIRTIO_NET_CTRL_NOTF_COAL_HIGH_TX_SET 4 //Only if VIRTIO_NET_F_NOTF_COAL_LOW_HIGH negotiated + #define VIRTIO_NET_CTRL_NOTF_COAL_HIGH_RX_SET 5 //Only if VIRTIO_NET_F_NOTF_COAL_LOW_HIGH negotiated \end{lstlisting} -Coalescing parameters: +TX Coalescing parameters: +\begin{itemize} +\item \field{usecs}: Maximum number of usecs to delay a TX notification. +\item \field{max_packets}: Maximum number of packets to send before a TX notification. +\end{itemize} + +RX Coalescing parameters: +\begin{itemize} +\item \field{usecs}: Maximum number of usecs to delay a RX notification. +\item \field{max_packets}: Maximum number of packets to receive before a RX notification. +\end{itemize} + +General Coalescing parameters: \begin{itemize} -\item \field{rx_usecs}: Maximum number of usecs to delay a RX notification. -\item \field{tx_usecs}: Maximum number of usecs to delay a TX notification. -\item \field{rx_max_packets}: Maximum number of packets to receive before a RX notification. -\item \field{tx_max_packets}: Maximum number of packets to send before a TX notification. +\item \field{pkt_rate}: Threshold for low/high packet rate coalescing parameters, in units of packets per second. \end{itemize} -The class VIRTIO_NET_CTRL_NOTF_COAL has 2 commands: +The class VIRTIO_NET_CTRL_NOTF_COAL has 6 commands: \begin{enumerate} -\item VIRTIO_NET_CTRL_NOTF_COAL_TX_SET: set the \field{tx_usecs} and \field{tx_max_packets} parameters. -\item VIRTIO_NET_CTRL_NOTF_COAL_RX_SET: set the \field{rx_usecs} and \field{rx_max_packets} parameters. +\item VIRTIO_NET_CTRL_NOTF_COAL_TX_SET: set the \field{usecs} and \field{max_packets} coalescing parameters for TX. +\item VIRTIO_NET_CTRL_NOTF_COAL_RX_SET: set the \field{usecs} and \field{max_packets} coalescing parameters for RX. +\item VIRTIO_NET_CTRL_NOTF_COAL_LOW_TX_SET: set the \field{usecs} and \field{max_packets} coalescing parameters for packet rate of \field{pkt_rate} or lower, for TX. +\item VIRTIO_NET_CTRL_NOTF_COAL_LOW_RX_SET: set the \field{usecs} and \field{max_packets} coalescing parameters for packet rate of \field{pkt_rate} or lower, for RX. +\item VIRTIO_NET_CTRL_NOTF_COAL_HIGH_TX_SET: set the \field{usecs} and \field{max_packets} coalescing parameters for packet rate of \field{pkt_rate} or higher, for TX. +\item VIRTIO_NET_CTRL_NOTF_COAL_HIGH_RX_SET: set the \field{usecs} and \field{max_packets} coalescing parameters for packet rate of \field{pkt_rate} or higher, for RX. \end{enumerate} \subparagraph{RX Notifications}\label{sec:Device Types / Network Device / Device Operation / Control Virtqueue / Notifications Coalescing / RX Notifications} If, for example: \begin{itemize} -\item \field{rx_usecs} = 10. -\item \field{rx_max_packets} = 15. +\item \field{usecs} = 10. +\item \field{max_packets} = 15. \end{itemize} The device will operate as follows:@@ -4543,8 +4565,8 @@ \subsubsection{Control Virtqueue}\label{sec:Device Types / Network Device / Devi If, for example: \begin{itemize} -\item \field{tx_usecs} = 10. -\item \field{tx_max_packets} = 15. +\item \field{usecs} = 10. +\item \field{max_packets} = 15. \end{itemize} The device will operate as follows:@@ -4554,18 +4576,65 @@ \subsubsection{Control Virtqueue}\label{sec:Device Types / Network Device / Devi \item If the notifications are not suppressed by the driver, the device will send an used buffer notification, otherwise, the device will not send an used buffer notification as long as the notifications are suppressed. \end{itemize} +\subparagraph{Notifications When Coalescing Parameters Change}\label{sec:Device Types / Network Device / Device Operation / Control Virtqueue / Notifications Coalescing / Notifications When Coalescing Parameters Change} + +When a device changes the coalescing parameters, the device needs to check if the new parameters are met and issue a notification if so. + +For example, \field{max_packets} = 15 for TX. + +If the device sends 10 packets, then it receives a VIRTIO_NET_CTRL_NOTF_COAL_TX_SET command with \field{max_packets} = 8, the device needs to immediately send a TX notification, if the notifications are not suppressed by the driver. + +\subparagraph{Low/High Rate Notifications}\label{sec:Device Types / Network Device / Device Operation / Control Virtqueue / Notifications Coalescing / Low/High Rate Notifications} + +If the VIRTIO_NET_F_NOTF_COAL_LOW_HIGH feature is negotiated, the driver can send the following low/high rate coalescing commands to the device: + +\begin{itemize} +\item VIRTIO_NET_CTRL_NOTF_COAL_LOW_TX_SET. +\item VIRTIO_NET_CTRL_NOTF_COAL_LOW_RX_SET. +\item VIRTIO_NET_CTRL_NOTF_COAL_HIGH_TX_SET. +\item VIRTIO_NET_CTRL_NOTF_COAL_HIGH_RX_SET. +\end{itemize} + +For VIRTIO_NET_CTRL_NOTF_COAL_LOW_TX_SET and VIRTIO_NET_CTRL_NOTF_COAL_LOW_RX_SET, struct virtio_net_ctrl_coal_threshold specifies the coalescing parameters to use when the packet rate is \field{pkt_rate} or lower. + +For VIRTIO_NET_CTRL_NOTF_COAL_HIGH_TX_SET and VIRTIO_NET_CTRL_NOTF_COAL_HIGH_RX_SET, struct virtio_net_ctrl_coal_threshold specifies the coalescing parameters to use when the packet rate is \field{pkt_rate} or higher. + \drivernormative{\subparagraph}{Notifications Coalescing}{Device Types / Network Device / Device Operation / Control Virtqueue / Notifications Coalescing} If the VIRTIO_NET_F_NOTF_COAL feature has not been negotiated, the driver MUST NOT issue VIRTIO_NET_CTRL_NOTF_COAL commands. +If the VIRTIO_NET_F_NOTF_COAL_LOW_HIGH feature has not been negotiated, the driver MUST NOT issue low/high rate coalescing commands. + +The driver SHOULD issue a low/high rate coalescing command with \field{pkt_rate} 0 in order to remove the low/high rate coalescing parameters. + +The driver MUST NOT issue a VIRTIO_NET_CTRL_NOTF_COAL_LOW_TX_SET command with \field{pkt_rate} equal or higher than a \field{pkt_rate} previously sent with a VIRTIO_NET_CTRL_NOTF_COAL_HIGH_TX_SET command. + +The driver MUST NOT issue a VIRTIO_NET_CTRL_NOTF_COAL_LOW_RX_SET command with \field{pkt_rate} equal or higher than a \field{pkt_rate} previously sent with a VIRTIO_NET_CTRL_NOTF_COAL_HIGH_RX_SET command. + +The driver MUST NOT issue a VIRTIO_NET_CTRL_NOTF_COAL_HIGH_TX_SET command with \field{pkt_rate} equal or lower than a \field{pkt_rate} previously sent with a VIRTIO_NET_CTRL_NOTF_COAL_LOW_TX_SET command. + +The driver MUST NOT issue a VIRTIO_NET_CTRL_NOTF_COAL_HIGH_RX_SET command with \field{pkt_rate} equal or lower than a \field{pkt_rate} previously sent with a VIRTIO_NET_CTRL_NOTF_COAL_LOW_RX_SET command. + \devicenormative{\subparagraph}{Notifications Coalescing}{Device Types / Network Device / Device Operation / Control Virtqueue / Notifications Coalescing} A device SHOULD respond to the VIRTIO_NET_CTRL_NOTF_COAL commands with VIRTIO_NET_ERR if it was not able to change the parameters. A device SHOULD NOT send used buffer notifications to the driver, if the notifications are suppressed as explained in \ref{sec:Basic Facilities of a Virtio Device / Virtqueues / Used Buffer Notification Suppression}, even if the coalescing counters expired. +A device MUST remove the low/high rate coalescing parameters, if a low/high rate coalescing command is received with \field{pkt_rate} 0. + +A device MUST respond with VIRTIO_NET_ERR to a VIRTIO_NET_CTRL_NOTF_COAL_LOW_TX_SET command with \field{pkt_rate} equal or higher than a \field{pkt_rate} previously received with a VIRTIO_NET_CTRL_NOTF_COAL_HIGH_TX_SET command. + +A device MUST respond with VIRTIO_NET_ERR to a VIRTIO_NET_CTRL_NOTF_COAL_LOW_RX_SET command with \field{pkt_rate} equal or higher than a \field{pkt_rate} previously received with a VIRTIO_NET_CTRL_NOTF_COAL_HIGH_RX_SET command. + +A device MUST respond with VIRTIO_NET_ERR to a VIRTIO_NET_CTRL_NOTF_COAL_HIGH_TX_SET command with \field{pkt_rate} equal or lower than a \field{pkt_rate} previously received with a VIRTIO_NET_CTRL_NOTF_COAL_LOW_TX_SET command. + +A device MUST respond with VIRTIO_NET_ERR to a VIRTIO_NET_CTRL_NOTF_COAL_HIGH_RX_SET command with \field{pkt_rate} equal or lower than a \field{pkt_rate} previously received with a VIRTIO_NET_CTRL_NOTF_COAL_LOW_RX_SET command. + Upon reset, a device MUST initialize all coalescing parameters to 0. +Upon reset, a device MUST not have a low/high rate coalescing parameters. + \subsubsection{Legacy Interface: Framing Requirements}\label{sec:Device Types / Network Device / Legacy Interface: Framing Requirements} --
Quoted 2 lines
2.32.0
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
ping
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Quoted 1 line · Alvaro Karsz
ping
You only posted v2 Wednesday, that's just 2 days for people to review. Sit tight next time pls :)
Signature · 18 lines
MST This publicly archived list offers a means to provide input to the OASIS Virtual I/O Device (VIRTIO) TC. In order to verify user consent to the Feedback License terms and to minimize spam in the list archive, subscription is required before posting. Subscribe: [email protected] Unsubscribe: [email protected] List help: [email protected] List archive: https://lists.oasis-open.org/archives/virtio-comment/ Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists Committee: https://www.oasis-open.org/committees/virtio/ Join OASIS: https://www.oasis-open.org/join/
Quoted 84 lines · Alvaro Karsz
This patch adds a new feature,
VIRTIO_NET_F_NOTF_COAL_LOW_HIGH, while clarifying/fixing existing coalescing concepts.The new feature adds 4 new commands to
VIRTIO_NET_CTRL_NOTF_COALclass:
-VIRTIO_NET_CTRL_NOTF_COAL_LOW_TX_SET- coalescing parameters to use when the packet rate is equal or lower then the low threshold for TX.
-VIRTIO_NET_CTRL_NOTF_COAL_LOW_RX_SET- coalescing parameters to use when the packet rate is equal or lower then the low threshold for RX.
-VIRTIO_NET_CTRL_NOTF_COAL_HIGH_TX_SET- coalescing parameters to use when the packet rate is equal or higher then the high threshold for TX.
-VIRTIO_NET_CTRL_NOTF_COAL_HIGH_RX_SET- coalescing parameters to use when the packet rate is equal or higher then the high threshold for RX.On top of the new feature, this patch:
- Unifies thevirtio_net_ctrl_coalstructs, since the one for tx and the one for rx are identical.
- Clarifies that the coalescing commands are best-effort.
- Specifies coalescing in respect to delivering interrupts when config changes.Signed-off-by Alvaro Karsz <[email protected]>---
v2:
- Remove the "set of coalescing parameters" concept, use "coalescing parameters" instead.
- Unify structvirtio_net_ctrl_coal_rxand strcutvirtio_net_ctrl_coal_txto structvirtio_net_ctrl_coal.
- Separate the commands to tx and rx, so devices could have different low/high rate coalescing parameters for tx and rx.
- Unify structvirtio_net_ctrl_coal_highand structvirtio_net_ctrl_coal_lowto structvirtio_net_ctrl_coal_threshold.
- Clarify that the packet rate is in packet per second units.
- Clarify that any notification coalescing command is best-effort.
- Specify coalescing in respect to delivering interrupts when config changes.--- content.tex | 105 +++++++++++++++++++++++++++++++++++++++++++--------- 1 file changed, 87 insertions(+), 18 deletions(-)content.tex +10
-6@@ -3088,6 +3088,8 @@ \subsection{Feature bits}\label{sec:Device Types / Network Device / Feature bits \item[VIRTIO_NET_F_CTRL_MAC_ADDR(23)] Set MAC address through control channel. +\item[VIRTIO_NET_F_NOTF_COAL_LOW_HIGH(50)] Device supports notifications coalescing low rate and high rate sets. + \item[VIRTIO_NET_F_NOTF_COAL(53)] Device supports notifications coalescing. \item[VIRTIO_NET_F_GUEST_USO4 (54)] Driver can receive USOv4 packets.@@ -3142,6 +3144,7 @@ \subsubsection{Feature bit requirements}\label{sec:Device Types / Network Device \item[VIRTIO_NET_F_MQ] Requires VIRTIO_NET_F_CTRL_VQ. \item[VIRTIO_NET_F_CTRL_MAC_ADDR] Requires VIRTIO_NET_F_CTRL_VQ. \item[VIRTIO_NET_F_NOTF_COAL] Requires VIRTIO_NET_F_CTRL_VQ. +\item[VIRTIO_NET_F_NOTF_COAL_LOW_HIGH] Requires VIRTIO_NET_F_NOTF_COAL. \item[VIRTIO_NET_F_RSC_EXT] Requires VIRTIO_NET_F_HOST_TSO4 or VIRTIO_NET_F_HOST_TSO6. \item[VIRTIO_NET_F_RSS] Requires VIRTIO_NET_F_CTRL_VQ. \end{description}@@ -4493,43 +4496,62 @@ \subsubsection{Control Virtqueue}\label{sec:Device Types / Network Device / Devi If the VIRTIO_NET_F_NOTF_COAL feature is negotiated, the driver can send control commands for dynamically changing the coalescing parameters. +Note: In general, these commands are best-effort: A device could send a notification even if it is not supposed to. + \begin{lstlisting} -struct virtio_net_ctrl_coal_rx { - le32 rx_max_packets; - le32 rx_usecs; +struct virtio_net_ctrl_coal { + le32 max_packets; + le32 usecs; }; -struct virtio_net_ctrl_coal_tx { - le32 tx_max_packets; - le32 tx_usecs; +struct virtio_net_ctrl_coal_threshold { + le32 pkt_rate;
content.tex \subsubsection{Control Virtqueue}\label{sec:Device Types / Network Device / Devi · line 4518
Quoted 10 lines
content.tex +5
-0+ struct virtio_net_ctrl_coal params; }; #define VIRTIO_NET_CTRL_NOTF_COAL 6 #define VIRTIO_NET_CTRL_NOTF_COAL_TX_SET 0 #define VIRTIO_NET_CTRL_NOTF_COAL_RX_SET 1 + #define VIRTIO_NET_CTRL_NOTF_COAL_LOW_TX_SET 2 //Only if VIRTIO_NET_F_NOTF_COAL_LOW_HIGH negotiated + #define VIRTIO_NET_CTRL_NOTF_COAL_LOW_RX_SET 3 //Only if VIRTIO_NET_F_NOTF_COAL_LOW_HIGH negotiated + #define VIRTIO_NET_CTRL_NOTF_COAL_HIGH_TX_SET 4 //Only if VIRTIO_NET_F_NOTF_COAL_LOW_HIGH negotiated + #define VIRTIO_NET_CTRL_NOTF_COAL_HIGH_RX_SET 5 //Only if VIRTIO_NET_F_NOTF_COAL_LOW_HIGH negotiated
add conformance statements for this requirement of VIRTIO_NET_F_NOTF_COAL_LOW_HIGH too?
Quoted 4 lines
content.tex +1
-1\end{lstlisting} -Coalescing parameters: +TX Coalescing parameters:
content.tex \subsubsection{Control Virtqueue}\label{sec:Device Types / Network Device / Devi · line 4531
Quoted 10 lines
content.tex +10
-0+\begin{itemize} +\item \field{usecs}: Maximum number of usecs to delay a TX notification. +\item \field{max_packets}: Maximum number of packets to send before a TX notification. +\end{itemize} + +RX Coalescing parameters: +\begin{itemize} +\item \field{usecs}: Maximum number of usecs to delay a RX notification. +\item \field{max_packets}: Maximum number of packets to receive before a RX notification. +\end{itemize}
just unify these - TX/RX notification?
content.tex \subsubsection{Control Virtqueue}\label{sec:Device Types / Network Device / Devi · line 4535
Quoted 8 lines
content.tex +3
-4+ +General Coalescing parameters: \begin{itemize} -\item \field{rx_usecs}: Maximum number of usecs to delay a RX notification. -\item \field{tx_usecs}: Maximum number of usecs to delay a TX notification. -\item \field{rx_max_packets}: Maximum number of packets to receive before a RX notification. -\item \field{tx_max_packets}: Maximum number of packets to send before a TX notification. +\item \field{pkt_rate}: Threshold for low/high packet rate coalescing parameters, in units of packets per second.
This isn't really clear. Besides it seems that with 100GB/s we can thinkably begin to overflow a 32 bit value pretty soon.
Also, how is this measured?
Do you expect device to average the rate over a whole second to decide? Ethernet speeds start at 1MBit/sec right? I think these days if we ask for # of packets in the last N usecs we won't go far off the field. How much is N so as not to incur too much latency? 100?
Or are we measuring X latest packets and testing how long it takes to get them?
Quoted 4 lines
content.tex +0
-1\end{itemize} -The class VIRTIO_NET_CTRL_NOTF_COAL has 2 commands:
content.tex \subsubsection{Control Virtqueue}\label{sec:Device Types / Network Device / Devi · line 4546
Quoted 10 lines
content.tex +7
-2+The class VIRTIO_NET_CTRL_NOTF_COAL has 6 commands: \begin{enumerate} -\item VIRTIO_NET_CTRL_NOTF_COAL_TX_SET: set the \field{tx_usecs} and \field{tx_max_packets} parameters. -\item VIRTIO_NET_CTRL_NOTF_COAL_RX_SET: set the \field{rx_usecs} and \field{rx_max_packets} parameters. +\item VIRTIO_NET_CTRL_NOTF_COAL_TX_SET: set the \field{usecs} and \field{max_packets} coalescing parameters for TX. +\item VIRTIO_NET_CTRL_NOTF_COAL_RX_SET: set the \field{usecs} and \field{max_packets} coalescing parameters for RX. +\item VIRTIO_NET_CTRL_NOTF_COAL_LOW_TX_SET: set the \field{usecs} and \field{max_packets} coalescing parameters for packet rate of \field{pkt_rate} or lower, for TX. +\item VIRTIO_NET_CTRL_NOTF_COAL_LOW_RX_SET: set the \field{usecs} and \field{max_packets} coalescing parameters for packet rate of \field{pkt_rate} or lower, for RX. +\item VIRTIO_NET_CTRL_NOTF_COAL_HIGH_TX_SET: set the \field{usecs} and \field{max_packets} coalescing parameters for packet rate of \field{pkt_rate} or higher, for TX. +\item VIRTIO_NET_CTRL_NOTF_COAL_HIGH_RX_SET: set the \field{usecs} and \field{max_packets} coalescing parameters for packet rate of \field{pkt_rate} or higher, for RX.
how is
content.tex \subsubsection{Control Virtqueue}\label{sec:Device Types / Network Device / Devi · line 4548
Quoted 2 lines
content.tex +0
-0\end{enumerate}
What is missing here is the theory of operation.
when are parameters set by VIRTIO_NET_CTRL_NOTF_COAL_RX_SET/VIRTIO_NET_CTRL_NOTF_COAL_TX_SET used?
Am I guessing right that with VIRTIO_NET_F_NOTF_COAL_LOW_HIGH there are 3 sets of parameters high medium and low, for rx and tx?
Quoted 23 lines
content.tex +4
-4\subparagraph{RX Notifications}\label{sec:Device Types / Network Device / Device Operation / Control Virtqueue / Notifications Coalescing / RX Notifications} If, for example: \begin{itemize} -\item \field{rx_usecs} = 10. -\item \field{rx_max_packets} = 15. +\item \field{usecs} = 10. +\item \field{max_packets} = 15. \end{itemize} The device will operate as follows:@@ -4543,8 +4565,8 @@ \subsubsection{Control Virtqueue}\label{sec:Device Types / Network Device / Devi If, for example: \begin{itemize} -\item \field{tx_usecs} = 10. -\item \field{tx_max_packets} = 15. +\item \field{usecs} = 10. +\item \field{max_packets} = 15. \end{itemize} The device will operate as follows:
content.tex \subsubsection{Control Virtqueue}\label{sec:Device Types / Network Device / Devi · line 4584
Quoted 9 lines
content.tex +6
-0@@ -4554,18 +4576,65 @@ \subsubsection{Control Virtqueue}\label{sec:Device Types / Network Device / Devi \item If the notifications are not suppressed by the driver, the device will send an used buffer notification, otherwise, the device will not send an used buffer notification as long as the notifications are suppressed. \end{itemize} +\subparagraph{Notifications When Coalescing Parameters Change}\label{sec:Device Types / Network Device / Device Operation / Control Virtqueue / Notifications Coalescing / Notifications When Coalescing Parameters Change} + +When a device changes the coalescing parameters, the device needs to check if the new parameters are met and issue a notification if so. + +For example, \field{max_packets} = 15 for TX. +
drop empty line here pls.
content.tex \subsubsection{Control Virtqueue}\label{sec:Device Types / Network Device / Devi · line 4597
Quoted 13 lines
content.tex +13
-0+If the device sends 10 packets, then it receives a VIRTIO_NET_CTRL_NOTF_COAL_TX_SET command with \field{max_packets} = 8, the device needs to immediately send a TX notification, if the notifications are not suppressed by the driver. + +\subparagraph{Low/High Rate Notifications}\label{sec:Device Types / Network Device / Device Operation / Control Virtqueue / Notifications Coalescing / Low/High Rate Notifications} + +If the VIRTIO_NET_F_NOTF_COAL_LOW_HIGH feature is negotiated, the driver can send the following low/high rate coalescing commands to the device: + +\begin{itemize} +\item VIRTIO_NET_CTRL_NOTF_COAL_LOW_TX_SET. +\item VIRTIO_NET_CTRL_NOTF_COAL_LOW_RX_SET. +\item VIRTIO_NET_CTRL_NOTF_COAL_HIGH_TX_SET. +\item VIRTIO_NET_CTRL_NOTF_COAL_HIGH_RX_SET. +\end{itemize} +
pls explain here how these commands all use virtio_net_ctrl_coal_threshold.
and maybe rearrange so the explanation is closer to the definition.
content.tex \subsubsection{Control Virtqueue}\label{sec:Device Types / Network Device / Devi · line 4608
Quoted 11 lines
content.tex +7
-0+For VIRTIO_NET_CTRL_NOTF_COAL_LOW_TX_SET and VIRTIO_NET_CTRL_NOTF_COAL_LOW_RX_SET, struct virtio_net_ctrl_coal_threshold specifies the coalescing parameters to use when the packet rate is \field{pkt_rate} or lower. + +For VIRTIO_NET_CTRL_NOTF_COAL_HIGH_TX_SET and VIRTIO_NET_CTRL_NOTF_COAL_HIGH_RX_SET, struct virtio_net_ctrl_coal_threshold specifies the coalescing parameters to use when the packet rate is \field{pkt_rate} or higher. + \drivernormative{\subparagraph}{Notifications Coalescing}{Device Types / Network Device / Device Operation / Control Virtqueue / Notifications Coalescing} If the VIRTIO_NET_F_NOTF_COAL feature has not been negotiated, the driver MUST NOT issue VIRTIO_NET_CTRL_NOTF_COAL commands. +If the VIRTIO_NET_F_NOTF_COAL_LOW_HIGH feature has not been negotiated, the driver MUST NOT issue low/high rate coalescing commands. + +The driver SHOULD issue a low/high rate coalescing command with \field{pkt_rate} 0 in order to remove the low/high rate coalescing parameters.
and presumably this 0 is an exception to the "packet rate for high threshold must not be lower than low threshold"?
content.tex \subsubsection{Control Virtqueue}\label{sec:Device Types / Network Device / Devi · line 4617
Quoted 9 lines
content.tex +9
-0+ +The driver MUST NOT issue a VIRTIO_NET_CTRL_NOTF_COAL_LOW_TX_SET command with \field{pkt_rate} equal or higher than a \field{pkt_rate} previously sent with a VIRTIO_NET_CTRL_NOTF_COAL_HIGH_TX_SET command. + +The driver MUST NOT issue a VIRTIO_NET_CTRL_NOTF_COAL_LOW_RX_SET command with \field{pkt_rate} equal or higher than a \field{pkt_rate} previously sent with a VIRTIO_NET_CTRL_NOTF_COAL_HIGH_RX_SET command. + +The driver MUST NOT issue a VIRTIO_NET_CTRL_NOTF_COAL_HIGH_TX_SET command with \field{pkt_rate} equal or lower than a \field{pkt_rate} previously sent with a VIRTIO_NET_CTRL_NOTF_COAL_LOW_TX_SET command. + +The driver MUST NOT issue a VIRTIO_NET_CTRL_NOTF_COAL_HIGH_RX_SET command with \field{pkt_rate} equal or lower than a \field{pkt_rate} previously sent with a VIRTIO_NET_CTRL_NOTF_COAL_LOW_RX_SET command. +
actually these rules are annoying.
if i want to lower packet rate for both high and low, and i happen to start with high, command may fail. possible to fix but really just introducing more code for now good reason.
what is the problem? if the rules make more than one set apply let's allow device whatever set it wants?
Quoted 25 lines
content.tex +12
-0\devicenormative{\subparagraph}{Notifications Coalescing}{Device Types / Network Device / Device Operation / Control Virtqueue / Notifications Coalescing} A device SHOULD respond to the VIRTIO_NET_CTRL_NOTF_COAL commands with VIRTIO_NET_ERR if it was not able to change the parameters. A device SHOULD NOT send used buffer notifications to the driver, if the notifications are suppressed as explained in \ref{sec:Basic Facilities of a Virtio Device / Virtqueues / Used Buffer Notification Suppression}, even if the coalescing counters expired. +A device MUST remove the low/high rate coalescing parameters, if a low/high rate coalescing command is received with \field{pkt_rate} 0. + +A device MUST respond with VIRTIO_NET_ERR to a VIRTIO_NET_CTRL_NOTF_COAL_LOW_TX_SET command with \field{pkt_rate} equal or higher than a \field{pkt_rate} previously received with a VIRTIO_NET_CTRL_NOTF_COAL_HIGH_TX_SET command. + +A device MUST respond with VIRTIO_NET_ERR to a VIRTIO_NET_CTRL_NOTF_COAL_LOW_RX_SET command with \field{pkt_rate} equal or higher than a \field{pkt_rate} previously received with a VIRTIO_NET_CTRL_NOTF_COAL_HIGH_RX_SET command. + +A device MUST respond with VIRTIO_NET_ERR to a VIRTIO_NET_CTRL_NOTF_COAL_HIGH_TX_SET command with \field{pkt_rate} equal or lower than a \field{pkt_rate} previously received with a VIRTIO_NET_CTRL_NOTF_COAL_LOW_TX_SET command. + +A device MUST respond with VIRTIO_NET_ERR to a VIRTIO_NET_CTRL_NOTF_COAL_HIGH_RX_SET command with \field{pkt_rate} equal or lower than a \field{pkt_rate} previously received with a VIRTIO_NET_CTRL_NOTF_COAL_LOW_RX_SET command. + Upon reset, a device MUST initialize all coalescing parameters to 0. +Upon reset, a device MUST not have a low/high rate coalescing parameters. + \subsubsection{Legacy Interface: Framing Requirements}\label{sec:Device Types / Network Device / Legacy Interface: Framing Requirements}Signature · 1 line
2.32.0
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Quoted 10 lines
Quoted 7 lines
#define
VIRTIO_NET_CTRL_NOTF_COAL6
#defineVIRTIO_NET_CTRL_NOTF_COAL_TX_SET0
#defineVIRTIO_NET_CTRL_NOTF_COAL_RX_SET1
+ #defineVIRTIO_NET_CTRL_NOTF_COAL_LOW_TX_SET2 //Only ifVIRTIO_NET_F_NOTF_COAL_LOW_HIGHnegotiated
+ #defineVIRTIO_NET_CTRL_NOTF_COAL_LOW_RX_SET3 //Only ifVIRTIO_NET_F_NOTF_COAL_LOW_HIGHnegotiated
+ #defineVIRTIO_NET_CTRL_NOTF_COAL_HIGH_TX_SET4 //Only ifVIRTIO_NET_F_NOTF_COAL_LOW_HIGHnegotiated
+ #defineVIRTIO_NET_CTRL_NOTF_COAL_HIGH_RX_SET5 //Only ifVIRTIO_NET_F_NOTF_COAL_LOW_HIGHnegotiatedadd conformance statements for this requirement of
VIRTIO_NET_F_NOTF_COAL_LOW_HIGHtoo?
I did:
Quoted 16 lines
+If the
VIRTIO_NET_F_NOTF_COAL_LOW_HIGHfeature has not been negotiated, the driver MUST NOT issue low/high rate coalescing commands.content.tex \subsubsection{Control Virtqueue}\label{sec:Device Types / Network Device / Devi · line 4531
Quoted 12 lines
content.tex +11
-1-Coalescing parameters: +TX Coalescing parameters: +\begin{itemize} +\item \field{usecs}: Maximum number of usecs to delay a TX notification. +\item \field{max_packets}: Maximum number of packets to send before a TX notification. +\end{itemize} + +RX Coalescing parameters: +\begin{itemize} +\item \field{usecs}: Maximum number of usecs to delay a RX notification. +\item \field{max_packets}: Maximum number of packets to receive before a RX notification. +\end{itemize}just unify these - TX/RX notification?
ok
Quoted 10 lines
Quoted 7 lines
+General Coalescing parameters:
\begin{itemize}
-\item \field{rx_usecs}: Maximum number of usecs to delay a RX notification.
-\item \field{tx_usecs}: Maximum number of usecs to delay a TX notification.
-\item \field{rx_max_packets}: Maximum number of packets to receive before a RX notification. -\item \field{tx_max_packets}: Maximum number of packets to send before a TX notification. +\item \field{pkt_rate}: Threshold for low/high packet rate coalescing parameters, in units of packets per second.This isn't really clear. Besides it seems that with 100GB/s we can thinkably begin to overflow a 32 bit value pretty soon.
I'll clarify the sentence.
You're right about the 100Gb/s links, I can think of a few ways to address this:
1) We could increase the type to u64.
2) We could change the pkt_rate units to packets per 100 usecs or some other arbitrary value.
3) The user could choose the packet rate units using new fields in virtio_net_config.
I think that the third option is the better choice here.
Quoted 9 lines
Also, how is this measured?
Do you expect device to average the rate over a whole second to decide? Ethernet speeds start at 1MBit/sec right? I think these days if we ask for # of packets in the last N usecs we won't go far off the field. How much is N so as not to incur too much latency? 100?Or are we measuring X latest packets and testing how long it takes to get them?
Why do we want this to be part of the spec?
I think that we should leave this to the device to decide.
Quoted 6 lines
Quoted 6 lines
content.tex +2
-3-The class VIRTIO_NET_CTRL_NOTF_COAL has 2 commands: +The class VIRTIO_NET_CTRL_NOTF_COAL has 6 commands: \begin{enumerate} -\item VIRTIO_NET_CTRL_NOTF_COAL_TX_SET: set the \field{tx_usecs} and \field{tx_max_packets} parameters. -\item VIRTIO_NET_CTRL_NOTF_COAL_RX_SET: set the \field{rx_usecs} and \field{rx_max_packets} parameters. +\item VIRTIO_NET_CTRL_NOTF_COAL_TX_SET: set the \field{usecs} and \field{max_packets} coalescing parameters for TX.
Quoted 8 lines
Quoted 5 lines
+\item
VIRTIO_NET_CTRL_NOTF_COAL_RX_SET: set the \field{usecs} and \field{max_packets} coalescing parameters for RX.
+\itemVIRTIO_NET_CTRL_NOTF_COAL_LOW_TX_SET: set the \field{usecs} and \field{max_packets} coalescing parameters for packet rate of \field{pkt_rate} or lower, for TX. +\itemVIRTIO_NET_CTRL_NOTF_COAL_LOW_RX_SET: set the \field{usecs} and \field{max_packets} coalescing parameters for packet rate of \field{pkt_rate} or lower, for RX. +\itemVIRTIO_NET_CTRL_NOTF_COAL_HIGH_TX_SET: set the \field{usecs} and \field{max_packets} coalescing parameters for packet rate of \field{pkt_rate} or higher, for TX. +\itemVIRTIO_NET_CTRL_NOTF_COAL_HIGH_RX_SET: set the \field{usecs} and \field{max_packets} coalescing parameters for packet rate of \field{pkt_rate} or higher, for RX.how is
I think that you missed a sentence here.
Quoted 9 lines
What is missing here is the theory of operation.
when are parameters set by VIRTIO_NET_CTRL_NOTF_COAL_RX_SET/VIRTIO_NET_CTRL_NOTF_COAL_TX_SET used?
Am I guessing right that with
VIRTIO_NET_F_NOTF_COAL_LOW_HIGHthere are 3 sets of parameters high medium and low, for rx and tx?
Yes, we have the low and high sets, and the normal one (VIRTIO_NET_CTRL_NOTF_COAL_RX_SET/VIRTIO_NET_CTRL_NOTF_COAL_TX_SET) that should be used when the packet rate is not higher than the high threshold (if any), and not lower than the low threshold (if any).
I'll explain it in the next version.
I removed the "set" concept after v1 following your suggestion. Maybe we should define the concept and use it in the spec after all?
Quoted 4 lines
Review
Quoted 2 lines
(quoted diff) +2
-0+For example, \field{max_packets} = 15 for TX. +drop empty line here pls.
ok
Quoted 5 lines
Quoted 5 lines
content.tex +5
-0+If the VIRTIO_NET_F_NOTF_COAL_LOW_HIGH feature is negotiated, the driver can send the following low/high rate coalescing commands to the device: + +\begin{itemize} +\item VIRTIO_NET_CTRL_NOTF_COAL_LOW_TX_SET. +\item VIRTIO_NET_CTRL_NOTF_COAL_LOW_RX_SET.
Quoted 8 lines
Quoted 4 lines
+\item
VIRTIO_NET_CTRL_NOTF_COAL_HIGH_TX_SET.
+\itemVIRTIO_NET_CTRL_NOTF_COAL_HIGH_RX_SET.
+\end{itemize}
+pls explain here how these commands all use
virtio_net_ctrl_coal_threshold.
and maybe rearrange so the explanation is closer to the definition.
This is explained in the following part:
-The class VIRTIO_NET_CTRL_NOTF_COAL has 2 commands:
+The class VIRTIO_NET_CTRL_NOTF_COAL has 6 commands:
\begin{enumerate}
-\item VIRTIO_NET_CTRL_NOTF_COAL_TX_SET: set the \field{tx_usecs} and \field{tx_max_packets} parameters.
-\item VIRTIO_NET_CTRL_NOTF_COAL_RX_SET: set the \field{rx_usecs} and \field{rx_max_packets} parameters.
+\item VIRTIO_NET_CTRL_NOTF_COAL_TX_SET: set the \field{usecs} and \field{max_packets} coalescing parameters for TX.
+\item VIRTIO_NET_CTRL_NOTF_COAL_RX_SET: set the \field{usecs} and \field{max_packets} coalescing parameters for RX.
+\item VIRTIO_NET_CTRL_NOTF_COAL_LOW_TX_SET: set the \field{usecs} and \field{max_packets} coalescing parameters for packet rate of \field{pkt_rate} or lower, for TX.
+\item VIRTIO_NET_CTRL_NOTF_COAL_LOW_RX_SET: set the \field{usecs} and \field{max_packets} coalescing parameters for packet rate of \field{pkt_rate} or lower, for RX.
+\item VIRTIO_NET_CTRL_NOTF_COAL_HIGH_TX_SET: set the \field{usecs} and \field{max_packets} coalescing parameters for packet rate of \field{pkt_rate} or higher, for TX.
+\item VIRTIO_NET_CTRL_NOTF_COAL_HIGH_RX_SET: set the \field{usecs} and \field{max_packets} coalescing parameters for packet rate of \field{pkt_rate} or higher, for RX.
I'll move it closer to the definition.
Quoted 6 lines
content.tex \subsubsection{Control Virtqueue}\label{sec:Device Types / Network Device / Devi · line 4608
Quoted 3 lines
content.tex +3
-0+If the VIRTIO_NET_F_NOTF_COAL_LOW_HIGH feature has not been negotiated, the driver MUST NOT issue low/high rate coalescing commands. + +The driver SHOULD issue a low/high rate coalescing command with \field{pkt_rate} 0 in order to remove the low/high rate coalescing parameters.and presumably this 0 is an exception to the "packet rate for high threshold must not be lower than low threshold"?
Yes, I'll add the exception.
Quoted 9 lines
content.tex \subsubsection{Control Virtqueue}\label{sec:Device Types / Network Device / Devi · line 4617
Quoted 8 lines
content.tex +8
-0+The driver MUST NOT issue a VIRTIO_NET_CTRL_NOTF_COAL_LOW_TX_SET command with \field{pkt_rate} equal or higher than a \field{pkt_rate} previously sent with a VIRTIO_NET_CTRL_NOTF_COAL_HIGH_TX_SET command. + +The driver MUST NOT issue a VIRTIO_NET_CTRL_NOTF_COAL_LOW_RX_SET command with \field{pkt_rate} equal or higher than a \field{pkt_rate} previously sent with a VIRTIO_NET_CTRL_NOTF_COAL_HIGH_RX_SET command. + +The driver MUST NOT issue a VIRTIO_NET_CTRL_NOTF_COAL_HIGH_TX_SET command with \field{pkt_rate} equal or lower than a \field{pkt_rate} previously sent with a VIRTIO_NET_CTRL_NOTF_COAL_LOW_TX_SET command. + +The driver MUST NOT issue a VIRTIO_NET_CTRL_NOTF_COAL_HIGH_RX_SET command with \field{pkt_rate} equal or lower than a \field{pkt_rate} previously sent with a VIRTIO_NET_CTRL_NOTF_COAL_LOW_RX_SET command. +
Quoted 7 lines
actually these rules are annoying.
if i want to lower packet rate for both high and low, and i happen to start with high, command may fail. possible to fix but really just introducing more code for now good reason.
what is the problem? if the rules make more than one set apply let's allow device whatever set it wants?
You're right, I'll drop these rules.
Thanks,
Alvaro
This publicly archived list offers a means to provide input to the OASIS Virtual I/O Device (VIRTIO) TC.
In order to verify user consent to the Feedback License terms and to minimize spam in the list archive, subscription is required before posting.
Subscribe: [email protected]
Unsubscribe: [email protected]
List help: [email protected]
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/
Hi Alvaro,
Quoted 6 lines
From: [email protected] <[email protected]> On Behalf Of Alvaro Karsz
This patch adds a new feature,
VIRTIO_NET_F_NOTF_COAL_LOW_HIGH, while clarifying/fixing existing coalescing concepts.
While its visible in the patch itself, what is being done,
its important to mention the problem statement/motivation for the feature at start of the commit log as first step.
So, can you please describe the motivation, what problem does the new command solve?
It is likely that control entity which want to set different coalescing parameters likely wants to more than just two values. This is something already supported today.
Quoted 16 lines
The new feature adds 4 new commands to
VIRTIO_NET_CTRL_NOTF_COALclass:
-VIRTIO_NET_CTRL_NOTF_COAL_LOW_TX_SET- coalescing
parameters
to use when the packet rate is equal or lower then the low threshold for TX.
-VIRTIO_NET_CTRL_NOTF_COAL_LOW_RX_SET- coalescing parameters to use when the packet rate is equal or lower then the low threshold for RX.
-VIRTIO_NET_CTRL_NOTF_COAL_HIGH_TX_SET- coalescing parameters to use when the packet rate is equal or higher then the high threshold for TX.
-VIRTIO_NET_CTRL_NOTF_COAL_HIGH_RX_SET- coalescing parameters to use when the packet rate is equal or higher then the high threshold for RX.
Quoted 7 lines
On top of the new feature, this patch:
- Unifies thevirtio_net_ctrl_coalstructs, since the one for tx and the one for rx are identical.
- Clarifies that the coalescing commands are best-effort.
- Specifies coalescing in respect to delivering interrupts when config changes.
They are best to be handled a pre-patch to this one.
Please split them so that spec improvement work can progress while new features are under review.
Quoted 4 lines
+\item[VIRTIO_NET_F_NOTF_COAL_LOW_HIGH(50)] Device supports notifications coalescing low rate and high rate sets. +
Even though historically, many feature bits exists, adding more feature bits is not scalable. It is also limits to enable such functionality at very early stage in the device configuration.
So, if at all this proceeds, we want the new admin q to discover and negotiate the new features. (Instead of features bits).
Quoted 23 lines
\item[VIRTIO_NET_F_NOTF_COAL(53)] Device supports notifications
coalescing.\item[VIRTIO_NET_F_GUEST_USO4 (54)] Driver can receive USOv4 packets.
(quoted diff) +0
-0@@ -3142,6 +3144,7 @@ \subsubsection{Feature bitrequirements}\label{sec:Device Types / Network Device \item[VIRTIO_NET_F_MQ] Requires
VIRTIO_NET_F_CTRL_VQ.
\item[VIRTIO_NET_F_CTRL_MAC_ADDR] RequiresVIRTIO_NET_F_CTRL_VQ. \item[VIRTIO_NET_F_NOTF_COAL] RequiresVIRTIO_NET_F_CTRL_VQ.
+\item[VIRTIO_NET_F_NOTF_COAL_LOW_HIGH] RequiresVIRTIO_NET_F_NOTF_COAL.
\item[VIRTIO_NET_F_RSC_EXT] RequiresVIRTIO_NET_F_HOST_TSO4orVIRTIO_NET_F_HOST_TSO6.
\item[VIRTIO_NET_F_RSS] RequiresVIRTIO_NET_F_CTRL_VQ. \end{description}(quoted diff) +0
-0@@ -4493,43 +4496,62 @@ \subsubsection{ControlVirtqueue}\label{sec:Device Types / Network Device / Devi If the
VIRTIO_NET_F_NOTF_COALfeature is negotiated, the driver can send control commands for dynamically changing the coalescing parameters.+Note: In general, these commands are best-effort: A device could send a notification even if it is not supposed to.
content.tex \subsubsection{Control Virtqueue}\label{sec:Device Types / Network Device / Devi · line 4505
Quoted 9 lines
content.tex +4
-3+ \begin{lstlisting} -struct virtio_net_ctrl_coal_rx { - le32 rx_max_packets; - le32 rx_usecs; +struct virtio_net_ctrl_coal { + le32 max_packets; + le32 usecs; };
Please put this change in a pre-patch along with additional rework of below.
Quoted 41 lines
content.tex +15
-4-struct virtio_net_ctrl_coal_tx { - le32 tx_max_packets; - le32 tx_usecs; +struct virtio_net_ctrl_coal_threshold { + le32 pkt_rate; + struct virtio_net_ctrl_coal params; }; #define VIRTIO_NET_CTRL_NOTF_COAL 6 #define VIRTIO_NET_CTRL_NOTF_COAL_TX_SET 0 #define VIRTIO_NET_CTRL_NOTF_COAL_RX_SET 1 + #define VIRTIO_NET_CTRL_NOTF_COAL_LOW_TX_SET 2 //Only if + VIRTIO_NET_F_NOTF_COAL_LOW_HIGH negotiated #define + VIRTIO_NET_CTRL_NOTF_COAL_LOW_RX_SET 3 //Only if + VIRTIO_NET_F_NOTF_COAL_LOW_HIGH negotiated #define + VIRTIO_NET_CTRL_NOTF_COAL_HIGH_TX_SET 4 //Only if + VIRTIO_NET_F_NOTF_COAL_LOW_HIGH negotiated #define + VIRTIO_NET_CTRL_NOTF_COAL_HIGH_RX_SET 5 //Only if + VIRTIO_NET_F_NOTF_COAL_LOW_HIGH negotiated \end{lstlisting} -Coalescing parameters: +TX Coalescing parameters: +\begin{itemize} +\item \field{usecs}: Maximum number of usecs to delay a TX notification. +\item \field{max_packets}: Maximum number of packets to send before a TXnotification.
+\end{itemize}content.tex +5
-0+ +RX Coalescing parameters: +\begin{itemize} +\item \field{usecs}: Maximum number of usecs to delay a RX notification. +\item \field{max_packets}: Maximum number of packets to receive before aRX notification.
+\end{itemize}content.tex +2
-3+ +General Coalescing parameters: \begin{itemize} -\item \field{rx_usecs}: Maximum number of usecs to delay a RX notification. -\item \field{tx_usecs}: Maximum number of usecs to delay a TX notification. -\item \field{rx_max_packets}: Maximum number of packets to receive before
Quoted 7 lines
a RX notification.
-\item \field{tx_max_packets}: Maximum number of packets to send before a TX notification.
+\item \field{pkt_rate}: Threshold for low/high packet rate coalescing parameters, in units of packets per second.
\end{itemize}
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Quoted 8 lines · Parav Pandit
Hi Alvaro,
Quoted 6 lines
From: [email protected] <[email protected]> On Behalf Of Alvaro Karsz
This patch adds a new feature,
VIRTIO_NET_F_NOTF_COAL_LOW_HIGH, while clarifying/fixing existing coalescing concepts.
Quoted 6 lines
While its visible in the patch itself, what is being done,
its important to mention the problem statement/motivation for the feature at start of the commit log as first step.So, can you please describe the motivation, what problem does the new command solve?
It is likely that control entity which want to set different coalescing parameters likely wants to more than just two values. This is something already supported today.
Interesting. Parav can you give some examples?
Quoted 23 lines
Quoted 16 lines
The new feature adds 4 new commands to
VIRTIO_NET_CTRL_NOTF_COALclass:
-VIRTIO_NET_CTRL_NOTF_COAL_LOW_TX_SET- coalescing
parameters
to use when the packet rate is equal or lower then the low threshold for TX.
-VIRTIO_NET_CTRL_NOTF_COAL_LOW_RX_SET- coalescing parameters to use when the packet rate is equal or lower then the low threshold for RX.
-VIRTIO_NET_CTRL_NOTF_COAL_HIGH_TX_SET- coalescing parameters to use when the packet rate is equal or higher then the high threshold for TX.
-VIRTIO_NET_CTRL_NOTF_COAL_HIGH_RX_SET- coalescing parameters to use when the packet rate is equal or higher then the high threshold for RX.Quoted 7 lines
On top of the new feature, this patch:
- Unifies thevirtio_net_ctrl_coalstructs, since the one for tx and the one for rx are identical.
- Clarifies that the coalescing commands are best-effort.
- Specifies coalescing in respect to delivering interrupts when config changes.
Quoted 2 lines
They are best to be handled a pre-patch to this one.
Please split them so that spec improvement work can progress while new features are under review.
Yea it's not a bad idea. I figured it out but
Quoted 5 lines
Quoted 4 lines
+\item[VIRTIO_NET_F_NOTF_COAL_LOW_HIGH(50)] Device supports notifications coalescing low rate and high rate sets. +
Even though historically, many feature bits exists, adding more feature bits is not scalable.
I disagree here. We have unlimited feature bits and I see no problem with their scalability. Devices (such as SIOV as opposed to SRIOV) which have very limited memory need to use the proposed vq transport that intel's working on, pci transport is wasting too much memory anyway. I hope we are not going to repeat this discussion for each and every new feature :)
Quoted 1 line
It is also limits to enable such functionality at very early stage in the device configuration.
Not really, it just handles compatibility and enables the commands.
Quoted 2 lines
So, if at all this proceeds, we want the new admin q to discover and negotiate the new features. (Instead of features bits).
Sounds like a new transport. I wouldn't make it gate this proposal.
Quoted 75 lines
Quoted 23 lines
\item[VIRTIO_NET_F_NOTF_COAL(53)] Device supports notifications
coalescing.\item[VIRTIO_NET_F_GUEST_USO4 (54)] Driver can receive USOv4 packets.
(quoted diff) +0
-0@@ -3142,6 +3144,7 @@ \subsubsection{Feature bitrequirements}\label{sec:Device Types / Network Device \item[VIRTIO_NET_F_MQ] Requires
VIRTIO_NET_F_CTRL_VQ.
\item[VIRTIO_NET_F_CTRL_MAC_ADDR] RequiresVIRTIO_NET_F_CTRL_VQ. \item[VIRTIO_NET_F_NOTF_COAL] RequiresVIRTIO_NET_F_CTRL_VQ.
+\item[VIRTIO_NET_F_NOTF_COAL_LOW_HIGH] RequiresVIRTIO_NET_F_NOTF_COAL.
\item[VIRTIO_NET_F_RSC_EXT] RequiresVIRTIO_NET_F_HOST_TSO4orVIRTIO_NET_F_HOST_TSO6.
\item[VIRTIO_NET_F_RSS] RequiresVIRTIO_NET_F_CTRL_VQ. \end{description}(quoted diff) +0
-0@@ -4493,43 +4496,62 @@ \subsubsection{ControlVirtqueue}\label{sec:Device Types / Network Device / Devi If the
VIRTIO_NET_F_NOTF_COALfeature is negotiated, the driver can send control commands for dynamically changing the coalescing parameters.+Note: In general, these commands are best-effort: A device could send a notification even if it is not supposed to.
content.tex \subsubsection{Control Virtqueue}\label{sec:Device Types / Network Device / Devi · line 4505
Quoted 9 lines
content.tex +4
-3+ \begin{lstlisting} -struct virtio_net_ctrl_coal_rx { - le32 rx_max_packets; - le32 rx_usecs; +struct virtio_net_ctrl_coal { + le32 max_packets; + le32 usecs; };Please put this change in a pre-patch along with additional rework of below.
Quoted 40 lines
content.tex +15
-4-struct virtio_net_ctrl_coal_tx { - le32 tx_max_packets; - le32 tx_usecs; +struct virtio_net_ctrl_coal_threshold { + le32 pkt_rate; + struct virtio_net_ctrl_coal params; }; #define VIRTIO_NET_CTRL_NOTF_COAL 6 #define VIRTIO_NET_CTRL_NOTF_COAL_TX_SET 0 #define VIRTIO_NET_CTRL_NOTF_COAL_RX_SET 1 + #define VIRTIO_NET_CTRL_NOTF_COAL_LOW_TX_SET 2 //Only if + VIRTIO_NET_F_NOTF_COAL_LOW_HIGH negotiated #define + VIRTIO_NET_CTRL_NOTF_COAL_LOW_RX_SET 3 //Only if + VIRTIO_NET_F_NOTF_COAL_LOW_HIGH negotiated #define + VIRTIO_NET_CTRL_NOTF_COAL_HIGH_TX_SET 4 //Only if + VIRTIO_NET_F_NOTF_COAL_LOW_HIGH negotiated #define + VIRTIO_NET_CTRL_NOTF_COAL_HIGH_RX_SET 5 //Only if + VIRTIO_NET_F_NOTF_COAL_LOW_HIGH negotiated \end{lstlisting} -Coalescing parameters: +TX Coalescing parameters: +\begin{itemize} +\item \field{usecs}: Maximum number of usecs to delay a TX notification. +\item \field{max_packets}: Maximum number of packets to send before a TXnotification.
+\end{itemize}content.tex +5
-0+ +RX Coalescing parameters: +\begin{itemize} +\item \field{usecs}: Maximum number of usecs to delay a RX notification. +\item \field{max_packets}: Maximum number of packets to receive before aRX notification.
+\end{itemize}content.tex +2
-2+ +General Coalescing parameters: \begin{itemize} -\item \field{rx_usecs}: Maximum number of usecs to delay a RX notification. -\item \field{tx_usecs}: Maximum number of usecs to delay a TX notification.
Quoted 8 lines
Quoted 8 lines
-\item \field{rx_max_packets}: Maximum number of packets to receive before a RX notification.
-\item \field{tx_max_packets}: Maximum number of packets to send before a TX notification.
+\item \field{pkt_rate}: Threshold for low/high packet rate coalescing parameters, in units of packets per second.
\end{itemize}
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Quoted 17 lines
From: Michael S. Tsirkin <[email protected]>
Sent: Monday, January 16, 2023 3:34 PMQuoted 10 lines · Parav Pandit
Hi Alvaro,
Quoted 6 lines
From: [email protected] <[email protected]> On Behalf Of Alvaro Karsz
This patch adds a new feature,
VIRTIO_NET_F_NOTF_COAL_LOW_HIGH, while clarifying/fixing existing coalescing concepts.While its visible in the patch itself, what is being done, its important to mention the problem statement/motivation for the feature at
start of the commit log as first step.
Quoted 1 line
Quoted 7 lines
Quoted 1 line
So, can you please describe the motivation, what problem does the new
command solve?
Quoted 1 line
It is likely that control entity which want to set different coalescing
parameters likely wants to more than just two values.
Quoted 1 line
This is something already supported today.
Interesting. Parav can you give some examples?
In a Linux OS based system, following drivers [1] uses the interrupt coalescing parameters by using net dim [2].
Net dim is the control entity which enables to use more than 2 moderation parameters based on the workload instead of two low and high static values.
[1] list of drivers
a. drivers/net/ethernet/amazon/ena/ena_netdev.c b. drivers/net/ethernet/broadcom/bcmsysport.c c. drivers/net/ethernet/broadcom/bnxt/bnxt.c d. drivers/net/ethernet/broadcom/genet/bcmgenet.c e. drivers/net/ethernet/hisilicon/hns3/hns3_enet.c f. drivers/net/ethernet/intel/ice/ice_txrx.c g. drivers/net/ethernet/mellanox/mlx5/core/en_txrx.c and more...
[2] https://docs.kernel.org/networking/net_dim.html
Quoted 10 lines
Quoted 2 lines
They are best to be handled a pre-patch to this one.
Please split them so that spec improvement work can progress while newfeatures are under review.
Yea it's not a bad idea. I figured it out but
Quoted 4 lines
Quoted 4 lines
+\item[VIRTIO_NET_F_NOTF_COAL_LOW_HIGH(50)] Device supports notifications coalescing low rate and high rate sets. +
Quoted 9 lines
Quoted 1 line
Even though historically, many feature bits exists, adding more feature bits is
not scalable.
I disagree here. We have unlimited feature bits and I see no problem with their scalability. Devices (such as SIOV as opposed to SRIOV) which have very limited memory need to use the proposed vq transport that intel's working on, pci transport is wasting too much memory anyway. I hope we are not going to repeat this discussion for each and every new feature :)
Pci transport of the virtio in current form asks to put things in device memory. And it doesn't need to continue this way regardless of SIOV.
It doesn't have any relation to SR-IOV either.
There are even PF devices too.
And putting things in feature bit limits the way these devices are composed. So lets discuss to use control vq for get and set both.
Feature bit doesn't even cut what is needed. Software doesn't even know what the valid range is. It needs to be discovered through cvq.
Quoted 5 lines
Quoted 1 line
It is also limits to enable such functionality at very early stage in the device
configuration.
Not really, it just handles compatibility and enables the commands.
Command is nothing but a functionality.
Quoted 6 lines
Quoted 1 line
So, if at all this proceeds, we want the new admin q to discover and negotiate
the new features.
Quoted 1 line
(Instead of features bits).
Sounds like a new transport. I wouldn't make it gate this proposal.
I am not gating the proposal.
Ctrl vq only does set.
Control VQ needs the ability to do get also on deciding what to control. Hence, new things like this can be discovered by net specific ctrol vq. No need for involving generic device feature bits framework here.
This publicly archived list offers a means to provide input to the OASIS Virtual I/O Device (VIRTIO) TC.
In order to verify user consent to the Feedback License terms and to minimize spam in the list archive, subscription is required before posting.
Subscribe: [email protected]
Unsubscribe: [email protected]
List help: [email protected]
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/
Quoted 39 lines · Parav Pandit
Quoted 17 lines
From: Michael S. Tsirkin <[email protected]>
Sent: Monday, January 16, 2023 3:34 PMQuoted 10 lines · Parav Pandit
Hi Alvaro,
Quoted 6 lines
From: [email protected] <[email protected]> On Behalf Of Alvaro Karsz
This patch adds a new feature,
VIRTIO_NET_F_NOTF_COAL_LOW_HIGH, while clarifying/fixing existing coalescing concepts.While its visible in the patch itself, what is being done, its important to mention the problem statement/motivation for the feature at
start of the commit log as first step.
Quoted 1 line
Quoted 7 lines
Quoted 1 line
So, can you please describe the motivation, what problem does the new
command solve?
Quoted 1 line
It is likely that control entity which want to set different coalescing
parameters likely wants to more than just two values.
Quoted 1 line
This is something already supported today.
Interesting. Parav can you give some examples?
In a Linux OS based system, following drivers [1] uses the interrupt coalescing parameters by using net dim [2].
Net dim is the control entity which enables to use more than 2 moderation parameters based on the workload instead of two low and high static values.
[1] list of drivers
a.drivers/net/ethernet/amazon/ena/ena_netdev.cb.drivers/net/ethernet/broadcom/bcmsysport.cc.drivers/net/ethernet/broadcom/bnxt/bnxt.cd.drivers/net/ethernet/broadcom/genet/bcmgenet.ce.drivers/net/ethernet/hisilicon/hns3/hns3_enet.cf.drivers/net/ethernet/intel/ice/ice_txrx.cg.drivers/net/ethernet/mellanox/mlx5/core/en_txrx.cand more...
Quoted 1 line
But this is net-dim. IIUC it's a software mechanism which tweaks coalescing parameters depending on traffic, right?
Consider for example bnxt_dim_work: all this does is set a single set of usecs/packets parameters. IOW it's need already seem to be addressed in the spec.
By comparison this patch is an attempt to offload ethtool's --coalesce parameters to the card. IMO what it misses is completeness, e.g. sample-interval is not specified. Also, introspection is missing and it's useful to avoid keeping all state in the driver.
Now clearly net dim has potential to make more clever decisions, but OTOH it's clearly heavier weight.
So I personally see potential for both to be useful. If no why not?
Quoted 23 lines
Quoted 10 lines
Quoted 2 lines
They are best to be handled a pre-patch to this one.
Please split them so that spec improvement work can progress while newfeatures are under review.
Yea it's not a bad idea. I figured it out but
Quoted 4 lines
Quoted 4 lines
+\item[VIRTIO_NET_F_NOTF_COAL_LOW_HIGH(50)] Device supports notifications coalescing low rate and high rate sets. +
Quoted 9 lines
Quoted 1 line
Even though historically, many feature bits exists, adding more feature bits is
not scalable.
I disagree here. We have unlimited feature bits and I see no problem with their scalability. Devices (such as SIOV as opposed to SRIOV) which have very limited memory need to use the proposed vq transport that intel's working on, pci transport is wasting too much memory anyway. I hope we are not going to repeat this discussion for each and every new feature :)
Pci transport of the virtio in current form asks to put things in device memory. And it doesn't need to continue this way regardless of SIOV.
It doesn't have any relation to SR-IOV either.
Quoted 7 lines
There are even PF devices too.
And putting things in feature bit limits the way these devices are composed. So lets discuss to use control vq for get and set both.
Feature bit doesn't even cut what is needed. Software doesn't even know what the valid range is. It needs to be discovered through cvq.
All well and good but let's not have this discussion in the thread please.
Quoted 6 lines
Quoted 5 lines
Quoted 1 line
It is also limits to enable such functionality at very early stage in the device
configuration.
Not really, it just handles compatibility and enables the commands.
Command is nothing but a functionality.
No, during init time it is useful to know whether to hook up to OS mechanisms dealing with specific offloads.
Quoted 7 lines
Quoted 6 lines
Quoted 1 line
So, if at all this proceeds, we want the new admin q to discover and negotiate
the new features.
Quoted 1 line
(Instead of features bits).
Sounds like a new transport. I wouldn't make it gate this proposal.
I am not gating the proposal.
what you are however doing is bringing issues unrealted to the patch in question. Let's focus on discussing coalescing here please.
Quoted 4 lines
Ctrl vq only does set.
Control VQ needs the ability to do get also on deciding what to control. Hence, new things like this can be discovered by net specific ctrol vq. No need for involving generic device feature bits framework here.
I see even less value in moving compatibility machinery over to a device specific mechanism such as cvq.
Signature · 6 lines
MST --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Quoted 30 lines
From: Michael S. Tsirkin <[email protected]>
Sent: Monday, January 16, 2023 4:30 PMQuoted 11 lines
Quoted 9 lines
Quoted 2 lines
So, can you please describe the motivation, what problem does the new
command solve?
Quoted 2 lines
It is likely that control entity which want to set different coalescing
parameters likely wants to more than just two values.
Quoted 1 line
This is something already supported today.
Interesting. Parav can you give some examples?
In a Linux OS based system, following drivers [1] uses the interrupt coalescing
parameters by using net dim [2].
Quoted 2 lines
Net dim is the control entity which enables to use more than 2 moderation
parameters based on the workload instead of two low and high static values.
Quoted 11 lines
[1] list of drivers
a.drivers/net/ethernet/amazon/ena/ena_netdev.cb.drivers/net/ethernet/broadcom/bcmsysport.cc.drivers/net/ethernet/broadcom/bnxt/bnxt.cd.drivers/net/ethernet/broadcom/genet/bcmgenet.ce.drivers/net/ethernet/hisilicon/hns3/hns3_enet.cf.drivers/net/ethernet/intel/ice/ice_txrx.cg.drivers/net/ethernet/mellanox/mlx5/core/en_txrx.cand more...
Quoted 5 lines
Quoted 1 line
But this is net-dim. IIUC it's a software mechanism which tweaks coalescing parameters depending on traffic, right?
It is more than tweak.
It adjusts the parameters to avoid system oscillate and constant manual tuning.
Quoted 7 lines
Consider for example
bnxt_dim_work: all this does is set a single set of usecs/packets parameters.
IOW it's need already seem to be addressed in the spec.By comparison this patch is an attempt to offload ethtool's --coalesce parameters to the card. IMO what it misses is completeness, e.g. sample- interval is not specified.
Yes. fallback rate above low and below high are also missing.
Quoted 6 lines
Also, introspection is missing and it's useful to avoid keeping all state in the driver.
Now clearly net dim has potential to make more clever decisions, but OTOH it's clearly heavier weight.
Its heavy weight that already exists in one OS.
OTOH constantly invoking ethtool from user space for a admin/sw is heavy weight too.
Quoted 3 lines
So I personally see potential for both to be useful. If no why not?
Lets describe the problem, value and its usefulness to begin.
I am not against it. I just fail to find its good use with the existence of net dim. So lets explain how it is better.
Quoted 31 lines
Quoted 31 lines
Quoted 13 lines
Quoted 3 lines
They are best to be handled a pre-patch to this one. Please split them so that spec improvement work can progress while new
features are under review.
Yea it's not a bad idea. I figured it out but
Quoted 6 lines
Quoted 4 lines
+\item[VIRTIO_NET_F_NOTF_COAL_LOW_HIGH(50)] Device supports notifications coalescing low rate and high rate sets. +
Even though historically, many feature bits exists, adding more feature bits is
Quoted 9 lines
not scalable.
I disagree here. We have unlimited feature bits and I see no problem with their scalability. Devices (such as SIOV as opposed to SRIOV) which have very limited memory need to use the proposed vq transport that intel's working on, pci transport is wasting too much memory anyway. I hope we are not going to repeat this discussion for each and every new feature :)
Pci transport of the virtio in current form asks to put things in device memory. And it doesn't need to continue this way regardless of SIOV.
It doesn't have any relation to SR-IOV either.There are even PF devices too.
And putting things in feature bit limits the way these devices are composed. So lets discuss to use control vq for get and set both.
Quoted 7 lines
Quoted 1 line
Feature bit doesn't even cut what is needed. Software doesn't even know
what the valid range is.
Quoted 1 line
It needs to be discovered through cvq.
All well and good but let's not have this discussion in the thread please.
Ok. This feature bit trying to do multiple features under single bit and all commands are not always useful. It needs two feature bits.
And things start not to scale.
But fine.
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
It is also limits to enable such functionality at very early stage in the device
Quoted 9 lines
Quoted 5 lines
Quoted 4 lines
configuration.
Not really, it just handles compatibility and enables the commands.
Command is nothing but a functionality.
No, during init time it is useful to know whether to hook up to OS mechanisms dealing with specific offloads.
It doesn't make sense to register a net device and set its feature bits without really enumerating it fully. And if that requires CVQ, it is more efficient anyway.
Quoted 12 lines
Quoted 8 lines
Quoted 7 lines
Quoted 2 lines
So, if at all this proceeds, we want the new admin q to discover and negotiate
the new features.
Quoted 1 line
(Instead of features bits).
Sounds like a new transport. I wouldn't make it gate this proposal.
I am not gating the proposal.
what you are however doing is bringing issues unrealted to the patch in question. Let's focus on discussing coalescing here please.
It is proposing and even modifying the existing feature bit without describing how better it is. If one feature requires to extend cvq, it is not a problem. We can surely extend it.
Quoted 8 lines
Quoted 5 lines
Ctrl vq only does set.
Control VQ needs the ability to do get also on deciding what to control. Hence, new things like this can be discovered by net specific ctrol vq. No need for involving generic device feature bits framework here.I see even less value in moving compatibility machinery over to a device specific mechanism such as cvq.
There is lot more to discuss, but I am avoid here in this thread.
we need to first establish a generic mechanism for discovering net specific features and its value (more than Boolean) in generic cvq way. This pave the way forward for more complex things to come in including this.
This publicly archived list offers a means to provide input to the OASIS Virtual I/O Device (VIRTIO) TC.
In order to verify user consent to the Feedback License terms and to minimize spam in the list archive, subscription is required before posting.
Subscribe: [email protected]
Unsubscribe: [email protected]
List help: [email protected]
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/
Quoted 38 lines · Parav Pandit
Quoted 30 lines
From: Michael S. Tsirkin <[email protected]>
Sent: Monday, January 16, 2023 4:30 PMQuoted 11 lines
Quoted 9 lines
Quoted 2 lines
So, can you please describe the motivation, what problem does the new
command solve?
Quoted 2 lines
It is likely that control entity which want to set different coalescing
parameters likely wants to more than just two values.
Quoted 1 line
This is something already supported today.
Interesting. Parav can you give some examples?
In a Linux OS based system, following drivers [1] uses the interrupt coalescing
parameters by using net dim [2].
Quoted 2 lines
Net dim is the control entity which enables to use more than 2 moderation
parameters based on the workload instead of two low and high static values.
Quoted 11 lines
[1] list of drivers
a.drivers/net/ethernet/amazon/ena/ena_netdev.cb.drivers/net/ethernet/broadcom/bcmsysport.cc.drivers/net/ethernet/broadcom/bnxt/bnxt.cd.drivers/net/ethernet/broadcom/genet/bcmgenet.ce.drivers/net/ethernet/hisilicon/hns3/hns3_enet.cf.drivers/net/ethernet/intel/ice/ice_txrx.cg.drivers/net/ethernet/mellanox/mlx5/core/en_txrx.cand more...Quoted 5 lines
Quoted 1 line
But this is net-dim. IIUC it's a software mechanism which tweaks coalescing parameters depending on traffic, right?
It is more than tweak.
It adjusts the parameters to avoid system oscillate and constant manual tuning.
Quoted 8 lines
Quoted 7 lines
Consider for example
bnxt_dim_work: all this does is set a single set of usecs/packets parameters.
IOW it's need already seem to be addressed in the spec.By comparison this patch is an attempt to offload ethtool's --coalesce parameters to the card. IMO what it misses is completeness, e.g. sample- interval is not specified.
Yes. fallback rate above low and below high are also missing.
I think what author meant is just mirroring ethtool here.
Quoted 9 lines
Quoted 6 lines
Also, introspection is missing and it's useful to avoid keeping all state in the driver.
Now clearly net dim has potential to make more clever decisions, but OTOH it's clearly heavier weight.
Its heavy weight that already exists in one OS.
OTOH constantly invoking ethtool from user space for a admin/sw is heavy weight too.
Quoted 6 lines
Quoted 3 lines
So I personally see potential for both to be useful. If no why not?
Lets describe the problem, value and its usefulness to begin.
I am not against it. I just fail to find its good use with the existence of net dim. So lets explain how it is better.
If all you want is what ethtool provides the new interface will do exactly that completely on the card without involving scheduler. That seems pretty clear.
Quoted 36 lines
Quoted 36 lines
Quoted 26 lines
Quoted 13 lines
Quoted 3 lines
They are best to be handled a pre-patch to this one. Please split them so that spec improvement work can progress while new
features are under review.
Yea it's not a bad idea. I figured it out but
Quoted 6 lines
Quoted 4 lines
+\item[VIRTIO_NET_F_NOTF_COAL_LOW_HIGH(50)] Device supports notifications coalescing low rate and high rate sets. +
Even though historically, many feature bits exists, adding more feature bits is
Quoted 9 lines
not scalable.
I disagree here. We have unlimited feature bits and I see no problem with their scalability. Devices (such as SIOV as opposed to SRIOV) which have very limited memory need to use the proposed vq transport that intel's working on, pci transport is wasting too much memory anyway. I hope we are not going to repeat this discussion for each and every new feature :)
Pci transport of the virtio in current form asks to put things in device memory. And it doesn't need to continue this way regardless of SIOV.
It doesn't have any relation to SR-IOV either.Quoted 6 lines
There are even PF devices too.
And putting things in feature bit limits the way these devices are composed. So lets discuss to use control vq for get and set both.
Feature bit doesn't even cut what is needed. Software doesn't even know
what the valid range is.
Quoted 1 line
It needs to be discovered through cvq.
Quoted 6 lines
Quoted 2 lines
All well and good but let's not have this discussion in the thread please.
Ok. This feature bit trying to do multiple features under single bit and all commands are not always useful. It needs two feature bits.
And things start not to scale.
But fine.
Which bits? For high and low? Maybe. It's really a single feature just for 2 thresholds. Which looks like a trivial generalization of one. But I wonder whether any devices have more than 2 thresholds, or just one. Do we want N thresholds? How about taking a look at some other OS-es? I never looked at net dim internal logic to see whether it can reasonably be offloaded to a card.
All these are valid concerns that need research to address. Any input here? Feature bits, use of cvq are all cosmetic issues by comparison.
Quoted 37 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
It is also limits to enable such functionality at very early stage in the device
Quoted 9 lines
Quoted 5 lines
Quoted 4 lines
configuration.
Not really, it just handles compatibility and enables the commands.
Command is nothing but a functionality.
No, during init time it is useful to know whether to hook up to OS mechanisms dealing with specific offloads.
It doesn't make sense to register a net device and set its feature bits without really enumerating it fully. And if that requires CVQ, it is more efficient anyway.
Quoted 12 lines
Quoted 8 lines
Quoted 7 lines
Quoted 2 lines
So, if at all this proceeds, we want the new admin q to discover and negotiate
the new features.
Quoted 1 line
(Instead of features bits).
Sounds like a new transport. I wouldn't make it gate this proposal.
I am not gating the proposal.
what you are however doing is bringing issues unrealted to the patch in question. Let's focus on discussing coalescing here please.
It is proposing and even modifying the existing feature bit without describing how better it is. If one feature requires to extend cvq, it is not a problem. We can surely extend it.
Quoted 6 lines
Quoted 5 lines
Ctrl vq only does set.
Control VQ needs the ability to do get also on deciding what to control. Hence, new things like this can be discovered by net specific ctrol vq. No need for involving generic device feature bits framework here.
Quoted 6 lines
Quoted 2 lines
I see even less value in moving compatibility machinery over to a device specific mechanism such as cvq.
There is lot more to discuss, but I am avoid here in this thread.
we need to first establish a generic mechanism for discovering net specific features and its value (more than Boolean) in generic cvq way. This pave the way forward for more complex things to come in including this.
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Quoted 72 lines
From: Michael S. Tsirkin <[email protected]>
Sent: Monday, January 16, 2023 7:28 PMQuoted 14 lines · Parav Pandit
Quoted 14 lines
From: Michael S. Tsirkin <[email protected]>
Sent: Monday, January 16, 2023 4:30 PMQuoted 10 lines
Quoted 9 lines
Quoted 2 lines
So, can you please describe the motivation, what problem does the new
command solve?
Quoted 2 lines
It is likely that control entity which want to set different coalescing
parameters likely wants to more than just two values.
Quoted 1 line
This is something already supported today.
Interesting. Parav can you give some examples?
Quoted 7 lines
Quoted 7 lines
Quoted 2 lines
In a Linux OS based system, following drivers [1] uses the interrupt coalescing
parameters by using net dim [2].
Quoted 3 lines
Net dim is the control entity which enables to use more than 2 moderation
parameters based on the workload instead of two low and high static
values.
Quoted 11 lines
Quoted 11 lines
Quoted 11 lines
[1] list of drivers
a.drivers/net/ethernet/amazon/ena/ena_netdev.cb.drivers/net/ethernet/broadcom/bcmsysport.cc.drivers/net/ethernet/broadcom/bnxt/bnxt.cd.drivers/net/ethernet/broadcom/genet/bcmgenet.ce.drivers/net/ethernet/hisilicon/hns3/hns3_enet.cf.drivers/net/ethernet/intel/ice/ice_txrx.cg.drivers/net/ethernet/mellanox/mlx5/core/en_txrx.cand more...Quoted 7 lines
Quoted 5 lines
Quoted 1 line
But this is net-dim. IIUC it's a software mechanism which tweaks coalescing parameters depending on traffic, right?
It is more than tweak.
It adjusts the parameters to avoid system oscillate and constant manualtuning.
Quoted 9 lines
Quoted 8 lines
Consider for example
bnxt_dim_work: all this does is set a single set of usecs/packets parameters.
IOW it's need already seem to be addressed in the spec.By comparison this patch is an attempt to offload ethtool's --coalesce parameters to the card. IMO what it misses is completeness, e.g. sample- interval is not specified.
Yes. fallback rate above low and below high are also missing.
I think what author meant is just mirroring ethtool here.
Quoted 8 lines
Quoted 6 lines
Also, introspection is missing and it's useful to avoid keeping all state in the driver.
Now clearly net dim has potential to make more clever decisions, but OTOH it's clearly heavier weight.
Its heavy weight that already exists in one OS.
OTOH constantly invoking ethtool from user space for a admin/sw is heavyweight too.
Quoted 4 lines
Quoted 4 lines
So I personally see potential for both to be useful. If no why not?
Quoted 8 lines
Quoted 3 lines
Lets describe the problem, value and its usefulness to begin.
I am not against it. I just fail to find its good use with the existence of net dim. So lets explain how it is better.If all you want is what ethtool provides the new interface will do exactly that completely on the card without involving scheduler.
That seems pretty clear.
What is clear from the commit message.
The motivation was not clear to me, with the advent of net dim support. Proposed ethtool matching parameters exists for more than decade in ethtool.
Quoted 2 lines
Which bits? For high and low? Maybe. It's really a single feature just for 2 thresholds. Which looks like a trivial generalization of one.
They are not as general the existing one.
It has specific meaning to support two different thresholds that a device needs to monitor for.
Quoted 8 lines
But I wonder whether any devices have more than 2 thresholds, or just one. Do we want N thresholds? How about taking a look at some other OS-es? I never looked at net dim internal logic to see whether it can reasonably be offloaded to a card.
All these are valid concerns that need research to address. Any input here? Feature bits, use of cvq are all cosmetic issues by comparison.
The proposed ethtool counter part is implemented by only handful of devices. Almost all those vendors have superseded those devices.
And those modern devices which superseded are not supporting them. I am part of at least two device history as benet and mlx.
Both of them have not carried forward these parameters.
Compare to that net dim logic introduced in 2018 has found more than 10 active devices. And it even extended to non nic device too.
This statistic clearly shows the value of it.
A next step would be as you suggest offloading this as well to the device.
However, this proposal is not addressing to offload it. This patch relies on sw entity to configure some static values for two range. So, it may be addressing some specific use case or OS scenario.
I like to see this motivation described in the commit log of v3, which helps to dial in the spec faster.
All I am asking is plan motivation on how it helps, instead of saying hey it exists in ethtool a decade ago, lets do it...
This publicly archived list offers a means to provide input to the OASIS Virtual I/O Device (VIRTIO) TC.
In order to verify user consent to the Feedback License terms and to minimize spam in the list archive, subscription is required before posting.
Subscribe: [email protected]
Unsubscribe: [email protected]
List help: [email protected]
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/
Quoted 3 lines
By comparison this patch is an attempt to offload ethtool's --coalesce parameters to the card. IMO what it misses is completeness, e.g. sample-interval is not specified.
AFAIK sample-interval refers to the adaptive coalescing (adaptive-rx/adaptive-tx) and not to high/low.
include/uapi/linux/ethtool.h:
* @rate_sample_interval: How often to do adaptive coalescing packet rate
* sampling, measured in seconds. Must not be zero.
Quoted 2 lines
Also, introspection is missing and it's useful to avoid keeping all state in the driver.
I don't get it, can you please elaborate?
This publicly archived list offers a means to provide input to the OASIS Virtual I/O Device (VIRTIO) TC.
In order to verify user consent to the Feedback License terms and to minimize spam in the list archive, subscription is required before posting.
Subscribe: [email protected]
Unsubscribe: [email protected]
List help: [email protected]
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/
Quoted 10 lines · Alvaro Karsz
Quoted 3 lines
By comparison this patch is an attempt to offload ethtool's --coalesce parameters to the card. IMO what it misses is completeness, e.g. sample-interval is not specified.
AFAIK sample-interval refers to the adaptive coalescing (adaptive-rx/adaptive-tx) and not to high/low.
include/uapi/linux/ethtool.h:
* @rate_sample_interval: How often to do adaptive coalescing packet rate
* sampling, measured in seconds. Must not be zero.
Oh you are right. So sampling interval here is arbitrary, defined by the device. Worth documenting.
Quoted 4 lines
Quoted 2 lines
Also, introspection is missing and it's useful to avoid keeping all state in the driver.
I don't get it, can you please elaborate?
I mean --show-coalesce
Signature · 18 lines
MST This publicly archived list offers a means to provide input to the OASIS Virtual I/O Device (VIRTIO) TC. In order to verify user consent to the Feedback License terms and to minimize spam in the list archive, subscription is required before posting. Subscribe: [email protected] Unsubscribe: [email protected] List help: [email protected] List archive: https://lists.oasis-open.org/archives/virtio-comment/ Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists Committee: https://www.oasis-open.org/committees/virtio/ Join OASIS: https://www.oasis-open.org/join/
Hi,
I'm going to drop this patch and focus on a new patch to fix the existing notification coalescing feature, including:
- Unifying the virtio_net_ctrl_coal structs.
- Clarifying that the coalescing commands are best-effort.
- Specifying coalescing in respect to delivering interrupts when config changes.
Thanks everyone for the comments.
This publicly archived list offers a means to provide input to the OASIS Virtual I/O Device (VIRTIO) TC.
In order to verify user consent to the Feedback License terms and to minimize spam in the list archive, subscription is required before posting.
Subscribe: [email protected]
Unsubscribe: [email protected]
List help: [email protected]
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/
Hi,
Quoted 9 lines
Quoted 3 lines
This patch adds a new feature,
VIRTIO_NET_F_NOTF_COAL_LOW_HIGH, while clarifying/fixing existing coalescing concepts.While its visible in the patch itself, what is being done,
its important to mention the problem statement/motivation for the feature at start of the commit log as first step.So, can you please describe the motivation, what problem does the new command solve?
It is likely that control entity which want to set different coalescing parameters likely wants to more than just two values. This is something already supported today.
I'll add a better commit log in the next version.
Quoted 9 lines
Quoted 7 lines
On top of the new feature, this patch:
- Unifies thevirtio_net_ctrl_coalstructs, since the one for tx and the one for rx are identical.
- Clarifies that the coalescing commands are best-effort.
- Specifies coalescing in respect to delivering interrupts when config changes.They are best to be handled a pre-patch to this one.
Please split them so that spec improvement work can progress while new features are under review.
ok
This publicly archived list offers a means to provide input to the OASIS Virtual I/O Device (VIRTIO) TC.
In order to verify user consent to the Feedback License terms and to minimize spam in the list archive, subscription is required before posting.
Subscribe: [email protected]
Unsubscribe: [email protected]
List help: [email protected]
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/