Skip to contentSkip to conversation

virtio-net: clarify coalescing parameters settings

PATCH Superseded Heng Qi 2 participants 4 messages May 23, 2024

v3 of 5 Superseded Latest is v5 →

lore
Share & tools
lore

Heng Qi

author
#0 lore raw

PATCH v3 1/1 virtio-net: clarify coalescing parameters settings

The device can set any initial coalescing parameters (0 or non-zero) for the receive/send queue before the setting command is executed, not just 0, enhancing device performance even without DIM enabled.

So we need to clarify descriptions that don't fit the behavior.

---
v2->v3:
- Clarify description to be more generic. @Parav

v1->v2:
- Update description. @Jason

 device-types/net/description.tex | 20 +++++++++++++-------
 1 file changed, 13 insertions(+), 7 deletions(-)
device-types/net/description.tex +13 -7
index 61cce1f..f51e0d8 100644
@@ -1803,6 +1803,14 @@ \subsubsection{Control Virtqueue}\label{sec:Device Types / Network Device / Devi
                                         for an enabled transmit/receive virtqueue whose index is \field{vq_index}.
 \end{enumerate}
 
+If the VIRTIO_NET_F_NOTF_COAL or VIRTIO_NET_F_VQ_NOTF_COAL feature is negotiated,
+the device may apply any coalescing parameters to each transmit/receive virtqueue
+before the driver successfully performs one of the VIRTIO_NET_CTRL_NOTF_COAL set commands.
+
+The driver can query the current coalescing parameters of any enabled transmit/receive
+virtqueue using the VIRTIO_NET_CTRL_NOTF_COAL_VQ_GET command, before or after any
+VIRTIO_NET_CTRL_NOTF_COAL set command is done.
+
 The device may generate notifications more or less frequently than specified by set commands of the VIRTIO_NET_CTRL_NOTF_COAL class.
 
 If coalescing parameters are being set, the device applies the last coalescing parameters set for a
@@ -1884,18 +1892,16 @@ \subsubsection{Control Virtqueue}\label{sec:Device Types / Network Device / Devi
 The device MUST respond to VIRTIO_NET_CTRL_NOTF_COAL_VQ_SET and VIRTIO_NET_CTRL_NOTF_COAL_VQ_GET commands with
 VIRTIO_NET_ERR if the designated virtqueue is not an enabled transmit or receive virtqueue.
 
-Upon disabling and re-enabling a transmit virtqueue, the device MUST set the coalescing parameters of the virtqueue
-to those configured through the VIRTIO_NET_CTRL_NOTF_COAL_TX_SET command, or, if the driver did not set any TX coalescing parameters, to 0.
-
-Upon disabling and re-enabling a receive virtqueue, the device MUST set the coalescing parameters of the virtqueue
-to those configured through the VIRTIO_NET_CTRL_NOTF_COAL_RX_SET command, or, if the driver did not set any RX coalescing parameters, to 0.
-
 The behavior of the device in response to set commands of the VIRTIO_NET_CTRL_NOTF_COAL class is best-effort:
 the device MAY generate notifications more or less frequently than specified.
 
 A device SHOULD NOT send used buffer notifications to the driver if the notifications are suppressed, even if the notification conditions are met.
 
-Upon reset, a device MUST initialize all coalescing parameters to 0.
+Upon disabling and then re-enabling a transmit or receive virtqueue, the device
+MUST apply the same coalescing parameters to the virtqueue as were set before
+it was disabled.
+
+Upon reset, a device MAY set any coalescing parameters for all transmit or receive virtqueues.
 
 \paragraph{Device Statistics}\label{sec:Device Types / Network Device / Device Operation / Control Virtqueue / Device Statistics}
 
Signature · 1 line
2.32.0.3.g01195cf9f
#1 lore raw
Quoted 61 lines

From: Heng Qi <[email protected]>
Sent: Thursday, May 23, 2024 8:54 AM

The device can set any initial coalescing parameters (0 or non-zero) for the receive/send queue before the setting command is executed, not just 0, enhancing device performance even without DIM enabled.

So we need to clarify descriptions that don't fit the behavior.

---
v2->v3:
- Clarify description to be more generic. @Parav

v1->v2:
- Update description. @Jason

 device-types/net/description.tex | 20 +++++++++++++-------
 1 file changed, 13 insertions(+), 7 deletions(-)
device- renamed from device-types/net/description.tex +0 -0

types/net/description.tex
index 61cce1f..f51e0d8 100644

device-types/net/description.tex +0 -0
@@ -1803,6 +1803,14 @@ \subsubsection{Control

Virtqueue}\label{sec:Device Types / Network Device / Devi
for an enabled transmit/receive virtqueue whose index is
\field{vq_index}.
\end{enumerate}

+If the VIRTIO_NET_F_NOTF_COAL or VIRTIO_NET_F_VQ_NOTF_COAL feature is

device-types/net/description.tex +2 -0
+negotiated, the device may apply any coalescing parameters to each
+transmit/receive virtqueue before the driver successfully performs one of the

VIRTIO_NET_CTRL_NOTF_COAL set commands.

device-types/net/description.tex +3 -0
+
+The driver can query the current coalescing parameters of any enabled
+transmit/receive virtqueue using the

VIRTIO_NET_CTRL_NOTF_COAL_VQ_GET
+command, before or after any VIRTIO_NET_CTRL_NOTF_COAL set command is done.
+
The device may generate notifications more or less frequently than specified
by set commands of the VIRTIO_NET_CTRL_NOTF_COAL class.

If coalescing parameters are being set, the device applies the last coalescing
parameters set for a @@ -1884,18 +1892,16 @@ \subsubsection{Control Virtqueue}\label{sec:Device Types / Network Device / Devi The device MUST respond to VIRTIO_NET_CTRL_NOTF_COAL_VQ_SET and VIRTIO_NET_CTRL_NOTF_COAL_VQ_GET commands with VIRTIO_NET_ERR if the designated virtqueue is not an enabled transmit or receive virtqueue.

-Upon disabling and re-enabling a transmit virtqueue, the device MUST set the coalescing parameters of the virtqueue -to those configured through the VIRTIO_NET_CTRL_NOTF_COAL_TX_SET command, or, if the driver did not set any TX coalescing parameters, to 0.

device-types/net/description.tex +0 -2
-
-Upon disabling and re-enabling a receive virtqueue, the device MUST set the
Quoted 4 lines

coalescing parameters of the virtqueue -to those configured through the VIRTIO_NET_CTRL_NOTF_COAL_RX_SET command, or, if the driver did not set any RX coalescing parameters, to 0.
-

These two normative statements are correct, the change should be only for 0 and only for TX and RX commands. (no need to extend that to per VQ level commands).

There is no need to burden the device to remember the old value for each queue when the q is disabled. No point in remember something when the device does not know if it will be ever enabled or not.

So it should be,

"or, If the driver did not set any coalescing parameters, the device MAY initialize it to any value."

device-types/net/description.tex

Quoted 13 lines

The behavior of the device in response to set commands of the
VIRTIO_NET_CTRL_NOTF_COAL class is best-effort:
the device MAY generate notifications more or less frequently than specified.

A device SHOULD NOT send used buffer notifications to the driver if the
notifications are suppressed, even if the notification conditions are met.

device-types/net/description.tex +4 -1
-Upon reset, a device MUST initialize all coalescing parameters to 0.
 
+Upon disabling and then re-enabling a transmit or receive virtqueue,
+the device MUST apply the same coalescing parameters to the virtqueue
+as were set before it was disabled.
+

I think it should be,
When VIRTIO_NET_F_VQ_NOTF_COAL feature is negotiated, upon disabling and then re-enabling a transmit or receive virtqueue, the device MAY set any coalescing parameters for all transmit or receive virtqueues.

This clarifies the current behavior and also does not burden the device to remember any values related to VQ resource that is disabled (or destroyed).

Quoted 8 lines

+Upon reset, a device MAY set any coalescing parameters for all transmit or receive virtqueues.

\paragraph{Device Statistics}\label{sec:Device Types / Network Device /
Device Operation / Control Virtqueue / Device Statistics}

--
2.32.0.3.g01195cf9f

author
#2 lore raw
Quoted 69 lines · Parav Pandit
Quoted 61 lines

From: Heng Qi <[email protected]>
Sent: Thursday, May 23, 2024 8:54 AM

The device can set any initial coalescing parameters (0 or non-zero) for the receive/send queue before the setting command is executed, not just 0, enhancing device performance even without DIM enabled.

So we need to clarify descriptions that don't fit the behavior.

---
v2->v3:
- Clarify description to be more generic. @Parav

v1->v2:
- Update description. @Jason

 device-types/net/description.tex | 20 +++++++++++++-------
 1 file changed, 13 insertions(+), 7 deletions(-)
device- renamed from device-types/net/description.tex +0 -0

types/net/description.tex
index 61cce1f..f51e0d8 100644

device-types/net/description.tex +0 -0
@@ -1803,6 +1803,14 @@ \subsubsection{Control

Virtqueue}\label{sec:Device Types / Network Device / Devi
for an enabled transmit/receive virtqueue whose index is
\field{vq_index}.
\end{enumerate}

+If the VIRTIO_NET_F_NOTF_COAL or VIRTIO_NET_F_VQ_NOTF_COAL feature is

device-types/net/description.tex +2 -0
+negotiated, the device may apply any coalescing parameters to each
+transmit/receive virtqueue before the driver successfully performs one of the

VIRTIO_NET_CTRL_NOTF_COAL set commands.

device-types/net/description.tex +3 -0
+
+The driver can query the current coalescing parameters of any enabled
+transmit/receive virtqueue using the

VIRTIO_NET_CTRL_NOTF_COAL_VQ_GET
+command, before or after any VIRTIO_NET_CTRL_NOTF_COAL set command is done.
+
The device may generate notifications more or less frequently than specified
by set commands of the VIRTIO_NET_CTRL_NOTF_COAL class.

If coalescing parameters are being set, the device applies the last coalescing
parameters set for a @@ -1884,18 +1892,16 @@ \subsubsection{Control Virtqueue}\label{sec:Device Types / Network Device / Devi The device MUST respond to VIRTIO_NET_CTRL_NOTF_COAL_VQ_SET and VIRTIO_NET_CTRL_NOTF_COAL_VQ_GET commands with VIRTIO_NET_ERR if the designated virtqueue is not an enabled transmit or receive virtqueue.

-Upon disabling and re-enabling a transmit virtqueue, the device MUST set the coalescing parameters of the virtqueue -to those configured through the VIRTIO_NET_CTRL_NOTF_COAL_TX_SET command, or, if the driver did not set any TX coalescing parameters, to 0.

device-types/net/description.tex +0 -2
-
-Upon disabling and re-enabling a receive virtqueue, the device MUST set the
Quoted 4 lines

coalescing parameters of the virtqueue -to those configured through the VIRTIO_NET_CTRL_NOTF_COAL_RX_SET command, or, if the driver did not set any RX coalescing parameters, to 0.
-

These two normative statements are correct, the change should be only for 0 and only for TX and RX commands. (no need to extend that to per VQ level commands).

Quoted 2 lines

There is no need to burden the device to remember the old value for each queue when the q is disabled. No point in remember something when the device does not know if it will be ever enabled or not.

The current spec already tells the device to remember the old value.

But I think it's ok to be relaxed for the descriptions.

Quoted 4 lines

So it should be,

"or, If the driver did not set any coalescing parameters, the device MAY initialize it to any value."

It works.

Quoted 19 lines
Quoted 8 lines

The behavior of the device in response to set commands of the
VIRTIO_NET_CTRL_NOTF_COAL class is best-effort:
the device MAY generate notifications more or less frequently than specified.

A device SHOULD NOT send used buffer notifications to the driver if the
notifications are suppressed, even if the notification conditions are met.

device-types/net/description.tex

Quoted 6 lines
device-types/net/description.tex +4 -1
-Upon reset, a device MUST initialize all coalescing parameters to 0.
 
+Upon disabling and then re-enabling a transmit or receive virtqueue,
+the device MUST apply the same coalescing parameters to the virtqueue
+as were set before it was disabled.
+

I think it should be,
When VIRTIO_NET_F_VQ_NOTF_COAL feature is negotiated, upon disabling and then re-enabling a transmit or receive virtqueue, the device MAY set any coalescing parameters for all transmit or receive virtqueues.

Quoted 1 line

This clarifies the current behavior and also does not burden the device to remember any values related to VQ resource that is disabled (or destroyed).

Makes sense.

Thanks.

Quoted 10 lines
Quoted 9 lines

+Upon reset, a device MAY set any coalescing parameters for all transmit or receive virtqueues.

\paragraph{Device Statistics}\label{sec:Device Types / Network Device /
Device Operation / Control Virtqueue / Device Statistics}

--
2.32.0.3.g01195cf9f

#3 lore raw
Quoted 79 lines

From: Heng Qi <[email protected]>
Sent: Thursday, May 23, 2024 12:06 PM

Quoted 32 lines · Parav Pandit
Quoted 32 lines

From: Heng Qi <[email protected]>
Sent: Thursday, May 23, 2024 8:54 AM

The device can set any initial coalescing parameters (0 or non-zero) for the receive/send queue before the setting command is executed, not just 0, enhancing device performance even without DIM enabled.

So we need to clarify descriptions that don't fit the behavior.

---
v2->v3:
- Clarify description to be more generic. @Parav

v1->v2:
- Update description. @Jason

 device-types/net/description.tex | 20 +++++++++++++-------
 1 file changed, 13 insertions(+), 7 deletions(-)
device- renamed from device-types/net/description.tex +0 -0

types/net/description.tex index 61cce1f..f51e0d8 100644

device-types/net/description.tex +0 -0
@@ -1803,6 +1803,14 @@ \subsubsection{Control

Virtqueue}\label{sec:Device Types / Network Device / Devi
for an enabled
transmit/receive virtqueue whose index is \field{vq_index}.
\end{enumerate}

Quoted 1 line
Quoted 1 line

+If the VIRTIO_NET_F_NOTF_COAL or VIRTIO_NET_F_VQ_NOTF_COAL

feature

Quoted 10 lines
Quoted 10 lines

is

device-types/net/description.tex +3 -0
+negotiated, the device may apply any coalescing parameters to each
+transmit/receive virtqueue before the driver successfully performs
+one of the

VIRTIO_NET_CTRL_NOTF_COAL set commands.

device-types/net/description.tex +3 -0
+
+The driver can query the current coalescing parameters of any
+enabled transmit/receive virtqueue using the

VIRTIO_NET_CTRL_NOTF_COAL_VQ_GET
+command, before or after any VIRTIO_NET_CTRL_NOTF_COAL set

command

Quoted 11 lines
Quoted 11 lines

is done.
+
The device may generate notifications more or less frequently than
specified by set commands of the VIRTIO_NET_CTRL_NOTF_COAL class.

If coalescing parameters are being set, the device applies the last
coalescing parameters set for a @@ -1884,18 +1892,16 @@ \subsubsection{Control Virtqueue}\label{sec:Device Types / Network Device / Devi The device MUST respond to VIRTIO_NET_CTRL_NOTF_COAL_VQ_SET and VIRTIO_NET_CTRL_NOTF_COAL_VQ_GET commands with

VIRTIO_NET_ERR if the designated virtqueue is not an enabled transmit or receive virtqueue.

Quoted 4 lines
Quoted 4 lines

-Upon disabling and re-enabling a transmit virtqueue, the device MUST set the coalescing parameters of the virtqueue -to those configured through the VIRTIO_NET_CTRL_NOTF_COAL_TX_SET

command, or,

Quoted 9 lines
Quoted 7 lines

if the driver did not set any TX coalescing parameters, to 0.

device-types/net/description.tex +0 -2
-
-Upon disabling and re-enabling a receive virtqueue, the device MUST

set the coalescing parameters of the virtqueue -to those configured through the VIRTIO_NET_CTRL_NOTF_COAL_RX_SET command, or, if the driver did not set any RX coalescing parameters, to 0.
-

These two normative statements are correct, the change should be only for 0

and only for TX and RX commands.

Quoted 9 lines
Quoted 3 lines

(no need to extend that to per VQ level commands).

There is no need to burden the device to remember the old value for each

queue when the q is disabled.

Quoted 1 line

No point in remember something when the device does not know if it will be

ever enabled or not.

The current spec already tells the device to remember the old value.

It only tells for VIRTIO_NET_F_NOTF_COAL in above two normatives.

For VIRTIO_NET_CTRL_NOTF_COAL_VQ_SET it is not called out explicitly, hence the device do not remember it.

Quoted 1 line

But I think it's ok to be relaxed for the descriptions.

Yes.

Quoted 49 lines
Quoted 5 lines

So it should be,

"or, If the driver did not set any coalescing parameters, the device MAY

initialize it to any value."

It works.

Quoted 4 lines
Quoted 4 lines

The behavior of the device in response to set commands of the
VIRTIO_NET_CTRL_NOTF_COAL class is best-effort:
the device MAY generate notifications more or less frequently than

specified.

Quoted 3 lines
Quoted 3 lines

A device SHOULD NOT send used buffer notifications to the driver if
the notifications are suppressed, even if the notification conditions are

met.

Quoted 11 lines

device-types/net/description.tex

Quoted 7 lines
device-types/net/description.tex +4 -1
-Upon reset, a device MUST initialize all coalescing parameters to 0.
 
+Upon disabling and then re-enabling a transmit or receive
+virtqueue, the device MUST apply the same coalescing parameters to
+the virtqueue as were set before it was disabled.
+

I think it should be,
When VIRTIO_NET_F_VQ_NOTF_COAL feature is negotiated, upon disabling and then re-enabling a transmit or receive virtqueue, the device MAY set any

coalescing parameters for all transmit or receive virtqueues.

Quoted 2 lines

This clarifies the current behavior and also does not burden the device to

remember any values related to VQ resource that is disabled (or destroyed).

Makes sense.

Thanks.

Quoted 11 lines
Quoted 10 lines
device-types/net/description.tex +2 -0
+Upon reset, a device MAY set any coalescing parameters for all
+transmit or

receive virtqueues.

\paragraph{Device Statistics}\label{sec:Device Types / Network
Device / Device Operation / Control Virtqueue / Device Statistics}

--
2.32.0.3.g01195cf9f