virtio: introduce SUSPEND and RESUME feature
PATCH v2 1/1 virtio: introduce SUSPEND and RESUME feature
This commit allows the driver to suspend the device through a new device status bit SUSPEND and resume the device running by re-setting DRIVER_OK bit in device status.
This commit re-orders the device status bits.
--- content.tex | 74 ++++++++++++++++++++++++++++++++++++++++++++++------- 1 file changed, 65 insertions(+), 9 deletions(-)
content.tex +65 -9
@@ -36,19 +36,22 @@ \section{\field{Device Status} Field}\label{sec:Basic Facilities of a Virtio Dev this bit. For example, under Linux, drivers can be loadable modules. \end{note} -\item[FAILED (128)] Indicates that something went wrong in the guest, - and it has given up on the device. This could be an internal - error, or the driver didn't like the device for some reason, or - even a fatal error during device operation. +\item[DRIVER_OK (4)] Indicates that the driver is set up and ready to + drive the device. + +\item[SUSPEND (16)] When VIRTIO_F_SUSPEND is negotiated, indicates that the + device has been suspended by the driver. \item[FEATURES_OK (8)] Indicates that the driver has acknowledged all the features it understands, and feature negotiation is complete. -\item[DRIVER_OK (4)] Indicates that the driver is set up and ready to - drive the device. - \item[DEVICE_NEEDS_RESET (64)] Indicates that the device has experienced an error from which it can't recover. + +\item[FAILED (128)] Indicates that something went wrong in the guest, + and it has given up on the device. This could be an internal + error, or the driver didn't like the device for some reason, or + even a fatal error during device operation. \end{description} The \field{device status} field starts out as 0, and is reinitialized to 0 by
@@ -99,10 +102,10 @@ \section{Feature Bits}\label{sec:Basic Facilities of a Virtio Device / Feature B \begin{description} \item[0 to 23, and 50 to 127] Feature bits for the specific device type -\item[24 to 41] Feature bits reserved for extensions to the queue and +\item[24 to 43] Feature bits reserved for extensions to the queue and feature negotiation mechanisms, see \ref{sec:Reserved Feature Bits} -\item[42 to 49, and 128 and above] Feature bits reserved for future extensions. +\item[44 to 49, and 128 and above] Feature bits reserved for future extensions. \end{description} \begin{note}
@@ -629,6 +632,54 @@ \section{Device Cleanup}\label{sec:General Initialization And Device Operation / Thus a driver MUST ensure a virtqueue isn't live (by device reset) before removing exposed buffers. +\section{Device Suspend}\label{sec:General Initialization And Device Operation / Device Suspend} + +If VIRTIO_F_SUSPEND is negotiated, the driver is eligible to suspend the device by setting the SUSPEND bit in \field{device status} to 1, and the device SHOULD set the DRIVER_OK bit to 0 once it has been suspended. + +If the device has been suspended, the driver can resume the device running by setting the DRIVER_OK bit in \field{device status} to 1, and the device should set the SUSPEND bit to 0 once it resumes running. + +\drivernormative{\subsection}{Device Suspend}{General Initialization And Device Operation / Device Suspend} + +The driver SHOULD NOT set SUSPEND bit if DRIVER_OK is not set or VIRTIO_F_SUSPEND is not negotiated. + +Once the driver sets SUSPEND bit in \field{device status} to 1: +\begin{itemize} +\item The driver MUST verify whether the device has been suspended by re-reading \field{device status}, examining whether the SUSPEND bit is set to 1 and the DRIVER_OK bit is set to 0. +\item The driver MUST NOT make any more buffers available to the device. +\item The driver MUST NOT send notifications for any virtqueues. +\item The driver MUST NOT make any changes to Device Configuration Space except for \field{device status} if it is part of the Configuration Space. +\end{itemize} + +\devicenormative{\subsection}{Device Suspend}{General Initialization And Device Operation / Device Suspend} + +The device MUST ignore any operations on the SUSPEND bit from the driver if the device has not been completely initialized by the procedures in \ref{sec:General Initialization And Device Operation / Device Initialization} + +The device SHOULD ignore any write access to its Configuration Space while +suspended, except for \field{device status} if it is part of the Configuration Space. + +A device MUST NOT send any notifications for any virtqueues, +access any virtqueues, or modify any fields in +its Configuration Space while suspended. + +If changes occur in the Configuration Space during suspended period, +the device MUST NOT send any configuration change notifications. +Instead, the device MUST send the notification when it resumes running. + +If the driver sets the SUSPEND bit in \field{device status} to 1, +the device MUST either suspend itself or set the DEVICE_NEEDS_RESET bit in \field{device status} to 1 when it fails to suspend. + +If the device has been suspended and the driver resumes the device running by setting the DRIVER_OK bit in \field{device status} to 1, +the device MUST either resume normal operation or set the DEVICE_NEEDS_RESET bit in \field{device status} to 1 when it fails to resume. + +When the driver sets the SUSPEND bit to 1, +the device SHOULD perform the following actions before presenting the SUSPEND bit as 1 and DRIVER_OK bit as 0 in the \field{device status}: + +\begin{itemize} +\item Stop consuming more buffers of any virtqueues. +\item Wait until all buffers that are being processed have been used. +\item Send used buffer notifications to the driver. +\end{itemize} + \chapter{Virtio Transport Options}\label{sec:Virtio Transport Options} Virtio can use various different buses, thus the standard is split
@@ -872,6 +923,11 @@ \chapter{Reserved Feature Bits}\label{sec:Reserved Feature Bits} \ref{devicenormative:Basic Facilities of a Virtio Device / Feature Bits} for handling features reserved for future use. + \item[VIRTIO_F_SUSPEND(43)] This feature indicates that the driver can + suspend the device by set the SUSPEND bit to 1. + See \ref{sec:Basic Facilities of a Virtio Device / Device Status Field}. + + \end{description} \drivernormative{\section}{Reserved Feature Bits}{Reserved Feature Bits}
Signature · 1 line
2.49.0
Hello Michael
No comments since last week, so shall we start the voting process?
Thanks
Zhu Lingshan
Quoted 127 lines · Zhu Lingshan
This commit allows the driver to suspend the device through a new device status bit SUSPEND and resume the device running by re-setting
DRIVER_OKbit in device status.This commit re-orders the device status bits.
Signed-off-by Zhu Lingshan <[email protected]>Signed-off-by Jason Wang <[email protected]>--- content.tex | 74 ++++++++++++++++++++++++++++++++++++++++++++++------- 1 file changed, 65 insertions(+), 9 deletions(-)content.tex +65
-9@@ -36,19 +36,22 @@ \section{\field{Device Status} Field}\label{sec:Basic Facilities of a Virtio Dev this bit. For example, under Linux, drivers can be loadable modules. \end{note} -\item[FAILED (128)] Indicates that something went wrong in the guest, - and it has given up on the device. This could be an internal - error, or the driver didn't like the device for some reason, or - even a fatal error during device operation. +\item[DRIVER_OK (4)] Indicates that the driver is set up and ready to + drive the device. + +\item[SUSPEND (16)] When VIRTIO_F_SUSPEND is negotiated, indicates that the + device has been suspended by the driver. \item[FEATURES_OK (8)] Indicates that the driver has acknowledged all the features it understands, and feature negotiation is complete. -\item[DRIVER_OK (4)] Indicates that the driver is set up and ready to - drive the device. - \item[DEVICE_NEEDS_RESET (64)] Indicates that the device has experienced an error from which it can't recover. + +\item[FAILED (128)] Indicates that something went wrong in the guest, + and it has given up on the device. This could be an internal + error, or the driver didn't like the device for some reason, or + even a fatal error during device operation. \end{description} The \field{device status} field starts out as 0, and is reinitialized to 0 by@@ -99,10 +102,10 @@ \section{Feature Bits}\label{sec:Basic Facilities of a Virtio Device / Feature B \begin{description} \item[0 to 23, and 50 to 127] Feature bits for the specific device type -\item[24 to 41] Feature bits reserved for extensions to the queue and +\item[24 to 43] Feature bits reserved for extensions to the queue and feature negotiation mechanisms, see \ref{sec:Reserved Feature Bits} -\item[42 to 49, and 128 and above] Feature bits reserved for future extensions. +\item[44 to 49, and 128 and above] Feature bits reserved for future extensions. \end{description} \begin{note}@@ -629,6 +632,54 @@ \section{Device Cleanup}\label{sec:General Initialization And Device Operation / Thus a driver MUST ensure a virtqueue isn't live (by device reset) before removing exposed buffers. +\section{Device Suspend}\label{sec:General Initialization And Device Operation / Device Suspend} + +If VIRTIO_F_SUSPEND is negotiated, the driver is eligible to suspend the device by setting the SUSPEND bit in \field{device status} to 1, and the device SHOULD set the DRIVER_OK bit to 0 once it has been suspended. + +If the device has been suspended, the driver can resume the device running by setting the DRIVER_OK bit in \field{device status} to 1, and the device should set the SUSPEND bit to 0 once it resumes running. + +\drivernormative{\subsection}{Device Suspend}{General Initialization And Device Operation / Device Suspend} + +The driver SHOULD NOT set SUSPEND bit if DRIVER_OK is not set or VIRTIO_F_SUSPEND is not negotiated. + +Once the driver sets SUSPEND bit in \field{device status} to 1: +\begin{itemize} +\item The driver MUST verify whether the device has been suspended by re-reading \field{device status}, examining whether the SUSPEND bit is set to 1 and the DRIVER_OK bit is set to 0. +\item The driver MUST NOT make any more buffers available to the device. +\item The driver MUST NOT send notifications for any virtqueues. +\item The driver MUST NOT make any changes to Device Configuration Space except for \field{device status} if it is part of the Configuration Space. +\end{itemize} + +\devicenormative{\subsection}{Device Suspend}{General Initialization And Device Operation / Device Suspend} + +The device MUST ignore any operations on the SUSPEND bit from the driver if the device has not been completely initialized by the procedures in \ref{sec:General Initialization And Device Operation / Device Initialization} + +The device SHOULD ignore any write access to its Configuration Space while +suspended, except for \field{device status} if it is part of the Configuration Space. + +A device MUST NOT send any notifications for any virtqueues, +access any virtqueues, or modify any fields in +its Configuration Space while suspended. + +If changes occur in the Configuration Space during suspended period, +the device MUST NOT send any configuration change notifications. +Instead, the device MUST send the notification when it resumes running. + +If the driver sets the SUSPEND bit in \field{device status} to 1, +the device MUST either suspend itself or set the DEVICE_NEEDS_RESET bit in \field{device status} to 1 when it fails to suspend. + +If the device has been suspended and the driver resumes the device running by setting the DRIVER_OK bit in \field{device status} to 1, +the device MUST either resume normal operation or set the DEVICE_NEEDS_RESET bit in \field{device status} to 1 when it fails to resume. + +When the driver sets the SUSPEND bit to 1, +the device SHOULD perform the following actions before presenting the SUSPEND bit as 1 and DRIVER_OK bit as 0 in the \field{device status}: + +\begin{itemize} +\item Stop consuming more buffers of any virtqueues. +\item Wait until all buffers that are being processed have been used. +\item Send used buffer notifications to the driver. +\end{itemize} + \chapter{Virtio Transport Options}\label{sec:Virtio Transport Options} Virtio can use various different buses, thus the standard is split@@ -872,6 +923,11 @@ \chapter{Reserved Feature Bits}\label{sec:Reserved Feature Bits} \ref{devicenormative:Basic Facilities of a Virtio Device / Feature Bits} for handling features reserved for future use. + \item[VIRTIO_F_SUSPEND(43)] This feature indicates that the driver can + suspend the device by set the SUSPEND bit to 1. + See \ref{sec:Basic Facilities of a Virtio Device / Device Status Field}. + + \end{description} \drivernormative{\section}{Reserved Feature Bits}{Reserved Feature Bits}
Hi,
Quoted 9 lines
From: Zhu Lingshan <[email protected]>
Sent: Thursday, May 29, 2025 1:54 PMThis commit allows the driver to suspend the device through a new device status bit SUSPEND and resume the device running by re-setting
DRIVER_OKbit in device status.This commit re-orders the device status bits.
I overall like this proposal. Few small comments to fix more below. It is lot cleaner as it eliminates the previous bit flipping complexities.
Can you please add one or two line on a use case/motivation,
For example,
Presently when an OS is suspended, the driver must fully stop and reset the device, and on resume the driver must re-initialize the device. This can be time consuming task to recreate large number of VQs.
Instead, extend the specification to allow drivers to suspend and resume the device, without recreating all the resources.
...
Quoted 32 lines
Signed-off-by Zhu Lingshan <[email protected]>Signed-off-by Jason Wang <[email protected]>--- content.tex | 74 ++++++++++++++++++++++++++++++++++++++++++++++----- -- 1 file changed, 65 insertions(+), 9 deletions(-)content.tex +0
-0@@ -36,19 +36,22 @@ \section{\field{Device Status} Field}\label{sec:BasicFacilities of a Virtio Dev
this bit. For example, under Linux, drivers can be loadable modules.
\end{note}-\item[FAILED (128)] Indicates that something went wrong in the guest,
content.tex +6
-4- and it has given up on the device. This could be an internal - error, or the driver didn't like the device for some reason, or - even a fatal error during device operation. +\item[DRIVER_OK (4)] Indicates that the driver is set up and ready to + drive the device. + +\item[SUSPEND (16)] When VIRTIO_F_SUSPEND is negotiated, indicates that +the + device has been suspended by the driver. \item[FEATURES_OK (8)] Indicates that the driver has acknowledged all the features it understands, and feature negotiation is complete. -\item[DRIVER_OK (4)] Indicates that the driver is set up and ready to
content.tex \section{\field{Device Status} Field}\label{sec:Basic Facilities of a Virtio Dev · line 55
Quoted 10 lines
content.tex +5
-2- drive the device. - \item[DEVICE_NEEDS_RESET (64)] Indicates that the device has experienced an error from which it can't recover. + +\item[FAILED (128)] Indicates that something went wrong in the guest, + and it has given up on the device. This could be an internal + error, or the driver didn't like the device for some reason, or + even a fatal error during device operation. \end{description}
A new bit SUSPEND introduction should be done at the end without re-organizing rest of the bits. Please add at the end.
Is the intention to rearrange the fields in ascending order of the bit number? If yes, its good change too but than it missed re-arranging FEATURE_OK.
If you want to re-arrange, please have the pre-patch in this series.
Either way is ok to me,
1. Either as pre-patch for re-arrangement + addition of SUSPEND in middle. Or
2. adding SUSPEND bit definition at the end
#1 is better to see in ascending order.
Quoted 9 lines
The \field{device status} field starts out as 0, and is reinitialized to 0 by @@ -
99,10 +102,10 @@ \section{Feature Bits}\label{sec:Basic Facilities of a Virtio Device / Feature B \begin{description}
\item[0 to 23, and 50 to 127] Feature bits for the specific device type-\item[24 to 41] Feature bits reserved for extensions to the queue and +\item[24 to 43] Feature bits reserved for extensions to the queue and
feature negotiation mechanisms, see \ref{sec:Reserved Feature Bits}
Only one feature bit is used by this proposal.
So it should be 24 to 42 and use bit 42.
Did you use bit 43 due to existence of VIRTIO_NET_F_GUEST_RSC6?
Quoted 17 lines
-\item[42 to 49, and 128 and above] Feature bits reserved for future extensions.
+\item[44 to 49, and 128 and above] Feature bits reserved for future extensions.
\end{description}\begin{note}
content.tex +0
-0@@ -629,6 +632,54 @@ \section{Device Cleanup}\label{sec:GeneralInitialization And Device Operation /
Thus a driver MUST ensure a virtqueue isn't live (by device reset) before
removing exposed buffers.+\section{Device Suspend}\label{sec:General Initialization And Device
content.tex +3
-0+Operation / Device Suspend} + +If VIRTIO_F_SUSPEND is negotiated, the driver is eligible to suspend the
Quoted 3 lines
device by setting the SUSPEND bit in \field{device status} to 1, and the device SHOULD set the
DRIVER_OKbit to 0 once it has been suspended.
+
This is not a normative section so please avoid using SHOULD. Better to word it as informative theory of operation text. Such as,
If X is not negotiated, the....
And the device sets the DRIVER_OK bit to 0 once it has been suspend.
Quoted 4 lines
+If the device has been suspended, the driver can resume the device running by setting the
DRIVER_OKbit in \field{device status} to 1, and the device should set the SUSPEND bit to 0 once it resumes running.
+
The device sets the SUSPEND bit to 0 once the device is running.
Quoted 39 lines
+\drivernormative{\subsection}{Device Suspend}{General Initialization
content.tex +3
-0+And Device Operation / Device Suspend} + +The driver SHOULD NOT set SUSPEND bit if DRIVER_OK is not set or
VIRTIO_F_SUSPENDis not negotiated.content.tex +4
-0+ +Once the driver sets SUSPEND bit in \field{device status} to 1: +\begin{itemize} +\item The driver MUST verify whether the device has been suspended byre-reading \field{device status}, examining whether the SUSPEND bit is set to 1 and the
DRIVER_OKbit is set to 0.
+\item The driver MUST NOT make any more buffers available to the device. +\item The driver MUST NOT send notifications for any virtqueues. +\item The driver MUST NOT make any changes to Device Configuration Space except for \field{device status} if it is part of the Configuration Space. +\end{itemize}content.tex +11
-0+ +\devicenormative{\subsection}{Device Suspend}{General Initialization +And Device Operation / Device Suspend} + +The device MUST ignore any operations on the SUSPEND bit from the +driver if the device has not been completely initialized by the +procedures in \ref{sec:General Initialization And Device Operation / +Device Initialization} + +The device SHOULD ignore any write access to its Configuration Space +while suspended, except for \field{device status} if it is part of theConfiguration Space.
content.tex +11
-0+ +A device MUST NOT send any notifications for any virtqueues, access any +virtqueues, or modify any fields in its Configuration Space while +suspended. + +If changes occur in the Configuration Space during suspended period, +the device MUST NOT send any configuration change notifications. +Instead, the device MUST send the notification when it resumes running. + +If the driver sets the SUSPEND bit in \field{device status} to 1, the +device MUST either suspend itself or set the DEVICE_NEEDS_RESET bit in
Quoted 2 lines
\field{device status} to 1 when it fails to suspend. +
Even though you have the driver normative, it is good to have device normative for suspend and resume operation:
Something like,
When the suspend operation completes in the device, the device MUST set SUSPEND bit to 1 and DRIVERK_OK to 0.
When the suspended device is resumed, the device MUST set SUSPEND bit to 0 and DRIVER_OK to 1.
content.tex
Quoted 6 lines
content.tex +2
-0+If the device has been suspended and the driver resumes the device +running by setting the DRIVER_OK bit in \field{device status} to 1, thedevice MUST either resume normal operation or set the
DEVICE_NEEDS_RESETbit in \field{device status} to 1 when it fails to resume.content.tex +2
-0+ +When the driver sets the SUSPEND bit to 1, the device SHOULD perform
It should be MUST.
Otherwise it can crash the system accessing memory that may not be any more accessible.
Quoted 29 lines
+the following actions before presenting the SUSPEND bit as 1 and
DRIVER_OKbit as 0 in the \field{device status}:content.tex +8
-0+ +\begin{itemize} +\item Stop consuming more buffers of any virtqueues. +\item Wait until all buffers that are being processed have been used. +\item Send used buffer notifications to the driver. +\end{itemize} + \chapter{Virtio Transport Options}\label{sec:Virtio Transport Options} Virtio can use various different buses, thus the standard is split @@ -872,6 +923,11 @@ \chapter{Reserved Feature Bits}\label{sec:Reserved FeatureBits}
\ref{devicenormative:Basic Facilities of a Virtio Device / Feature Bits}
forcontent.tex +5
-1handling features reserved for future use. + \item[VIRTIO_F_SUSPEND(43)] This feature indicates that the driver can + suspend the device by set the SUSPEND bit to 1. + See \ref{sec:Basic Facilities of a Virtio Device / Device Status Field}. + + \end{description} \drivernormative{\section}{Reserved Feature Bits}{Reserved Feature Bits} --
Quoted 2 lines
2.49.0
Apart from these small comments, please add the line in v3 commit log:
https://github.com/oasis-tcs/virtio-spec/issues/<issue_number>
This will help to link the two sides.
Quoted 16 lines · Parav Pandit
Hi,
Quoted 9 lines
From: Zhu Lingshan <[email protected]>
Sent: Thursday, May 29, 2025 1:54 PMThis commit allows the driver to suspend the device through a new device status bit SUSPEND and resume the device running by re-setting
DRIVER_OKbit in device status.This commit re-orders the device status bits.
I overall like this proposal. Few small comments to fix more below. It is lot cleaner as it eliminates the previous bit flipping complexities.
Can you please add one or two line on a use case/motivation,
Quoted 7 lines
For example,
Presently when an OS is suspended, the driver must fully stop and reset the device, and on resume the driver must re-initialize the device. This can be time consuming task to recreate large number of VQs.
Instead, extend the specification to allow drivers to suspend and resume the device, without recreating all the resources.
...
Hello Parav
Here I think the spec provides mechanisms, not a policy. The user may employ this feature for other tasks other than live migration which requires OS suspending.
Quoted 47 lines
Quoted 34 lines
Signed-off-by Zhu Lingshan <[email protected]>Signed-off-by Jason Wang <[email protected]>--- content.tex | 74 ++++++++++++++++++++++++++++++++++++++++++++++----- -- 1 file changed, 65 insertions(+), 9 deletions(-)content.tex +0
-0@@ -36,19 +36,22 @@ \section{\field{Device Status} Field}\label{sec:BasicFacilities of a Virtio Dev
this bit. For example, under Linux, drivers can be loadable modules.
\end{note}-\item[FAILED (128)] Indicates that something went wrong in the guest,
content.tex +6
-4- and it has given up on the device. This could be an internal - error, or the driver didn't like the device for some reason, or - even a fatal error during device operation. +\item[DRIVER_OK (4)] Indicates that the driver is set up and ready to + drive the device. + +\item[SUSPEND (16)] When VIRTIO_F_SUSPEND is negotiated, indicates that +the + device has been suspended by the driver. \item[FEATURES_OK (8)] Indicates that the driver has acknowledged all the features it understands, and feature negotiation is complete. -\item[DRIVER_OK (4)] Indicates that the driver is set up and ready tocontent.tex \section{\field{Device Status} Field}\label{sec:Basic Facilities of a Virtio Dev · line 55
Quoted 10 lines
content.tex +5
-2- drive the device. - \item[DEVICE_NEEDS_RESET (64)] Indicates that the device has experienced an error from which it can't recover. + +\item[FAILED (128)] Indicates that something went wrong in the guest, + and it has given up on the device. This could be an internal + error, or the driver didn't like the device for some reason, or + even a fatal error during device operation. \end{description}A new bit SUSPEND introduction should be done at the end without re-organizing rest of the bits. Please add at the end.
Quoted 2 lines
Is the intention to rearrange the fields in ascending order of the bit number? If yes, its good change too but than it missed re-arranging
FEATURE_OK.
oh yes, good catch, will fix
Quoted 9 lines
If you want to re-arrange, please have the pre-patch in this series.
Either way is ok to me,
1. Either as pre-patch for re-arrangement + addition of SUSPEND in middle. Or
2. adding SUSPEND bit definition at the end#1 is better to see in ascending order.
I was required to re-arrange these status bits, but this is a trivial change, so lets keep them in a single patch for easier review
Quoted 6 lines
Quoted 6 lines
The \field{device status} field starts out as 0, and is reinitialized to 0 by @@ -
99,10 +102,10 @@ \section{Feature Bits}\label{sec:Basic Facilities of a Virtio Device / Feature B \begin{description}
\item[0 to 23, and 50 to 127] Feature bits for the specific device type
Quoted 8 lines
Quoted 4 lines
-\item[24 to 41] Feature bits reserved for extensions to the queue and +\item[24 to 43] Feature bits reserved for extensions to the queue and
feature negotiation mechanisms, see \ref{sec:Reserved Feature Bits}Only one feature bit is used by this proposal.
So it should be 24 to 42 and use bit 42.Did you use bit 43 due to existence of
VIRTIO_NET_F_GUEST_RSC6?
Yes, bit 42 is taken for VIRTIO_NET_F_GUEST_RSC6.
Quoted 14 lines
Quoted 14 lines
-\item[42 to 49, and 128 and above] Feature bits reserved for future extensions.
+\item[44 to 49, and 128 and above] Feature bits reserved for future extensions.
\end{description}\begin{note}
content.tex +0
-0@@ -629,6 +632,54 @@ \section{Device Cleanup}\label{sec:GeneralInitialization And Device Operation /
Thus a driver MUST ensure a virtqueue isn't live (by device reset) before
removing exposed buffers.
Quoted 10 lines
Quoted 7 lines
+\section{Device Suspend}\label{sec:General Initialization And Device
content.tex +3
-0+Operation / Device Suspend} + +If VIRTIO_F_SUSPEND is negotiated, the driver is eligible to suspend thedevice by setting the SUSPEND bit in \field{device status} to 1, and the device SHOULD set the
DRIVER_OKbit to 0 once it has been suspended.
+This is not a normative section so please avoid using SHOULD. Better to word it as informative theory of operation text. Such as,
why "SHOULD" can only be used in normative sections?
Quoted 9 lines
If X is not negotiated, the....
And the device sets theDRIVER_OKbit to 0 once it has been suspend.Quoted 4 lines
+If the device has been suspended, the driver can resume the device running by setting the
DRIVER_OKbit in \field{device status} to 1, and the device should set the SUSPEND bit to 0 once it resumes running.
+The device sets the SUSPEND bit to 0 once the device is running.
What is the difference in semantic?
Quoted 43 lines
Quoted 39 lines
+\drivernormative{\subsection}{Device Suspend}{General Initialization
content.tex +3
-0+And Device Operation / Device Suspend} + +The driver SHOULD NOT set SUSPEND bit if DRIVER_OK is not set or
VIRTIO_F_SUSPENDis not negotiated.content.tex +4
-0+ +Once the driver sets SUSPEND bit in \field{device status} to 1: +\begin{itemize} +\item The driver MUST verify whether the device has been suspended byre-reading \field{device status}, examining whether the SUSPEND bit is set to 1 and the
DRIVER_OKbit is set to 0.
+\item The driver MUST NOT make any more buffers available to the device. +\item The driver MUST NOT send notifications for any virtqueues. +\item The driver MUST NOT make any changes to Device Configuration Space except for \field{device status} if it is part of the Configuration Space. +\end{itemize}content.tex +11
-0+ +\devicenormative{\subsection}{Device Suspend}{General Initialization +And Device Operation / Device Suspend} + +The device MUST ignore any operations on the SUSPEND bit from the +driver if the device has not been completely initialized by the +procedures in \ref{sec:General Initialization And Device Operation / +Device Initialization} + +The device SHOULD ignore any write access to its Configuration Space +while suspended, except for \field{device status} if it is part of theConfiguration Space.
content.tex +11
-0+ +A device MUST NOT send any notifications for any virtqueues, access any +virtqueues, or modify any fields in its Configuration Space while +suspended. + +If changes occur in the Configuration Space during suspended period, +the device MUST NOT send any configuration change notifications. +Instead, the device MUST send the notification when it resumes running. + +If the driver sets the SUSPEND bit in \field{device status} to 1, the +device MUST either suspend itself or set the DEVICE_NEEDS_RESET bit inQuoted 2 lines
\field{device status} to 1 when it fails to suspend. +
Even though you have the driver normative, it is good to have device normative for suspend and resume operation:
Quoted 6 lines
Something like,
When the suspend operation completes in the device, the device MUST set SUSPEND bit to 1 and
DRIVERK_OKto 0.When the suspended device is resumed, the device MUST set SUSPEND bit to 0 and
DRIVER_OKto 1.
I think we already have similar descriptions in the beginning of this section
Quoted 10 lines
content.tex
Quoted 8 lines
content.tex +2
-0+If the device has been suspended and the driver resumes the device +running by setting the DRIVER_OK bit in \field{device status} to 1, thedevice MUST either resume normal operation or set the
DEVICE_NEEDS_RESETbit in \field{device status} to 1 when it fails to resume.content.tex +2
-0+ +When the driver sets the SUSPEND bit to 1, the device SHOULD performIt should be MUST.
Otherwise it can crash the system accessing memory that may not be any more accessible.
Not sure how this can crash system memory, but "MUST" is OK
Quoted 33 lines
Quoted 29 lines
+the following actions before presenting the SUSPEND bit as 1 and
DRIVER_OKbit as 0 in the \field{device status}:content.tex +8
-0+ +\begin{itemize} +\item Stop consuming more buffers of any virtqueues. +\item Wait until all buffers that are being processed have been used. +\item Send used buffer notifications to the driver. +\end{itemize} + \chapter{Virtio Transport Options}\label{sec:Virtio Transport Options} Virtio can use various different buses, thus the standard is split @@ -872,6 +923,11 @@ \chapter{Reserved Feature Bits}\label{sec:Reserved FeatureBits}
\ref{devicenormative:Basic Facilities of a Virtio Device / Feature Bits}
forcontent.tex +5
-1handling features reserved for future use. + \item[VIRTIO_F_SUSPEND(43)] This feature indicates that the driver can + suspend the device by set the SUSPEND bit to 1. + See \ref{sec:Basic Facilities of a Virtio Device / Device Status Field}. + + \end{description} \drivernormative{\section}{Reserved Feature Bits}{Reserved Feature Bits} --Quoted 2 lines
2.49.0
Apart from these small comments, please add the line in v3 commit log:
Quoted 3 lines
https://github.com/oasis-tcs/virtio-spec/issues/<issue_number>
This will help to link the two sides.
sure
Quoted 29 lines
From: Zhu Lingshan <[email protected]>
Sent: Friday, June 6, 2025 4:01 PMQuoted 11 lines · Parav Pandit
Hi,
Quoted 9 lines
From: Zhu Lingshan <[email protected]>
Sent: Thursday, May 29, 2025 1:54 PMThis commit allows the driver to suspend the device through a new device status bit SUSPEND and resume the device running by re-setting
DRIVER_OKbit in device status.This commit re-orders the device status bits.
Quoted 8 lines
I overall like this proposal. Few small comments to fix more below. It is lot cleaner as it eliminates the previous bit flipping complexities.
Can you please add one or two line on a use case/motivation,
For example,
Presently when an OS is suspended, the driver must fully stop and reset the
device, and on resume the driver must re-initialize the device.
Quoted 3 lines
This can be time consuming task to recreate large number of VQs.
Instead, extend the specification to allow drivers to suspend and resume
Quoted 9 lines
the device, without recreating all the resources.
Quoted 1 line
...
Hello Parav
Here I think the spec provides mechanisms, not a policy. The user may employ this feature for other tasks other than live migration which requires OS suspending.
Its not about some policy.
Sure user may use the feature in N different use cases.
The intent is very simple. For the reviewers to review the patch, why something is needed should be presented. So motivation/usecase/need is worth the entry.
Some reviewers do rely on the commit logs to understand the design intent behind it.
Without the need of the patch, I am not sure why would even one write and review the spec patch. Please add known use cases. I am familiar with one that I suggested above.
Quoted 63 lines
Quoted 6 lines
Quoted 6 lines
Signed-off-by Zhu Lingshan <[email protected]>Signed-off-by Jason Wang <[email protected]>--- content.tex | 74 ++++++++++++++++++++++++++++++++++++++++++++++-----
Quoted 24 lines
Quoted 24 lines
--
1 file changed, 65 insertions(+), 9 deletions(-)content.tex +0
-0@@ -36,19 +36,22 @@ \section{\field{Device Status}Field}\label{sec:Basic Facilities of a Virtio Dev
this bit. For example, under Linux, drivers can be loadable modules.
\end{note}-\item[FAILED (128)] Indicates that something went wrong in the guest,
content.tex +7
-3- and it has given up on the device. This could be an internal - error, or the driver didn't like the device for some reason, or - even a fatal error during device operation. +\item[DRIVER_OK (4)] Indicates that the driver is set up and ready +to + drive the device. + +\item[SUSPEND (16)] When VIRTIO_F_SUSPEND is negotiated, indicates +that the + device has been suspended by the driver.Quoted 1 line
Quoted 1 line
\item[FEATURES_OK (8)] Indicates that the driver has acknowledged all
the
Quoted 7 lines
content.tex \section{\field{Device Status} Field}\label{sec:Basic Facilities of a Virtio Dev
Quoted 7 lines
features it understands, and feature negotiation is complete.
-\item[DRIVER_OK (4)] Indicates that the driver is set up and ready to
content.tex +0
-2- drive the device. - \item[DEVICE_NEEDS_RESET (64)] Indicates that the device hasexperienced
Quoted 10 lines
content.tex \section{\field{Device Status} Field}\label{sec:Basic Facilities of a Virtio Dev · line 55
Quoted 9 lines
an error from which it can't recover.
content.tex +6
-0+ +\item[FAILED (128)] Indicates that something went wrong in the +guest, + and it has given up on the device. This could be an internal + error, or the driver didn't like the device for some reason, or + even a fatal error during device operation. \end{description}A new bit SUSPEND introduction should be done at the end without re-
organizing rest of the bits.
Quoted 3 lines
Please add at the end.
Is the intention to rearrange the fields in ascending order of the bit
number?
Quoted 1 line
If yes, its good change too but than it missed re-arranging
FEATURE_OK.oh yes, good catch, will fix
Quoted 3 lines
If you want to re-arrange, please have the pre-patch in this series.
Quoted 8 lines
Quoted 6 lines
Either way is ok to me,
1. Either as pre-patch for re-arrangement + addition of SUSPEND in middle. Or
2. adding SUSPEND bit definition at the end#1 is better to see in ascending order.
I was required to re-arrange these status bits, but this is a trivial change, so lets keep them in a single patch for easier review
Having logically separate patches are always good. Re-arrangement is pure cosmetic vs this one that adds new functionality.
Quoted 15 lines
Quoted 15 lines
Quoted 15 lines
The \field{device status} field starts out as 0, and is
reinitialized to 0 by @@ -
99,10 +102,10 @@ \section{Feature Bits}\label{sec:Basic Facilities of a Virtio Device / Feature B \begin{description}
\item[0 to 23, and 50 to 127] Feature bits for the specific device
type-\item[24 to 41] Feature bits reserved for extensions to the queue and
+\item[24 to 43] Feature bits reserved for extensions to the queue +and
feature negotiation mechanisms, see \ref{sec:Reserved Feature
Bits}
Quoted 6 lines
Quoted 4 lines
Only one feature bit is used by this proposal.
So it should be 24 to 42 and use bit 42.Did you use bit 43 due to existence of
VIRTIO_NET_F_GUEST_RSC6?Yes, bit 42 is taken for
VIRTIO_NET_F_GUEST_RSC6.
So the current spec wordings is incorrect 0 to 23, and 50 to 127 Feature bits for the specific device type
We need to says,
0 to 23, 41 to 42 and 50 to 127 Feature bits for the specific device type This should also be pre-patch, so that it can be backported to master branch. Another reason for the small patch to be separate.
If you want me to write up this pre-patch, please let me know. You can include in the series.
Quoted 23 lines
Quoted 15 lines
Quoted 15 lines
-\item[42 to 49, and 128 and above] Feature bits reserved for future extensions.
+\item[44 to 49, and 128 and above] Feature bits reserved for future extensions.
\end{description}\begin{note}
content.tex +0
-0@@ -629,6 +632,54 @@ \section{Device Cleanup}\label{sec:GeneralInitialization And Device Operation /
Thus a driver MUST ensure a virtqueue isn't live (by device reset)
before removing exposed buffers.Quoted 7 lines
Quoted 7 lines
+\section{Device Suspend}\label{sec:General Initialization And Device
content.tex +4
-0+Operation / Device Suspend} + +If VIRTIO_F_SUSPEND is negotiated, the driver is eligible to suspend +thedevice by setting the SUSPEND bit in \field{device status} to 1, and the device SHOULD set the
DRIVER_OKbit to 0 once it has beensuspended.
Quoted 7 lines
Quoted 4 lines
Quoted 1 line
+
This is not a normative section so please avoid using SHOULD. Better to word it as informative theory of operation text. Such as,
why "SHOULD" can only be used in normative sections?
Because:
1. To clarify the intent:
These keywords carry precise semantic weight. When a spec says MUST, it means compliance is not optional. Using such terms in general or explanatory text risks misinterpreting guidance as a requirement.
2. Avoiding Ambiguity:
If normative language is used in non-normative (informative) sections, it becomes unclear whether the reader is required to follow it or if it's just background or suggestion.
3. Informative sections explain context, rationale, or examples. Normative sections define rules. Mixing the two muddies the structure and weakens the authority of the normative part.
Quoted 4 lines
Quoted 4 lines
If X is not negotiated, the....
And the device sets theDRIVER_OKbit to 0 once it has been suspend.
Quoted 9 lines
Quoted 6 lines
Quoted 5 lines
content.tex +2
-0+If the device has been suspended, the driver can resume the device +runningby setting the
DRIVER_OKbit in \field{device status} to 1, and the device should set the SUSPEND bit to 0 once it resumes running. +The device sets the SUSPEND bit to 0 once the device is running.
What is the difference in semantic?
"resumes running" does not read correct.
And normative "should" is used...
Hence the suggestion to re-word it.
Quoted 52 lines
Quoted 6 lines
Quoted 6 lines
+\drivernormative{\subsection}{Device Suspend}{General Initialization
content.tex +3
-0+And Device Operation / Device Suspend} + +The driver SHOULD NOT set SUSPEND bit if DRIVER_OK is not set or
VIRTIO_F_SUSPENDis not negotiated.
+Quoted 8 lines
Quoted 8 lines
content.tex +4
-0+Once the driver sets SUSPEND bit in \field{device status} to 1: +\begin{itemize} +\item The driver MUST verify whether the device has been suspended +byre-reading \field{device status}, examining whether the SUSPEND bit is set to
1 and theDRIVER_OKbit is set to 0.
+\item The driver MUST NOT make any more buffers available to thedevice.
Quoted 3 lines
Quoted 3 lines
+\item The driver MUST NOT send notifications for any virtqueues. +\item The driver MUST NOT make any changes to Device Configuration Space except for \field{device status} if it is part of the Configuration
Space.
Quoted 21 lines
Quoted 21 lines
+\end{itemize}
content.tex +12
-0+ +\devicenormative{\subsection}{Device Suspend}{General Initialization +And Device Operation / Device Suspend} + +The device MUST ignore any operations on the SUSPEND bit from the +driver if the device has not been completely initialized by the +procedures in \ref{sec:General Initialization And Device Operation / +Device Initialization} + +The device SHOULD ignore any write access to its Configuration Space +while suspended, except for \field{device status} if it is part of +theConfiguration Space.
content.tex +7
-0+ +A device MUST NOT send any notifications for any virtqueues, access +any virtqueues, or modify any fields in its Configuration Space +while suspended. + +If changes occur in the Configuration Space during suspended period, +the device MUST NOT send any configuration change notifications.Quoted 8 lines
Quoted 7 lines
content.tex +5
-0+Instead, the device MUST send the notification when it resumes running. + +If the driver sets the SUSPEND bit in \field{device status} to 1, +the device MUST either suspend itself or set the DEVICE_NEEDS_RESET +bit in\field{device status} to 1 when it fails to suspend. +
Even though you have the driver normative, it is good to have device
normative for suspend and resume operation:
Quoted 3 lines
Something like,
Quoted 8 lines
Quoted 4 lines
When the suspend operation completes in the device, the device MUST set SUSPEND bit to 1 and
DRIVERK_OKto 0.When the suspended device is resumed, the device MUST set SUSPEND bit
to 0 and
DRIVER_OKto 1.I think we already have similar descriptions in the beginning of this section
I didn't find it in the normative device section.
Quoted 8 lines
Quoted 7 lines
Quoted 7 lines
content.tex +3
-0+If the device has been suspended and the driver resumes the device +running by setting the DRIVER_OK bit in \field{device status} to 1, +thedevice MUST either resume normal operation or set the
DEVICE_NEEDS_RESETbit in \field{device status} to 1 when it fails toresume.
Quoted 8 lines
Quoted 4 lines
content.tex
Quoted 2 lines
content.tex +2
-0+ +When the driver sets the SUSPEND bit to 1, the device SHOULD performIt should be MUST.
Otherwise it can crash the system accessing memory that may not be anymore accessible.
Not sure how this can crash system memory, but "MUST" is OK
Ok.
Quoted 49 lines
Quoted 36 lines
Quoted 30 lines
+the following actions before presenting the SUSPEND bit as 1 and
DRIVER_OKbit as 0 in the \field{device status}:content.tex +7
-0+ +\begin{itemize} +\item Stop consuming more buffers of any virtqueues. +\item Wait until all buffers that are being processed have been used. +\item Send used buffer notifications to the driver. +\end{itemize} + \chapter{Virtio Transport Options}\label{sec:Virtio TransportOptions}
Virtio can use various different buses, thus the standard is split
@@ -872,6
+923,11 @@ \chapter{Reserved Feature Bits}\label{sec:Reserved Feature Bits}
\ref{devicenormative:Basic Facilities of a Virtio Device / Feature
Bits} forcontent.tex +5
-0handling features reserved for future use. + \item[VIRTIO_F_SUSPEND(43)] This feature indicates that the driver can + suspend the device by set the SUSPEND bit to 1. + See \ref{sec:Basic Facilities of a Virtio Device / Device Status Field}. + + \end{description} \drivernormative{\section}{Reserved Feature Bits}{Reserved FeatureQuoted 4 lines
Bits}
--
2.49.0Apart from these small comments, please add the line in v3 commit log:
Quoted 2 lines
https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgith
ub.com%2Foasis-tcs%2Fvirtio-spec%2Fissues%2F&data=05%7C02%7Cparav%40nv
Quoted 1 line
idia.com%7C0ef2b01694d942339bb308dda4e5365c%7C43083d15727340c1b 7db39ef
Quoted 1 line
d9ccc17a%7C0%7C0%7C638848026619639978%7CUnknown%7CTWFpbGZsb 3d8eyJFbXB0
Quoted 1 line
eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpb CIsIl
Quoted 1 line
Quoted 8 lines
dUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdata=0st2osYsAxascHHmHj0RgSyYAn9l yQRKqxOl
Quoted 3 lines
aIjE8dk%3D&reserved=0<issue_number>
This will help to link the two sides.
sure
Thanks.
Quoted 39 lines · Parav Pandit
Quoted 27 lines
From: Zhu Lingshan <[email protected]>
Sent: Friday, June 6, 2025 4:01 PMQuoted 11 lines · Parav Pandit
Hi,
Quoted 9 lines
From: Zhu Lingshan <[email protected]>
Sent: Thursday, May 29, 2025 1:54 PMThis commit allows the driver to suspend the device through a new device status bit SUSPEND and resume the device running by re-setting
DRIVER_OKbit in device status.This commit re-orders the device status bits.
Quoted 8 lines
I overall like this proposal. Few small comments to fix more below. It is lot cleaner as it eliminates the previous bit flipping complexities.
Can you please add one or two line on a use case/motivation,
For example,
Presently when an OS is suspended, the driver must fully stop and reset the
device, and on resume the driver must re-initialize the device.
Quoted 2 lines
This can be time consuming task to recreate large number of VQs.
Quoted 9 lines
Quoted 1 line
Instead, extend the specification to allow drivers to suspend and resume
the device, without recreating all the resources.
Quoted 1 line
...
Hello Parav
Here I think the spec provides mechanisms, not a policy. The user may employ this feature for other tasks other than live migration which requires OS suspending.
Its not about some policy.
Sure user may use the feature in N different use cases.
Quoted 6 lines
The intent is very simple. For the reviewers to review the patch, why something is needed should be presented. So motivation/usecase/need is worth the entry.
Some reviewers do rely on the commit logs to understand the design intent behind it.Without the need of the patch, I am not sure why would even one write and review the spec patch. Please add known use cases. I am familiar with one that I suggested above.
we can add these descriptions in the cover letter.
Quoted 66 lines
Quoted 66 lines
Quoted 6 lines
Quoted 6 lines
Signed-off-by Zhu Lingshan <[email protected]>Signed-off-by Jason Wang <[email protected]>--- content.tex | 74 ++++++++++++++++++++++++++++++++++++++++++++++-----
Quoted 24 lines
Quoted 24 lines
--
1 file changed, 65 insertions(+), 9 deletions(-)content.tex +0
-0@@ -36,19 +36,22 @@ \section{\field{Device Status}Field}\label{sec:Basic Facilities of a Virtio Dev
this bit. For example, under Linux, drivers can be loadable modules.
\end{note}-\item[FAILED (128)] Indicates that something went wrong in the guest,
content.tex +7
-3- and it has given up on the device. This could be an internal - error, or the driver didn't like the device for some reason, or - even a fatal error during device operation. +\item[DRIVER_OK (4)] Indicates that the driver is set up and ready +to + drive the device. + +\item[SUSPEND (16)] When VIRTIO_F_SUSPEND is negotiated, indicates +that the + device has been suspended by the driver.Quoted 1 line
Quoted 1 line
\item[FEATURES_OK (8)] Indicates that the driver has acknowledged all
the
Quoted 7 lines
content.tex \section{\field{Device Status} Field}\label{sec:Basic Facilities of a Virtio Dev
Quoted 7 lines
features it understands, and feature negotiation is complete.
-\item[DRIVER_OK (4)] Indicates that the driver is set up and ready to
content.tex +0
-2- drive the device. - \item[DEVICE_NEEDS_RESET (64)] Indicates that the device hasexperienced
Quoted 10 lines
content.tex \section{\field{Device Status} Field}\label{sec:Basic Facilities of a Virtio Dev · line 55
Quoted 9 lines
an error from which it can't recover.
content.tex +6
-0+ +\item[FAILED (128)] Indicates that something went wrong in the +guest, + and it has given up on the device. This could be an internal + error, or the driver didn't like the device for some reason, or + even a fatal error during device operation. \end{description}A new bit SUSPEND introduction should be done at the end without re-
organizing rest of the bits.
Quoted 3 lines
Please add at the end.
Is the intention to rearrange the fields in ascending order of the bit
number?
Quoted 1 line
If yes, its good change too but than it missed re-arranging
FEATURE_OK.oh yes, good catch, will fix
Quoted 7 lines
If you want to re-arrange, please have the pre-patch in this series.
Either way is ok to me,
1. Either as pre-patch for re-arrangement + addition of SUSPEND in middle. Or
2. adding SUSPEND bit definition at the end
Quoted 4 lines
Quoted 3 lines
Quoted 1 line
#1 is better to see in ascending order.
I was required to re-arrange these status bits, but this is a trivial change, so lets keep them in a single patch for easier review
Having logically separate patches are always good. Re-arrangement is pure cosmetic vs this one that adds new functionality.
ok, we can split this patch into a series if this is preferred.
Quoted 15 lines
Quoted 15 lines
Quoted 15 lines
Quoted 15 lines
The \field{device status} field starts out as 0, and is
reinitialized to 0 by @@ -
99,10 +102,10 @@ \section{Feature Bits}\label{sec:Basic Facilities of a Virtio Device / Feature B \begin{description}
\item[0 to 23, and 50 to 127] Feature bits for the specific device
type-\item[24 to 41] Feature bits reserved for extensions to the queue and
+\item[24 to 43] Feature bits reserved for extensions to the queue +and
feature negotiation mechanisms, see \ref{sec:Reserved Feature
Bits}
Quoted 6 lines
Quoted 5 lines
Quoted 4 lines
Only one feature bit is used by this proposal.
So it should be 24 to 42 and use bit 42.Did you use bit 43 due to existence of
VIRTIO_NET_F_GUEST_RSC6?Yes, bit 42 is taken for
VIRTIO_NET_F_GUEST_RSC6.So the current spec wordings is incorrect 0 to 23, and 50 to 127 Feature bits for the specific device type
current spec missed documenting bit 42.
Quoted 9 lines
We need to says,
0 to 23, 41 to 42 and 50 to 127 Feature bits for the specific device type This should also be pre-patch, so that it can be backported to master branch. Another reason for the small patch to be separate.
If you want me to write up this pre-patch, please let me know. You can include in the series.
There could a a separate patch in a series says:
41 to 42 Feature bits reserved for extensions to the queue and feature negotiation mechanisms 43 to 49, and 128 and above Feature bits reserved for future extensions.
which fixes commit 9438414
Quoted 30 lines
Quoted 22 lines
Quoted 14 lines
Quoted 14 lines
-\item[42 to 49, and 128 and above] Feature bits reserved for future extensions.
+\item[44 to 49, and 128 and above] Feature bits reserved for future extensions.
\end{description}\begin{note}
content.tex +0
-0@@ -629,6 +632,54 @@ \section{Device Cleanup}\label{sec:GeneralInitialization And Device Operation /
Thus a driver MUST ensure a virtqueue isn't live (by device reset)
before removing exposed buffers.Quoted 7 lines
Quoted 7 lines
+\section{Device Suspend}\label{sec:General Initialization And Device
content.tex +4
-0+Operation / Device Suspend} + +If VIRTIO_F_SUSPEND is negotiated, the driver is eligible to suspend +thedevice by setting the SUSPEND bit in \field{device status} to 1, and the device SHOULD set the
DRIVER_OKbit to 0 once it has beensuspended.
Quoted 6 lines
Quoted 4 lines
Quoted 1 line
+
This is not a normative section so please avoid using SHOULD. Better to word it as informative theory of operation text. Such as,
why "SHOULD" can only be used in normative sections?
Because:
Quoted 7 lines
1. To clarify the intent:
These keywords carry precise semantic weight. When a spec says MUST, it means compliance is not optional. Using such terms in general or explanatory text risks misinterpreting guidance as a requirement.2. Avoiding Ambiguity:
If normative language is used in non-normative (informative) sections, it becomes unclear whether the reader is required to follow it or if it's just background or suggestion.3. Informative sections explain context, rationale, or examples. Normative sections define rules. Mixing the two muddies the structure and weakens the authority of the normative part.
There are no such rules.
In the spec section 1.3 Terminology, it says:
The key words “MUST”, “MUST NOT”, “REQUIRED”, “SHALL”, “SHALL NOT”, “SHOULD”, “SHOULD NOT”, “RECOMMENDED”, “NOT RECOMMENDED”, “MAY”, and “OPTIONAL” in this document are to be interpreted as described in [RFC2119] and [RFC8174] when, and only when, they appear in all capitals, as shown here.
RFC 2119 says:
SHOULD This word, or the adjective "RECOMMENDED", mean that there
may exist valid reasons in particular circumstances to ignore a particular item, but the full implications must be understood and carefully weighed before choosing a different course.
Here this "SHOULD" exactly conforms to the definition.
"SHOULD" has already been used in non-normative sections, for example:
2.5.4 Legacy Interfaces: when using the legacy interface, drivers SHOULD read these fields multiple times until two reads generate a consistent result.
The spec does not say: “SHOULD” can only be used in driver or device requirements section.
Quoted 10 lines
Quoted 10 lines
Quoted 10 lines
If X is not negotiated, the....
And the device sets theDRIVER_OKbit to 0 once it has been suspend.Quoted 5 lines
content.tex +2
-0+If the device has been suspended, the driver can resume the device +runningby setting the
DRIVER_OKbit in \field{device status} to 1, and the device should set the SUSPEND bit to 0 once it resumes running. +The device sets the SUSPEND bit to 0 once the device is running.
Quoted 5 lines
Quoted 2 lines
What is the difference in semantic?
"resumes running" does not read correct.
And normative "should" is used...
Hence the suggestion to re-word it.
Looking into a dictionary https://www.merriam-webster.com/dictionary/resume
There is quite a lot of examples of resume + noun.
And here, "SHOULD" still conforms to RFC 2119.
Quoted 55 lines
Quoted 55 lines
Quoted 15 lines
Quoted 11 lines
+\drivernormative{\subsection}{Device Suspend}{General Initialization
content.tex +3
-0+And Device Operation / Device Suspend} + +The driver SHOULD NOT set SUSPEND bit if DRIVER_OK is not set or
VIRTIO_F_SUSPENDis not negotiated.content.tex +5
-0+ +Once the driver sets SUSPEND bit in \field{device status} to 1: +\begin{itemize} +\item The driver MUST verify whether the device has been suspended +byQuoted 4 lines
re-reading \field{device status}, examining whether the SUSPEND bit is set to
1 and theDRIVER_OKbit is set to 0.
+\item The driver MUST NOT make any more buffers available to thedevice.
Quoted 3 lines
Quoted 3 lines
+\item The driver MUST NOT send notifications for any virtqueues. +\item The driver MUST NOT make any changes to Device Configuration Space except for \field{device status} if it is part of the Configuration
Space.
Quoted 21 lines
Quoted 21 lines
+\end{itemize}
content.tex +12
-0+ +\devicenormative{\subsection}{Device Suspend}{General Initialization +And Device Operation / Device Suspend} + +The device MUST ignore any operations on the SUSPEND bit from the +driver if the device has not been completely initialized by the +procedures in \ref{sec:General Initialization And Device Operation / +Device Initialization} + +The device SHOULD ignore any write access to its Configuration Space +while suspended, except for \field{device status} if it is part of +theConfiguration Space.
content.tex +7
-0+ +A device MUST NOT send any notifications for any virtqueues, access +any virtqueues, or modify any fields in its Configuration Space +while suspended. + +If changes occur in the Configuration Space during suspended period, +the device MUST NOT send any configuration change notifications.Quoted 8 lines
Quoted 7 lines
content.tex +5
-0+Instead, the device MUST send the notification when it resumes running. + +If the driver sets the SUSPEND bit in \field{device status} to 1, +the device MUST either suspend itself or set the DEVICE_NEEDS_RESET +bit in\field{device status} to 1 when it fails to suspend. +
Even though you have the driver normative, it is good to have device
normative for suspend and resume operation:
Quoted 5 lines
Something like,
When the suspend operation completes in the device, the device MUST set SUSPEND bit to 1 and
DRIVERK_OKto 0.
Quoted 6 lines
Quoted 5 lines
Quoted 1 line
When the suspended device is resumed, the device MUST set SUSPEND bit
to 0 and
DRIVER_OKto 1.I think we already have similar descriptions in the beginning of this section
I didn't find it in the normative device section.
+\section{Device Suspend}\label{sec:General Initialization And Device Operation / Device Suspend}
+
+If VIRTIO_F_SUSPEND is negotiated, the driver is eligible to suspend the device by setting the SUSPEND bit in \field{device status} to 1, and the device SHOULD set the DRIVER_OK b it to 0 once it has been suspended.
+
+If the device has been suspended, the driver can resume the device running by setting the DRIVER_OK bit in \field{device status} to 1, and the device SHOULD set the SUSPEND bit to 0 once it resumes running.
Quoted 68 lines
Quoted 8 lines
Quoted 7 lines
Quoted 7 lines
content.tex +3
-0+If the device has been suspended and the driver resumes the device +running by setting the DRIVER_OK bit in \field{device status} to 1, +thedevice MUST either resume normal operation or set the
DEVICE_NEEDS_RESETbit in \field{device status} to 1 when it fails toresume.
Quoted 8 lines
Quoted 4 lines
content.tex
Quoted 2 lines
content.tex +2
-0+ +When the driver sets the SUSPEND bit to 1, the device SHOULD performIt should be MUST.
Otherwise it can crash the system accessing memory that may not be anymore accessible.
Not sure how this can crash system memory, but "MUST" is OK
Ok.
Quoted 45 lines
Quoted 34 lines
Quoted 28 lines
+the following actions before presenting the SUSPEND bit as 1 and
DRIVER_OKbit as 0 in the \field{device status}:content.tex +7
-0+ +\begin{itemize} +\item Stop consuming more buffers of any virtqueues. +\item Wait until all buffers that are being processed have been used. +\item Send used buffer notifications to the driver. +\end{itemize} + \chapter{Virtio Transport Options}\label{sec:Virtio TransportOptions}
Virtio can use various different buses, thus the standard is split
@@ -872,6
+923,11 @@ \chapter{Reserved Feature Bits}\label{sec:Reserved Feature Bits}
\ref{devicenormative:Basic Facilities of a Virtio Device / Feature
Bits} forcontent.tex +5
-0handling features reserved for future use. + \item[VIRTIO_F_SUSPEND(43)] This feature indicates that the driver can + suspend the device by set the SUSPEND bit to 1. + See \ref{sec:Basic Facilities of a Virtio Device / Device Status Field}. + + \end{description} \drivernormative{\section}{Reserved Feature Bits}{Reserved FeatureQuoted 4 lines
Bits}
--
2.49.0Apart from these small comments, please add the line in v3 commit log:
Quoted 2 lines
https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgith
ub.com%2Foasis-tcs%2Fvirtio-spec%2Fissues%2F&data=05%7C02%7Cparav%40nv idia.com%7C0ef2b01694d942339bb308dda4e5365c%7C43083d15727340c1b 7db39ef d9ccc17a%7C0%7C0%7C638848026619639978%7CUnknown%7CTWFpbGZsb 3d8eyJFbXB0 eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpb CIsIl dUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdata=0st2osYsAxascHHmHj0RgSyYAn9l yQRKqxOl
Quoted 5 lines
Quoted 3 lines
aIjE8dk%3D&reserved=0<issue_number>
This will help to link the two sides.
sure
Thanks.
Quoted 51 lines
From: Zhu Lingshan <[email protected]>
Sent: Monday, June 9, 2025 1:31 PMQuoted 34 lines · Parav Pandit
Quoted 34 lines
From: Zhu Lingshan <[email protected]>
Sent: Friday, June 6, 2025 4:01 PMQuoted 14 lines · Parav Pandit
Hi,
Quoted 9 lines
From: Zhu Lingshan <[email protected]>
Sent: Thursday, May 29, 2025 1:54 PMThis commit allows the driver to suspend the device through a new device status bit SUSPEND and resume the device running by re-setting
DRIVER_OKbit in device status.This commit re-orders the device status bits.
I overall like this proposal. Few small comments to fix more below. It is lot cleaner as it eliminates the previous bit flipping complexities.
Quoted 6 lines
Can you please add one or two line on a use case/motivation,
For example,
Presently when an OS is suspended, the driver must fully stop and reset the
device, and on resume the driver must re-initialize the device.
Quoted 4 lines
This can be time consuming task to recreate large number of VQs.
Instead, extend the specification to allow drivers to suspend and resume
the device, without recreating all the resources.
Quoted 1 line
...
Hello Parav
Quoted 8 lines
Quoted 4 lines
Here I think the spec provides mechanisms, not a policy. The user may employ this feature for other tasks other than live migration which requires OS suspending.
Its not about some policy.
Sure user may use the feature in N different use cases.The intent is very simple. For the reviewers to review the patch, why
something is needed should be presented.
Quoted 2 lines
So motivation/usecase/need is worth the entry.
Some reviewers do rely on the commit logs to understand the design intent
Quoted 7 lines
behind it.
Quoted 2 lines
Without the need of the patch, I am not sure why would even one write
and review the spec patch.
Quoted 1 line
Please add known use cases. I am familiar with one that I suggested above.
we can add these descriptions in the cover letter.
Yes, cover letter covers for the series as you will have 3 patches and commit log of individual patch will have one two sentences would be good.
Quoted 73 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Signed-off-by Zhu Lingshan <[email protected]>Signed-off-by Jason Wang <[email protected]>--- content.tex | 74++++++++++++++++++++++++++++++++++++++++++++++-
Quoted 13 lines
Quoted 13 lines
----
Quoted 12 lines
Quoted 12 lines
--
1 file changed, 65 insertions(+), 9 deletions(-)content.tex index d3fc6a4..caef5d0 renamed from content.tex +0
-0100644
content.tex +0
-0@@ -36,19 +36,22 @@ \section{\field{Device Status}Field}\label{sec:Basic Facilities of a Virtio Dev
this bit. For example, under Linux, drivers can be loadable modules.
\end{note}Quoted 10 lines
Quoted 10 lines
Quoted 10 lines
content.tex \section{\field{Device Status} Field}\label{sec:Basic Facilities of a Virtio Dev
Quoted 10 lines
-\item[FAILED (128)] Indicates that something went wrong in the guest,
content.tex +5
-3- and it has given up on the device. This could be an internal - error, or the driver didn't like the device for some reason, or - even a fatal error during device operation. +\item[DRIVER_OK (4)] Indicates that the driver is set up and ready +to + drive the device. + +\item[SUSPEND (16)] When VIRTIO_F_SUSPEND is negotiated,indicates
Quoted 29 lines
Quoted 29 lines
Quoted 5 lines
Quoted 5 lines
content.tex +2
-0+that the + device has been suspended by the driver. \item[FEATURES_OK (8)] Indicates that the driver has acknowledgedall
the
Quoted 7 lines
content.tex \section{\field{Device Status} Field}\label{sec:Basic Facilities of a Virtio Dev
Quoted 7 lines
features it understands, and feature negotiation is complete.
-\item[DRIVER_OK (4)] Indicates that the driver is set up and ready to
content.tex +0
-2- drive the device. - \item[DEVICE_NEEDS_RESET (64)] Indicates that the device hasexperienced
Quoted 10 lines
content.tex \section{\field{Device Status} Field}\label{sec:Basic Facilities of a Virtio Dev · line 55
Quoted 9 lines
an error from which it can't recover.
content.tex +6
-0+ +\item[FAILED (128)] Indicates that something went wrong in the +guest, + and it has given up on the device. This could be an internal + error, or the driver didn't like the device for some reason, or + even a fatal error during device operation. \end{description}A new bit SUSPEND introduction should be done at the end without re-
organizing rest of the bits.
Quoted 4 lines
Please add at the end.
Is the intention to rearrange the fields in ascending order of the bit
Quoted 8 lines
Quoted 8 lines
number?
Quoted 1 line
If yes, its good change too but than it missed re-arranging
FEATURE_OK.oh yes, good catch, will fix
Quoted 4 lines
If you want to re-arrange, please have the pre-patch in this series.
Either way is ok to me,
1. Either as pre-patch for re-arrangement + addition of SUSPEND inmiddle.
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Or
2. adding SUSPEND bit definition at the end
Quoted 8 lines
Quoted 4 lines
Quoted 3 lines
Quoted 1 line
#1 is better to see in ascending order.
I was required to re-arrange these status bits, but this is a trivial change, so lets keep them in a single patch for easier review
Having logically separate patches are always good. Re-arrangement is pure
cosmetic vs this one that adds new functionality.
ok, we can split this patch into a series if this is preferred.
Sounds good.
Quoted 33 lines
Quoted 15 lines
Quoted 15 lines
Quoted 15 lines
Quoted 15 lines
The \field{device status} field starts out as 0, and is
reinitialized to 0 by @@ -
99,10 +102,10 @@ \section{Feature Bits}\label{sec:Basic Facilities of a Virtio Device / Feature B \begin{description}
\item[0 to 23, and 50 to 127] Feature bits for the specific device
type-\item[24 to 41] Feature bits reserved for extensions to the queue and
+\item[24 to 43] Feature bits reserved for extensions to the queue +and
feature negotiation mechanisms, see \ref{sec:Reserved Feature
Bits}Quoted 7 lines
Quoted 5 lines
Quoted 4 lines
Only one feature bit is used by this proposal.
So it should be 24 to 42 and use bit 42.Did you use bit 43 due to existence of
VIRTIO_NET_F_GUEST_RSC6?Yes, bit 42 is taken for
VIRTIO_NET_F_GUEST_RSC6.So the current spec wordings is incorrect 0 to 23, and 50 to 127 Feature bits for the specific device type
current spec missed documenting bit 42.
Quoted 5 lines
We need to says,
0 to 23, 41 to 42 and 50 to 127 Feature bits for the specific device type This should also be pre-patch, so that it can be backported to master
branch.
Quoted 2 lines
Another reason for the small patch to be separate.
Quoted 8 lines
Quoted 2 lines
If you want me to write up this pre-patch, please let me know. You can include in the series.
There could a a separate patch in a series says:
41 to 42 Feature bits reserved for extensions to the queue and feature negotiation mechanisms
43 to 49, and 128 and above Feature bits reserved for future extensions.
41-42 are used by the device specific types. So they are not reserved. They are in_use by device specific type. It should be worded accordingly.
Quoted 40 lines
which fixes commit 9438414
Quoted 27 lines
Quoted 27 lines
Quoted 15 lines
Quoted 15 lines
-\item[42 to 49, and 128 and above] Feature bits reserved for future extensions.
+\item[44 to 49, and 128 and above] Feature bits reserved for +future
extensions.
\end{description}\begin{note}
content.tex +0
-0@@ -629,6 +632,54 @@ \section{Device Cleanup}\label{sec:GeneralInitialization And Device Operation /
Thus a driver MUST ensure a virtqueue isn't live (by device reset)
before removing exposed buffers.Quoted 7 lines
Quoted 7 lines
+\section{Device Suspend}\label{sec:General Initialization And
content.tex +4
-0+Device Operation / Device Suspend} + +If VIRTIO_F_SUSPEND is negotiated, the driver is eligible to +suspend thedevice by setting the SUSPEND bit in \field{device status} to 1, and the device SHOULD set the
DRIVER_OKbit to 0 once it has beensuspended.
Quoted 4 lines
Quoted 1 line
+
This is not a normative section so please avoid using SHOULD. Better to word it as informative theory of operation text. Such as,
Quoted 6 lines
Quoted 2 lines
why "SHOULD" can only be used in normative sections?
Because:
1. To clarify the intent:
These keywords carry precise semantic weight. When a spec says MUST, itmeans compliance is not optional. Using such terms in general or explanatory text risks misinterpreting guidance as a requirement.
Quoted 3 lines
2. Avoiding Ambiguity:
If normative language is used in non-normative (informative) sections, it
Quoted 9 lines
becomes unclear whether the reader is required to follow it or if it's just background or suggestion.
Quoted 2 lines
3. Informative sections explain context, rationale, or examples. Normative
sections define rules. Mixing the two muddies the structure and weakens the authority of the normative part.
There are no such rules.
It is not about the rule.
It is the way virtio spec is structured today.
What I explained above : is what happens when one writes theory of operation as normative tone. Description/theory of operation etc sections are written with simple present declarative tone. And requirements are written as normative language tone following the RFC 1.3.
So please avoid normative wordings in description.
Quoted 16 lines
In the spec section 1.3 Terminology, it says:
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED",
"MAY", and "OPTIONAL" in this document are to be interpreted as described in [RFC2119] and [RFC8174] when, and only when, they appear in all capitals, as shown here.RFC 2119 says:
SHOULD This word, or the adjective "RECOMMENDED", mean that there
may exist valid reasons in particular circumstances to ignore a particular item, but the full implications must be understood and carefully weighed before choosing a different course.
Quoted 9 lines
Here this "SHOULD" exactly conforms to the definition.
"SHOULD" has already been used in non-normative sections, for example:
2.5.4 Legacy Interfaces: when using the legacy interface, drivers SHOULD read these fields multiple times until two reads generate a consistent result.The spec does not say: "SHOULD" can only be used in driver or device requirements section.
It follows the style to not have normative words in descriptions as I explained above.
Quoted 16 lines
Quoted 10 lines
Quoted 10 lines
Quoted 10 lines
If X is not negotiated, the....
And the device sets theDRIVER_OKbit to 0 once it has been suspend.Quoted 5 lines
content.tex +2
-0+If the device has been suspended, the driver can resume the device +runningby setting the
DRIVER_OKbit in \field{device status} to 1, and the device should set the SUSPEND bit to 0 once it resumes running. +The device sets the SUSPEND bit to 0 once the device is running.
Quoted 5 lines
Quoted 2 lines
What is the difference in semantic?
"resumes running" does not read correct.
And normative "should" is used...
Hence the suggestion to re-word it.
Quoted 11 lines
Looking into a dictionary
https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww
.merriam- webster.com%2Fdictionary%2Fresume&data=05%7C02%7Cparav%40nvidia.c om%7C2676f0bbbacd46d2944608dda72bc149%7C43083d15727340c1b7db3 9efd9ccc17a%7C0%7C0%7C638850528606897234%7CUnknown%7CTWFpbG Zsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiI sIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdata=FftxcTQOXxj hvCNeIs3%2FHzphS5Q1DVUn2VjLJAt75oA%3D&reserved=0
There is quite a lot of examples of resume + noun.
Running is not a good noun in this context.
It can be worded better which clarifies things more crisply as proposed.
Quoted 2 lines
And here, "SHOULD" still conforms to RFC 2119.
It does not belong in descriptive section. It belongs to requirements section.
Quoted 76 lines
Quoted 34 lines
Quoted 34 lines
Quoted 15 lines
Quoted 11 lines
+\drivernormative{\subsection}{Device Suspend}{General
content.tex +3
-0+Initialization And Device Operation / Device Suspend} + +The driver SHOULD NOT set SUSPEND bit if DRIVER_OK is not set or
VIRTIO_F_SUSPENDis not negotiated.content.tex +5
-0+ +Once the driver sets SUSPEND bit in \field{device status} to 1: +\begin{itemize} +\item The driver MUST verify whether the device has been suspended +byQuoted 4 lines
re-reading \field{device status}, examining whether the SUSPEND bit is set to
1 and theDRIVER_OKbit is set to 0.
+\item The driver MUST NOT make any more buffers available to thedevice.
Quoted 4 lines
Quoted 4 lines
+\item The driver MUST NOT send notifications for any virtqueues. +\item The driver MUST NOT make any changes to Device Configuration Space except for \field{device status} if it is part of the Configuration
Space.
Quoted 13 lines
Quoted 13 lines
+\end{itemize}
content.tex +12
-0+ +\devicenormative{\subsection}{Device Suspend}{General +Initialization And Device Operation / Device Suspend} + +The device MUST ignore any operations on the SUSPEND bit from the +driver if the device has not been completely initialized by the +procedures in \ref{sec:General Initialization And Device Operation +/ Device Initialization} + +The device SHOULD ignore any write access to its Configuration +Space while suspended, except for \field{device status} if it is +part of theQuoted 8 lines
Quoted 8 lines
Quoted 8 lines
content.tex
Quoted 8 lines
Configuration Space.
content.tex +7
-0+ +A device MUST NOT send any notifications for any virtqueues, +access any virtqueues, or modify any fields in its Configuration +Space while suspended. + +If changes occur in the Configuration Space during suspended +period, the device MUST NOT send any configuration changenotifications.
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
+Instead, the device MUST send the notification when it resumes
running.
Quoted 13 lines
Quoted 13 lines
Quoted 7 lines
Quoted 6 lines
content.tex +4
-0+ +If the driver sets the SUSPEND bit in \field{device status} to 1, +the device MUST either suspend itself or set the +DEVICE_NEEDS_RESET bit in\field{device status} to 1 when it fails to suspend. +
Even though you have the driver normative, it is good to have device
normative for suspend and resume operation:
Quoted 5 lines
Something like,
When the suspend operation completes in the device, the device MUST set SUSPEND bit to 1 and
DRIVERK_OKto 0.Quoted 8 lines
Quoted 7 lines
Quoted 2 lines
When the suspended device is resumed, the device MUST set SUSPEND bit
to 0 and
DRIVER_OKto 1.I think we already have similar descriptions in the beginning of this section
I didn't find it in the normative device section.
+\section{Device Suspend}\label{sec:General Initialization And Device
(quoted diff) +5
-0+Operation / Device Suspend} + +If VIRTIO_F_SUSPEND is negotiated, the driver is eligible to suspend +the device by setting the SUSPEND bit in \field{device status} to 1, +and the device SHOULD set the DRIVER_OK bit to 0 once it has been suspended.
(quoted diff) +2
-0+ +If the device has been suspended, the driver can resume the device running
Quoted 3 lines
by setting the
DRIVER_OKbit in \field{device status} to 1, and the device SHOULD set the SUSPEND bit to 0 once it resumes running.
As I explained, it is not the requirements section of the device. It is the description/theory of operation section. And you gave the good example that creates this confusion of mixing the normative tone in description. :)
Quoted 79 lines
Quoted 41 lines
Quoted 15 lines
Quoted 8 lines
Quoted 8 lines
content.tex +3
-0+If the device has been suspended and the driver resumes the device +running by setting the DRIVER_OK bit in \field{device status} to +1, thedevice MUST either resume normal operation or set the
DEVICE_NEEDS_RESETbit in \field{device status} to 1 when it fails toresume.
Quoted 6 lines
content.tex
Quoted 3 lines
content.tex +3
-0+ +When the driver sets the SUSPEND bit to 1, the device SHOULD +performIt should be MUST.
Otherwise it can crash the system accessing memory that may not be anyQuoted 4 lines
more accessible.
Not sure how this can crash system memory, but "MUST" is OK
Ok.
Quoted 21 lines
Quoted 21 lines
Quoted 21 lines
+the following actions before presenting the SUSPEND bit as 1 and
DRIVER_OKbit as 0 in the \field{device status}:content.tex +7
-0+ +\begin{itemize} +\item Stop consuming more buffers of any virtqueues. +\item Wait until all buffers that are being processed have been used. +\item Send used buffer notifications to the driver. +\end{itemize} + \chapter{Virtio Transport Options}\label{sec:Virtio TransportOptions}
Virtio can use various different buses, thus the standard is split
@@ -872,6
+923,11 @@ \chapter{Reserved Feature Bits}\label{sec:Reserved +Feature
Bits}
\ref{devicenormative:Basic Facilities of a Virtio Device /
Feature Bits} for
handling features reserved for future use.Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
+ \item[VIRTIO_F_SUSPEND(43)] This feature indicates that the driver
can
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
content.tex +4
-0+ suspend the device by set the SUSPEND bit to 1. + See \ref{sec:Basic Facilities of a Virtio Device / Device Status Field}. + + \end{description}Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 5 lines
\drivernormative{\section}{Reserved Feature Bits}{Reserved Feature
Bits}
--
2.49.0Apart from these small comments, please add the line in v3 commit log:
https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgi
Quoted 1 line
Quoted 1 line
Quoted 1 line
th%2F&data=05%7C02%7Cparav%40nvidia.com%7C2676f0bbbacd46d29446 08dda7
Quoted 1 line
Quoted 1 line
Quoted 1 line
2bc149%7C43083d15727340c1b7db39efd9ccc17a%7C0%7C0%7C6388505286 069551
Quoted 1 line
Quoted 1 line
Quoted 1 line
99%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjA uMDAwM
Quoted 1 line
Quoted 1 line
Quoted 1 line
CIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C& sda
Quoted 1 line
Quoted 1 line
Quoted 1 line
ta=JM7RqfmpPNS3eVrlq0ky93nh6NGLhKsm97QZkSCnq%2B4%3D&reserved= 0
Quoted 3 lines
Quoted 3 lines
Quoted 1 line
ub.com%2Foasis-tcs%2Fvirtio-
spec%2Fissues%2F&data=05%7C02%7Cparav%40nv
idia.com%7C0ef2b01694d942339bb308dda4e5365c%7C43083d15727340c1b
Quoted 2 lines
Quoted 2 lines
7db39ef
Quoted 15 lines
d9ccc17a%7C0%7C0%7C638848026619639978%7CUnknown%7CTWFpbGZsb
Quoted 2 lines
Quoted 2 lines
3d8eyJFbXB0
eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpb
Quoted 2 lines
Quoted 2 lines
CIsIl
dUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdata=0st2osYsAxascHHmHj0RgSyYAn9l
Quoted 7 lines
Quoted 6 lines
yQRKqxOl
Quoted 3 lines
aIjE8dk%3D&reserved=0<issue_number>
This will help to link the two sides.
sure
Thanks.
Please send v3 with these short fixes and 2 pre-patch.
For fixing feature bits if you want me to write up the patch, please let me know.