Introduce device group and device management

Add conformance clauses for admin commands and admin virtqueues.

Signed-off-by Michael S. Tsirkin <[email protected]>

---
 admin.tex | 216 +++++++++++++++++++++++++++++++++++++++++++++++++++++-
 1 file changed, 215 insertions(+), 1 deletion(-)

admin.tex +17 -0

index 1172054..6c4f79c 100644
@@ -251,6 +251,145 @@ \subsection{Group administration commands}\label{sec:Basic Facilities of a Virti
 supporting multiple group types, the list of supported commands
 might differ between different group types.
 
+\devicenormative{\paragraph}{Group administration commands}{Basic Facilities of a Virtio Device / Device groups / Group administration commands}
+
+The device MUST validate \field{opcode}, \field{group_type} and
+\field{group_member_id}, and if any of these has an invalid or
+unsupported value, set \field{status} to
+VIRTIO_ADMIN_STATUS_EINVAL and set \field{status_qualifier}
+accordingly:
+\begin{itemize}
+\item if \field{group_type} is invalid, \field{status_qualifier}
+	MUST be set to VIRTIO_ADMIN_STATUS_Q_INVALID_GROUP;
+\item otherwise, if \field{opcode} is invalid,
+	\field{status_qualifier} MUST be set to
+	VIRTIO_ADMIN_STATUS_Q_INVALID_OPCODE;
+\item otherwise, if \field{group_member_id} is used by the
+	specific command and is invalid, \field{status_qualifier} MUST be
+	set to VIRTIO_ADMIN_STATUS_Q_INVALID_MEMBER.
+\end{itemize}

admin.tex +18 -0

+enter an error state as a result of this command.
+
+If a command fails, the device state generally SHOULD NOT change,
+as far as possible.
+
+The device MAY enforce additional restrictions and dependencies on
+opcodes used by the driver and MAY fail the command
+VIRTIO_ADMIN_CMD_LIST_USE with \field{status} set to VIRTIO_ADMIN_STATUS_EINVAL
+and \field{status_qualifier} set to VIRTIO_ADMIN_STATUS_Q_INVALID_FIELD
+if the list of commands used violate internal device dependencies.
+
+If the device supports multiple group types, commands for each group
+type MUST operate independently of each other, in particular,
+the device MAY return different results for VIRTIO_ADMIN_CMD_LIST_QUERY
+for different group types.
+
+After reset, if the device supports a given group type
+and before receiving VIRTIO_ADMIN_CMD_LIST_USE for this group type

admin.tex +52 -0

+The device MUST validate commands against the list used by
+the driver and MUST fail any commands not in the list with
+\field{status} set to VIRTIO_ADMIN_STATUS_EINVAL
+and \field{status_qualifier} set to
+VIRTIO_ADMIN_STATUS_Q_INVALID_OPCODE.
+
+The list of supported commands MUST NOT shrink (but MAY expand):
+after reporting a given command as supported through
+VIRTIO_ADMIN_CMD_LIST_QUERY the device MUST NOT later report it
+as unsupported.  Further, after a given set of commands has been
+used (via a successful VIRTIO_ADMIN_CMD_LIST_USE), then after a
+device or system reset the device SHOULD complete successfully
+any following calls to VIRTIO_ADMIN_CMD_LIST_USE with the same
+list of commands; if this command VIRTIO_ADMIN_CMD_LIST_USE fails
+after a device or system reset, the device MUST not fail it
+solely because of the command list used.  Failure to do so would
+interfere with resuming from suspend and error recovery.
+
+When processing a command with the SR-IOV group type,
+if the device does not have an SR-IOV Extended Capability or
+if \field{VF Enable} is clear
+then the device MUST fail all commands with
+\field{status} set to VIRTIO_ADMIN_STATUS_EINVAL and
+\field{status_qualifier} set to
+VIRTIO_ADMIN_STATUS_Q_INVALID_GROUP;
+otherwise, if \field{group_member_id} is not
+between $1$ and \field{NumVFs} inclusive,
+the device MUST fail all commands with
+\field{status} set to VIRTIO_ADMIN_STATUS_EINVAL and
+\field{status_qualifier} set to
+VIRTIO_ADMIN_STATUS_Q_INVALID_MEMBER;
+\field{NumVFs}, \field{VF Migration Capable}  and
+\field{VF Enable} refer to registers within the SR-IOV Extended
+Capability as specified by \hyperref[intro:PCIe]{[PCIe]}.
+
+\drivernormative{\paragraph}{Group administration commands}{Basic Facilities of a Virtio Device / Device groups / Group administration commands}
+
+The driver MAY discover whether device supports a specific group type
+by issuing VIRTIO_ADMIN_CMD_LIST_QUERY with the matching
+\field{group_type}.
+
+The driver MUST issue VIRTIO_ADMIN_CMD_LIST_USE
+and wait for it to be completed with status
+VIRTIO_ADMIN_STATUS_OK before issuing any commands
+(except for the initial VIRTIO_ADMIN_CMD_LIST_QUERY
+and VIRTIO_ADMIN_CMD_LIST_USE).
+
+The driver SHOULD NOT set bits in device_admin_cmds
+if it is not familiar with how the command opcode
+is used, as dependencies between command opcodes might exist.
+
+The driver MUST NOT request (via VIRTIO_ADMIN_CMD_LIST_USE)

admin.tex +20 -0

+The driver MUST handle a command error with a reserved \field{status}
+value in the same way as \field{status} set to VIRTIO_ADMIN_STATUS_EINVAL
+(except possibly for different error reporting/diagnostic messages).
+
+The driver MUST handle a command error with a reserved
+\field{status_qualifier} value in the same way as
+\field{status_qualifier} set to
+VIRTIO_ADMIN_STATUS_Q_INVALID_COMMAND (except possibly for
+different error reporting/diagnostic messages).
+
+When sending commands with the SR-IOV group type,
+the driver specify a value for \field{group_member_id}
+between $1$ and \field{NumVFs} inclusive,
+the driver MUST also make sure that as long as any such command
+is outstanding, \field{VF Migration Capable} is clear and
+\field{VF Enable} is set;
+\field{NumVFs}, \field{VF Migration Capable}  and
+\field{VF Enable} refer to registers within the SR-IOV Extended
+Capability as specified by \hyperref[intro:PCIe]{[PCIe]}.
+
 \section{Administration Virtqueues}\label{sec:Basic Facilities of a Virtio Device / Administration Virtqueues}
 
 An administration virtqueue of an owner device is used to submit

admin.tex +50 -0

+
+\devicenormative{\paragraph}{Group administration commands}{Basic Facilities of a Virtio Device / Administration virtqueues}
+
+The device MUST support device-readable and device-writeable buffers
+shorter than described in this specification, by
+\begin{enumerate}
+\item acting as if any data that would be read outside the
+device-readable buffers is set to zero, and
+\item discarding data that would be written outside the
+specified device-writeable buffers.
+\end{enumerate}
+
+The device MUST support device-readable and device-writeable buffers
+longer than described in this specification, by
+\begin{enumerate}
+\item ignoring any data in device-readable buffers outside
+the expected length, and
+\item only writing the expected structure to the device-writeable
+buffers, ignoring any extra buffers, and reporting the
+actual length of data written, in bytes,
+as buffer used length.
+\end{enumerate}
+
+The device SHOULD initialize the device-writeable buffer
+up to the length of the structure described by this specification or
+the length of the buffer supplied by the driver (even if the buffer is
+all set to zero), whichever is shorter.
+
+The device MUST NOT fail a command solely because the buffers
+provided are shorter or longer than described in this
+specification.
+
+The device MUST initialize the device-writeable part of
+\field{struct virtio_admin_cmd} that is a multiple of 64 bit in
+size.
+
+The device MUST initialize \field{status} in \field{struct
+virtio_admin_cmd}.
+
+The device MUST process commands on a given administration virtqueue
+in the order in which they are queued.
+
+If multiple administration virtqueues have been configured,
+device MAY process commands on distinct virtqueues with
+no order constraints.
+
+\drivernormative{\paragraph}{Group administration commands}{Basic Facilities of a Virtio Device / Administration virtqueues}
+
+The driver MAY supply device-readable or device-writeable parts
+of \field{struct virtio_admin_cmd} that are longer than described in

admin.tex +15 -0

+The device MUST supply both device-readable or device-writeable parts
+of \field{struct virtio_admin_cmd} that are larger than zero in
+length. However, \field{command_specific_data} and
+\field{command_specific_result} MAY be zero in length, unless
+specified otherwise for the command.
+
+The driver MUST NOT assume that the device will initialize the whole
+device-writeable part of \field{struct virtio_admin_cmd} as described in the specification; instead,
+the driver MUST act as if the structure
+outside the part of the buffer used by the device
+is set to zero.
+
+If multiple administration virtqueues have been configured,
+the driver MUST ensure ordering for commands
+placed on different administration virtqueues.

Signature · 18 lines

MST


This publicly archived list offers a means to provide input to the
OASIS Virtual I/O Device (VIRTIO) TC.

In order to verify user consent to the Feedback License terms and
to minimize spam in the list archive, subscription is required
before posting.

Subscribe: [email protected]
Unsubscribe: [email protected]
List help: [email protected]
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/

Quoted 36 lines · Michael S. Tsirkin

Add conformance clauses for admin commands and admin virtqueues.

Signed-off-by Michael S. Tsirkin <[email protected]>

---
 admin.tex | 216 +++++++++++++++++++++++++++++++++++++++++++++++++++++-
 1 file changed, 215 insertions(+), 1 deletion(-)

admin.tex +21 -0

index 1172054..6c4f79c 100644
@@ -251,6 +251,145 @@ \subsection{Group administration commands}\label{sec:Basic Facilities of a Virti
 supporting multiple group types, the list of supported commands
 might differ between different group types.
 
+\devicenormative{\paragraph}{Group administration commands}{Basic Facilities of a Virtio Device / Device groups / Group administration commands}
+
+The device MUST validate \field{opcode}, \field{group_type} and
+\field{group_member_id}, and if any of these has an invalid or
+unsupported value, set \field{status} to
+VIRTIO_ADMIN_STATUS_EINVAL and set \field{status_qualifier}
+accordingly:
+\begin{itemize}
+\item if \field{group_type} is invalid, \field{status_qualifier}
+	MUST be set to VIRTIO_ADMIN_STATUS_Q_INVALID_GROUP;
+\item otherwise, if \field{opcode} is invalid,
+	\field{status_qualifier} MUST be set to
+	VIRTIO_ADMIN_STATUS_Q_INVALID_OPCODE;
+\item otherwise, if \field{group_member_id} is used by the
+	specific command and is invalid, \field{status_qualifier} MUST be
+	set to VIRTIO_ADMIN_STATUS_Q_INVALID_MEMBER.
+\end{itemize}
+
+If a command completes successfully, the device MUST set
+\field{status} to VIRTIO_ADMIN_STATUS_OK.
+

admin.tex \subsection{Group administration commands}\label{sec:Basic Facilities of a Virti · line 280

Quoted 6 lines

admin.tex +6 -0

+If a command fails, the device MUST set
+\field{status} to a value different from VIRTIO_ADMIN_STATUS_OK.
+
+If \field{status} is set to VIRTIO_ADMIN_STATUS_EINVAL, the
+device state MUST NOT change, that is the command MUST NOT have
+any side effects on the device, in particular the device MUST not

MUST NOT

Quoted 22 lines

admin.tex +22 -0

+enter an error state as a result of this command.
+
+If a command fails, the device state generally SHOULD NOT change,
+as far as possible.
+
+The device MAY enforce additional restrictions and dependencies on
+opcodes used by the driver and MAY fail the command
+VIRTIO_ADMIN_CMD_LIST_USE with \field{status} set to VIRTIO_ADMIN_STATUS_EINVAL
+and \field{status_qualifier} set to VIRTIO_ADMIN_STATUS_Q_INVALID_FIELD
+if the list of commands used violate internal device dependencies.
+
+If the device supports multiple group types, commands for each group
+type MUST operate independently of each other, in particular,
+the device MAY return different results for VIRTIO_ADMIN_CMD_LIST_QUERY
+for different group types.
+
+After reset, if the device supports a given group type
+and before receiving VIRTIO_ADMIN_CMD_LIST_USE for this group type
+the device MUST assume
+that the list of legal commands used by the driver consists of
+the two commands VIRTIO_ADMIN_CMD_LIST_QUERY and VIRTIO_ADMIN_CMD_LIST_USE.
+

admin.tex \subsection{Group administration commands}\label{sec:Basic Facilities of a Virti · line 308

Quoted 6 lines

admin.tex +6 -0

+After completing VIRTIO_ADMIN_CMD_LIST_USE successfully,
+the device MUST set the list of legal commands used by the driver
+to the one supplied in \field{command_specific_data}.
+
+The device MUST set the list of legal commands used by the driver
+to the one supplied in VIRTIO_ADMIN_CMD_LIST_USE.

Are these last two paragraphs not saying the same thing?

Quoted 59 lines

admin.tex +59 -0

+The device MUST validate commands against the list used by
+the driver and MUST fail any commands not in the list with
+\field{status} set to VIRTIO_ADMIN_STATUS_EINVAL
+and \field{status_qualifier} set to
+VIRTIO_ADMIN_STATUS_Q_INVALID_OPCODE.
+
+The list of supported commands MUST NOT shrink (but MAY expand):
+after reporting a given command as supported through
+VIRTIO_ADMIN_CMD_LIST_QUERY the device MUST NOT later report it
+as unsupported.  Further, after a given set of commands has been
+used (via a successful VIRTIO_ADMIN_CMD_LIST_USE), then after a
+device or system reset the device SHOULD complete successfully
+any following calls to VIRTIO_ADMIN_CMD_LIST_USE with the same
+list of commands; if this command VIRTIO_ADMIN_CMD_LIST_USE fails
+after a device or system reset, the device MUST not fail it
+solely because of the command list used.  Failure to do so would
+interfere with resuming from suspend and error recovery.
+
+When processing a command with the SR-IOV group type,
+if the device does not have an SR-IOV Extended Capability or
+if \field{VF Enable} is clear
+then the device MUST fail all commands with
+\field{status} set to VIRTIO_ADMIN_STATUS_EINVAL and
+\field{status_qualifier} set to
+VIRTIO_ADMIN_STATUS_Q_INVALID_GROUP;
+otherwise, if \field{group_member_id} is not
+between $1$ and \field{NumVFs} inclusive,
+the device MUST fail all commands with
+\field{status} set to VIRTIO_ADMIN_STATUS_EINVAL and
+\field{status_qualifier} set to
+VIRTIO_ADMIN_STATUS_Q_INVALID_MEMBER;
+\field{NumVFs}, \field{VF Migration Capable}  and
+\field{VF Enable} refer to registers within the SR-IOV Extended
+Capability as specified by \hyperref[intro:PCIe]{[PCIe]}.
+
+\drivernormative{\paragraph}{Group administration commands}{Basic Facilities of a Virtio Device / Device groups / Group administration commands}
+
+The driver MAY discover whether device supports a specific group type
+by issuing VIRTIO_ADMIN_CMD_LIST_QUERY with the matching
+\field{group_type}.
+
+The driver MUST issue VIRTIO_ADMIN_CMD_LIST_USE
+and wait for it to be completed with status
+VIRTIO_ADMIN_STATUS_OK before issuing any commands
+(except for the initial VIRTIO_ADMIN_CMD_LIST_QUERY
+and VIRTIO_ADMIN_CMD_LIST_USE).
+
+The driver SHOULD NOT set bits in device_admin_cmds
+if it is not familiar with how the command opcode
+is used, as dependencies between command opcodes might exist.
+
+The driver MUST NOT request (via VIRTIO_ADMIN_CMD_LIST_USE)
+the use of any commands not previously reported as
+supported for the same group type by VIRTIO_ADMIN_CMD_LIST_QUERY.
+
+The driver MUST NOT use any commands for a given group type
+before sending VIRTIO_ADMIN_CMD_LIST_USE with the correct
+list of command opcodes and group type.
+

admin.tex \subsection{Group administration commands}\label{sec:Basic Facilities of a Virti · line 371

Quoted 3 lines

admin.tex +3 -0

+The driver MAY block use of VIRTIO_ADMIN_CMD_LIST_QUERY and
+VIRTIO_ADMIN_CMD_LIST_USE by issuing VIRTIO_ADMIN_CMD_LIST_USE
+with respective bits cleared in \field{command_specific_data}.

This runs contrary to the assertion "The list of supported commands MUST NOT shrink", given that a driver is told to assume that VIRTIO_ADMIN_CMD_LIST_QUERY and VIRTIO_ADMIN_CMD_LIST_USE are the only commands initially available.

Quoted 23 lines

admin.tex +20 -0

+The driver MUST handle a command error with a reserved \field{status}
+value in the same way as \field{status} set to VIRTIO_ADMIN_STATUS_EINVAL
+(except possibly for different error reporting/diagnostic messages).
+
+The driver MUST handle a command error with a reserved
+\field{status_qualifier} value in the same way as
+\field{status_qualifier} set to
+VIRTIO_ADMIN_STATUS_Q_INVALID_COMMAND (except possibly for
+different error reporting/diagnostic messages).
+
+When sending commands with the SR-IOV group type,
+the driver specify a value for \field{group_member_id}
+between $1$ and \field{NumVFs} inclusive,
+the driver MUST also make sure that as long as any such command
+is outstanding, \field{VF Migration Capable} is clear and
+\field{VF Enable} is set;
+\field{NumVFs}, \field{VF Migration Capable}  and
+\field{VF Enable} refer to registers within the SR-IOV Extended
+Capability as specified by \hyperref[intro:PCIe]{[PCIe]}.
+
 \section{Administration Virtqueues}\label{sec:Basic Facilities of a Virtio Device / Administration Virtqueues}
 

admin.tex \section{Administration Virtqueues}\label{sec:Basic Facilities of a Virtio Devic · line 326

Quoted 6 lines

admin.tex +0 -1

 An administration virtqueue of an owner device is used to submit

@@ -323,4 +462,79 @@ \section{Administration Virtqueues}\label{sec:Basic Facilities of a Virtio Devic
 of the specification are designed, new fields can be added to the
 tail of a structure, with the driver/device using the full
 structure without concern for versioning.
->>>>>>> 0edc690... admin: introduce virtio admin virtqueues

Oh, here is where it got removed :-)

Quoted 50 lines

admin.tex +50 -0

+
+\devicenormative{\paragraph}{Group administration commands}{Basic Facilities of a Virtio Device / Administration virtqueues}
+
+The device MUST support device-readable and device-writeable buffers
+shorter than described in this specification, by
+\begin{enumerate}
+\item acting as if any data that would be read outside the
+device-readable buffers is set to zero, and
+\item discarding data that would be written outside the
+specified device-writeable buffers.
+\end{enumerate}
+
+The device MUST support device-readable and device-writeable buffers
+longer than described in this specification, by
+\begin{enumerate}
+\item ignoring any data in device-readable buffers outside
+the expected length, and
+\item only writing the expected structure to the device-writeable
+buffers, ignoring any extra buffers, and reporting the
+actual length of data written, in bytes,
+as buffer used length.
+\end{enumerate}
+
+The device SHOULD initialize the device-writeable buffer
+up to the length of the structure described by this specification or
+the length of the buffer supplied by the driver (even if the buffer is
+all set to zero), whichever is shorter.
+
+The device MUST NOT fail a command solely because the buffers
+provided are shorter or longer than described in this
+specification.
+
+The device MUST initialize the device-writeable part of
+\field{struct virtio_admin_cmd} that is a multiple of 64 bit in
+size.
+
+The device MUST initialize \field{status} in \field{struct
+virtio_admin_cmd}.
+
+The device MUST process commands on a given administration virtqueue
+in the order in which they are queued.
+
+If multiple administration virtqueues have been configured,
+device MAY process commands on distinct virtqueues with
+no order constraints.
+
+\drivernormative{\paragraph}{Group administration commands}{Basic Facilities of a Virtio Device / Administration virtqueues}
+
+The driver MAY supply device-readable or device-writeable parts
+of \field{struct virtio_admin_cmd} that are longer than described in

admin.tex \section{Administration Virtqueues}\label{sec:Basic Facilities of a Virtio Devic · line 524

Quoted 10 lines

admin.tex +10 -0

+this specification.
+
+The driver SHOULD supply device-readable part of
+\field{struct virtio_admin_cmd} that is at least as
+large as the structure described by this specification
+(even if the structure is all set to zero).
+
+The driver MUST supply both device-readable or device-writeable parts
+of \field{struct virtio_admin_cmd} that are a multiple of 64 bit
+in length.

This addresses my question on a previous patch.

Quoted 26 lines

admin.tex +15 -0

+The device MUST supply both device-readable or device-writeable parts
+of \field{struct virtio_admin_cmd} that are larger than zero in
+length. However, \field{command_specific_data} and
+\field{command_specific_result} MAY be zero in length, unless
+specified otherwise for the command.
+
+The driver MUST NOT assume that the device will initialize the whole
+device-writeable part of \field{struct virtio_admin_cmd} as described in the specification; instead,
+the driver MUST act as if the structure
+outside the part of the buffer used by the device
+is set to zero.
+
+If multiple administration virtqueues have been configured,
+the driver MUST ensure ordering for commands
+placed on different administration virtqueues.

Signature · 9 lines

MST


This publicly archived list offers a means to provide input to the
OASIS Virtual I/O Device (VIRTIO) TC.

In order to verify user consent to the Feedback License terms and
to minimize spam in the list archive, subscription is required
before posting.

Quoted 8 lines

Subscribe: [email protected]
Unsubscribe: [email protected]
List help: [email protected]
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/

Signature · 1 line

Music has magic, it's good clear syncopation.

Quoted 9 lines · David Edmondson

admin.tex \subsection{Group administration commands}\label{sec:Basic Facilities of a Virti · line 371

Quoted 3 lines · Michael S. Tsirkin

admin.tex +3 -0

+The driver MAY block use of VIRTIO_ADMIN_CMD_LIST_QUERY and
+VIRTIO_ADMIN_CMD_LIST_USE by issuing VIRTIO_ADMIN_CMD_LIST_USE
+with respective bits cleared in \field{command_specific_data}.

This runs contrary to the assertion "The list of supported commands MUST NOT shrink", given that a driver is told to assume that VIRTIO_ADMIN_CMD_LIST_QUERY and VIRTIO_ADMIN_CMD_LIST_USE are the only commands initially available.

Commands are still available just disabled, the meaning of "MUST NOT shrink" is clarified by the following:

> > +after reporting a given command as supported through > > +VIRTIO_ADMIN_CMD_LIST_QUERY the device MUST NOT later report it > > +as unsupported.
I will stick an "i.e." there to make it hopefully clearer.

Quoted 14 lines · Michael S. Tsirkin

Quoted 9 lines · David Edmondson

admin.tex \subsection{Group administration commands}\label{sec:Basic Facilities of a Virti · line 371

Quoted 3 lines · Michael S. Tsirkin

admin.tex +3 -0

+The driver MAY block use of VIRTIO_ADMIN_CMD_LIST_QUERY and
+VIRTIO_ADMIN_CMD_LIST_USE by issuing VIRTIO_ADMIN_CMD_LIST_USE
+with respective bits cleared in \field{command_specific_data}.

This runs contrary to the assertion "The list of supported commands MUST NOT shrink", given that a driver is told to assume that VIRTIO_ADMIN_CMD_LIST_QUERY and VIRTIO_ADMIN_CMD_LIST_USE are the only commands initially available.

Commands are still available just disabled, the meaning of "MUST NOT shrink" is clarified by the following:

Quoted 4 lines

> > +after reporting a given command as supported through > > +VIRTIO_ADMIN_CMD_LIST_QUERY the device MUST NOT later report it > > +as unsupported.
I will stick an "i.e." there to make it hopefully clearer.

This puzzles me.

I can see the point of blocking LIST_USE, but not of blocking

LIST_QUERY. What's the purpose of this?

Signature · 6 lines

Time is waiting to explain, why refuse?

---------------------------------------------------------------------
To unsubscribe from this mail list, you must leave the OASIS TC that 
generates this mail.  Follow this link to all your TCs in OASIS at:
https://www.oasis-open.org/apps/org/workgroup/portal/my_workgroups.php 

Quoted 16 lines · Wed, Mar 08, 2023 at 01:59:10PM CET, [email protected] wrote:

Quoted 14 lines · Michael S. Tsirkin

Quoted 9 lines · David Edmondson

admin.tex \subsection{Group administration commands}\label{sec:Basic Facilities of a Virti · line 371

Quoted 3 lines · Michael S. Tsirkin

admin.tex +3 -0

+The driver MAY block use of VIRTIO_ADMIN_CMD_LIST_QUERY and
+VIRTIO_ADMIN_CMD_LIST_USE by issuing VIRTIO_ADMIN_CMD_LIST_USE
+with respective bits cleared in \field{command_specific_data}.

This runs contrary to the assertion "The list of supported commands MUST NOT shrink", given that a driver is told to assume that VIRTIO_ADMIN_CMD_LIST_QUERY and VIRTIO_ADMIN_CMD_LIST_USE are the only commands initially available.

Commands are still available just disabled, the meaning of "MUST NOT shrink" is clarified by the following:

Quoted 8 lines

Quoted 4 lines

> > +after reporting a given command as supported through > > +VIRTIO_ADMIN_CMD_LIST_QUERY the device MUST NOT later report it > > +as unsupported.
I will stick an "i.e." there to make it hopefully clearer.

This puzzles me.

I can see the point of blocking LIST_USE, but not of blocking

What is that point, I'm curious.

Quoted 1 line

LIST_QUERY. What's the purpose of this?

I think that this might be just for the sake of treating all commands equally.

Quoted 8 lines

Signature · 6 lines

Time is waiting to explain, why refuse?

---------------------------------------------------------------------
To unsubscribe from this mail list, you must leave the OASIS TC that 
generates this mail.  Follow this link to all your TCs in OASIS at:
https://www.oasis-open.org/apps/org/workgroup/portal/my_workgroups.php 

Quoted 21 lines · Wed, Mar 08, 2023 at 01:59:10PM CET, [email protected] wrote:

Quoted 18 lines · Michael S. Tsirkin

Quoted 9 lines · David Edmondson

admin.tex \subsection{Group administration commands}\label{sec:Basic Facilities of a Virti · line 371

Quoted 3 lines · Michael S. Tsirkin

admin.tex +3 -0

+The driver MAY block use of VIRTIO_ADMIN_CMD_LIST_QUERY and
+VIRTIO_ADMIN_CMD_LIST_USE by issuing VIRTIO_ADMIN_CMD_LIST_USE
+with respective bits cleared in \field{command_specific_data}.

This runs contrary to the assertion "The list of supported commands MUST NOT shrink", given that a driver is told to assume that VIRTIO_ADMIN_CMD_LIST_QUERY and VIRTIO_ADMIN_CMD_LIST_USE are the only commands initially available.

Commands are still available just disabled, the meaning of "MUST NOT shrink" is clarified by the following:

> > +after reporting a given command as supported through > > +VIRTIO_ADMIN_CMD_LIST_QUERY the device MUST NOT later report it > > +as unsupported.
I will stick an "i.e." there to make it hopefully clearer.

Quoted 3 lines

This puzzles me.

I can see the point of blocking LIST_USE, but not of blocking

What is that point, I'm curious.

Quoted 2 lines

LIST_QUERY. What's the purpose of this?

I think that this might be just for the sake of treating all commands equally.

Quoted 8 lines

Signature · 6 lines

Time is waiting to explain, why refuse?

---------------------------------------------------------------------
To unsubscribe from this mail list, you must leave the OASIS TC that 
generates this mail.  Follow this link to all your TCs in OASIS at:
https://www.oasis-open.org/apps/org/workgroup/portal/my_workgroups.php 

Quoted 22 lines · Jiri Pirko

Quoted 21 lines · Wed, Mar 08, 2023 at 01:59:10PM CET, [email protected] wrote:

Quoted 18 lines · Michael S. Tsirkin

Quoted 9 lines · David Edmondson

admin.tex \subsection{Group administration commands}\label{sec:Basic Facilities of a Virti · line 371

Quoted 3 lines · Michael S. Tsirkin

admin.tex +3 -0

+The driver MAY block use of VIRTIO_ADMIN_CMD_LIST_QUERY and
+VIRTIO_ADMIN_CMD_LIST_USE by issuing VIRTIO_ADMIN_CMD_LIST_USE
+with respective bits cleared in \field{command_specific_data}.

This runs contrary to the assertion "The list of supported commands MUST NOT shrink", given that a driver is told to assume that VIRTIO_ADMIN_CMD_LIST_QUERY and VIRTIO_ADMIN_CMD_LIST_USE are the only commands initially available.

Commands are still available just disabled, the meaning of "MUST NOT shrink" is clarified by the following:

> > +after reporting a given command as supported through > > +VIRTIO_ADMIN_CMD_LIST_QUERY the device MUST NOT later report it > > +as unsupported.
I will stick an "i.e." there to make it hopefully clearer.

Quoted 5 lines

Quoted 3 lines

This puzzles me.

I can see the point of blocking LIST_USE, but not of blocking

What is that point, I'm curious.

I might restrict the commands permitted by a device before handing it off to some subordinate less trusted code.

Quoted 8 lines

Quoted 3 lines

LIST_QUERY. What's the purpose of this?

I think that this might be just for the sake of treating all commands equally.

Quoted 8 lines

Quoted 8 lines

Signature · 6 lines

Time is waiting to explain, why refuse?

---------------------------------------------------------------------
To unsubscribe from this mail list, you must leave the OASIS TC that 
generates this mail.  Follow this link to all your TCs in OASIS at:
https://www.oasis-open.org/apps/org/workgroup/portal/my_workgroups.php 

Signature · 1 line

I used to get mad at my school, the teachers who taught me weren't cool.

Quoted 26 lines · Wed, Mar 08, 2023 at 02:44:08PM CET, [email protected] wrote:

Quoted 24 lines · Jiri Pirko

Quoted 23 lines · Wed, Mar 08, 2023 at 01:59:10PM CET, [email protected] wrote:

Quoted 11 lines · Michael S. Tsirkin

Quoted 9 lines · David Edmondson

Review

Quoted 3 lines · Michael S. Tsirkin

(quoted diff) +3 -0

+The driver MAY block use of VIRTIO_ADMIN_CMD_LIST_QUERY and
+VIRTIO_ADMIN_CMD_LIST_USE by issuing VIRTIO_ADMIN_CMD_LIST_USE
+with respective bits cleared in \field{command_specific_data}.

This runs contrary to the assertion "The list of supported commands MUST NOT shrink", given that a driver is told to assume that VIRTIO_ADMIN_CMD_LIST_QUERY and VIRTIO_ADMIN_CMD_LIST_USE are the only commands initially available.

Quoted 7 lines

Commands are still available just disabled, the meaning of "MUST NOT shrink" is clarified by the following:

> > +after reporting a given command as supported through > > +VIRTIO_ADMIN_CMD_LIST_QUERY the device MUST NOT later report it > > +as unsupported.
I will stick an "i.e." there to make it hopefully clearer.

This puzzles me.

Quoted 6 lines

Quoted 3 lines

Quoted 1 line

I can see the point of blocking LIST_USE, but not of blocking

What is that point, I'm curious.

I might restrict the commands permitted by a device before handing it off to some subordinate less trusted code.

Hmm, that is theoretical, or is there a usecase for that?

Quoted 11 lines

Quoted 11 lines

Quoted 3 lines

LIST_QUERY. What's the purpose of this?

I think that this might be just for the sake of treating all commands equally.

Quoted 3 lines

Signature · 1 line

Time is waiting to explain, why refuse?

Quoted 7 lines

Quoted 5 lines

Quoted 5 lines

---------------------------------------------------------------------
To unsubscribe from this mail list, you must leave the OASIS TC that generates this mail. Follow this link to all your TCs in OASIS at:
https://www.oasis-open.org/apps/org/workgroup/portal/my_workgroups.php

Signature · 1 line

I used to get mad at my school, the teachers who taught me weren't cool.

Add conformance clauses for admin commands and admin virtqueues.

Signed-off-by Michael S. Tsirkin <[email protected]>

---
  admin.tex | 216 +++++++++++++++++++++++++++++++++++++++++++++++++++++-
  1 file changed, 215 insertions(+), 1 deletion(-)

admin.tex +56 -0

index 1172054..6c4f79c 100644
@@ -251,6 +251,145 @@ \subsection{Group administration commands}\label{sec:Basic Facilities of a Virti
  supporting multiple group types, the list of supported commands
  might differ between different group types.
 
+\devicenormative{\paragraph}{Group administration commands}{Basic Facilities of a Virtio Device / Device groups / Group administration commands}
+
+The device MUST validate \field{opcode}, \field{group_type} and
+\field{group_member_id}, and if any of these has an invalid or
+unsupported value, set \field{status} to
+VIRTIO_ADMIN_STATUS_EINVAL and set \field{status_qualifier}
+accordingly:
+\begin{itemize}
+\item if \field{group_type} is invalid, \field{status_qualifier}
+	MUST be set to VIRTIO_ADMIN_STATUS_Q_INVALID_GROUP;
+\item otherwise, if \field{opcode} is invalid,
+	\field{status_qualifier} MUST be set to
+	VIRTIO_ADMIN_STATUS_Q_INVALID_OPCODE;
+\item otherwise, if \field{group_member_id} is used by the
+	specific command and is invalid, \field{status_qualifier} MUST be
+	set to VIRTIO_ADMIN_STATUS_Q_INVALID_MEMBER.
+\end{itemize}
+
+If a command completes successfully, the device MUST set
+\field{status} to VIRTIO_ADMIN_STATUS_OK.
+
+If a command fails, the device MUST set
+\field{status} to a value different from VIRTIO_ADMIN_STATUS_OK.
+
+If \field{status} is set to VIRTIO_ADMIN_STATUS_EINVAL, the
+device state MUST NOT change, that is the command MUST NOT have
+any side effects on the device, in particular the device MUST not
+enter an error state as a result of this command.
+
+If a command fails, the device state generally SHOULD NOT change,
+as far as possible.
+
+The device MAY enforce additional restrictions and dependencies on
+opcodes used by the driver and MAY fail the command
+VIRTIO_ADMIN_CMD_LIST_USE with \field{status} set to VIRTIO_ADMIN_STATUS_EINVAL
+and \field{status_qualifier} set to VIRTIO_ADMIN_STATUS_Q_INVALID_FIELD
+if the list of commands used violate internal device dependencies.
+
+If the device supports multiple group types, commands for each group
+type MUST operate independently of each other, in particular,
+the device MAY return different results for VIRTIO_ADMIN_CMD_LIST_QUERY
+for different group types.
+
+After reset, if the device supports a given group type
+and before receiving VIRTIO_ADMIN_CMD_LIST_USE for this group type
+the device MUST assume
+that the list of legal commands used by the driver consists of
+the two commands VIRTIO_ADMIN_CMD_LIST_QUERY and VIRTIO_ADMIN_CMD_LIST_USE.
+
+After completing VIRTIO_ADMIN_CMD_LIST_USE successfully,
+the device MUST set the list of legal commands used by the driver
+to the one supplied in \field{command_specific_data}.
+
+The device MUST set the list of legal commands used by the driver
+to the one supplied in VIRTIO_ADMIN_CMD_LIST_USE.
+

admin.tex +135 -1

+after a device or system reset, the device MUST not fail it
+solely because of the command list used.  Failure to do so would
+interfere with resuming from suspend and error recovery.
+
+When processing a command with the SR-IOV group type,
+if the device does not have an SR-IOV Extended Capability or
+if \field{VF Enable} is clear
+then the device MUST fail all commands with
+\field{status} set to VIRTIO_ADMIN_STATUS_EINVAL and
+\field{status_qualifier} set to
+VIRTIO_ADMIN_STATUS_Q_INVALID_GROUP;
+otherwise, if \field{group_member_id} is not
+between $1$ and \field{NumVFs} inclusive,
+the device MUST fail all commands with
+\field{status} set to VIRTIO_ADMIN_STATUS_EINVAL and
+\field{status_qualifier} set to
+VIRTIO_ADMIN_STATUS_Q_INVALID_MEMBER;
+\field{NumVFs}, \field{VF Migration Capable}  and
+\field{VF Enable} refer to registers within the SR-IOV Extended
+Capability as specified by \hyperref[intro:PCIe]{[PCIe]}.
+
+\drivernormative{\paragraph}{Group administration commands}{Basic Facilities of a Virtio Device / Device groups / Group administration commands}
+
+The driver MAY discover whether device supports a specific group type
+by issuing VIRTIO_ADMIN_CMD_LIST_QUERY with the matching
+\field{group_type}.
+
+The driver MUST issue VIRTIO_ADMIN_CMD_LIST_USE
+and wait for it to be completed with status
+VIRTIO_ADMIN_STATUS_OK before issuing any commands
+(except for the initial VIRTIO_ADMIN_CMD_LIST_QUERY
+and VIRTIO_ADMIN_CMD_LIST_USE).
+
+The driver SHOULD NOT set bits in device_admin_cmds
+if it is not familiar with how the command opcode
+is used, as dependencies between command opcodes might exist.
+
+The driver MUST NOT request (via VIRTIO_ADMIN_CMD_LIST_USE)
+the use of any commands not previously reported as
+supported for the same group type by VIRTIO_ADMIN_CMD_LIST_QUERY.
+
+The driver MUST NOT use any commands for a given group type
+before sending VIRTIO_ADMIN_CMD_LIST_USE with the correct
+list of command opcodes and group type.
+
+The driver MAY block use of VIRTIO_ADMIN_CMD_LIST_QUERY and
+VIRTIO_ADMIN_CMD_LIST_USE by issuing VIRTIO_ADMIN_CMD_LIST_USE
+with respective bits cleared in \field{command_specific_data}.
+
+The driver MUST handle a command error with a reserved \field{status}
+value in the same way as \field{status} set to VIRTIO_ADMIN_STATUS_EINVAL
+(except possibly for different error reporting/diagnostic messages).
+
+The driver MUST handle a command error with a reserved
+\field{status_qualifier} value in the same way as
+\field{status_qualifier} set to
+VIRTIO_ADMIN_STATUS_Q_INVALID_COMMAND (except possibly for
+different error reporting/diagnostic messages).
+
+When sending commands with the SR-IOV group type,
+the driver specify a value for \field{group_member_id}
+between $1$ and \field{NumVFs} inclusive,
+the driver MUST also make sure that as long as any such command
+is outstanding, \field{VF Migration Capable} is clear and
+\field{VF Enable} is set;
+\field{NumVFs}, \field{VF Migration Capable}  and
+\field{VF Enable} refer to registers within the SR-IOV Extended
+Capability as specified by \hyperref[intro:PCIe]{[PCIe]}.
+
  \section{Administration Virtqueues}\label{sec:Basic Facilities of a Virtio Device / Administration Virtqueues}
 
  An administration virtqueue of an owner device is used to submit

@@ -323,4 +462,79 @@ \section{Administration Virtqueues}\label{sec:Basic Facilities of a Virtio Devic
  of the specification are designed, new fields can be added to the
  tail of a structure, with the driver/device using the full
  structure without concern for versioning.
->>>>>>> 0edc690... admin: introduce virtio admin virtqueues
+
+\devicenormative{\paragraph}{Group administration commands}{Basic Facilities of a Virtio Device / Administration virtqueues}
+
+The device MUST support device-readable and device-writeable buffers
+shorter than described in this specification, by
+\begin{enumerate}
+\item acting as if any data that would be read outside the
+device-readable buffers is set to zero, and
+\item discarding data that would be written outside the
+specified device-writeable buffers.
+\end{enumerate}
+
+The device MUST support device-readable and device-writeable buffers
+longer than described in this specification, by
+\begin{enumerate}
+\item ignoring any data in device-readable buffers outside
+the expected length, and
+\item only writing the expected structure to the device-writeable
+buffers, ignoring any extra buffers, and reporting the
+actual length of data written, in bytes,
+as buffer used length.
+\end{enumerate}
+
+The device SHOULD initialize the device-writeable buffer
+up to the length of the structure described by this specification or
+the length of the buffer supplied by the driver (even if the buffer is
+all set to zero), whichever is shorter.
+
+The device MUST NOT fail a command solely because the buffers
+provided are shorter or longer than described in this
+specification.
+
+The device MUST initialize the device-writeable part of
+\field{struct virtio_admin_cmd} that is a multiple of 64 bit in
+size.
+
+The device MUST initialize \field{status} in \field{struct
+virtio_admin_cmd}.
+
+The device MUST process commands on a given administration virtqueue
+in the order in which they are queued.
+
+If multiple administration virtqueues have been configured,
+device MAY process commands on distinct virtqueues with
+no order constraints.
+
+\drivernormative{\paragraph}{Group administration commands}{Basic Facilities of a Virtio Device / Administration virtqueues}
+
+The driver MAY supply device-readable or device-writeable parts
+of \field{struct virtio_admin_cmd} that are longer than described in
+this specification.
+
+The driver SHOULD supply device-readable part of
+\field{struct virtio_admin_cmd} that is at least as
+large as the structure described by this specification
+(even if the structure is all set to zero).
+
+The driver MUST supply both device-readable or device-writeable parts
+of \field{struct virtio_admin_cmd} that are a multiple of 64 bit
+in length.
+
+The device MUST supply both device-readable or device-writeable parts
+of \field{struct virtio_admin_cmd} that are larger than zero in
+length. However, \field{command_specific_data} and
+\field{command_specific_result} MAY be zero in length, unless
+specified otherwise for the command.

Quoted 70 lines · Michael S. Tsirkin

Add conformance clauses for admin commands and admin virtqueues.

Signed-off-by Michael S. Tsirkin <[email protected]>

---
  admin.tex | 216 +++++++++++++++++++++++++++++++++++++++++++++++++++++-
  1 file changed, 215 insertions(+), 1 deletion(-)

admin.tex +56 -0

index 1172054..6c4f79c 100644
@@ -251,6 +251,145 @@ \subsection{Group administration commands}\label{sec:Basic Facilities of a Virti
  supporting multiple group types, the list of supported commands
  might differ between different group types.
+\devicenormative{\paragraph}{Group administration commands}{Basic Facilities of a Virtio Device / Device groups / Group administration commands}
+
+The device MUST validate \field{opcode}, \field{group_type} and
+\field{group_member_id}, and if any of these has an invalid or
+unsupported value, set \field{status} to
+VIRTIO_ADMIN_STATUS_EINVAL and set \field{status_qualifier}
+accordingly:
+\begin{itemize}
+\item if \field{group_type} is invalid, \field{status_qualifier}
+	MUST be set to VIRTIO_ADMIN_STATUS_Q_INVALID_GROUP;
+\item otherwise, if \field{opcode} is invalid,
+	\field{status_qualifier} MUST be set to
+	VIRTIO_ADMIN_STATUS_Q_INVALID_OPCODE;
+\item otherwise, if \field{group_member_id} is used by the
+	specific command and is invalid, \field{status_qualifier} MUST be
+	set to VIRTIO_ADMIN_STATUS_Q_INVALID_MEMBER.
+\end{itemize}
+
+If a command completes successfully, the device MUST set
+\field{status} to VIRTIO_ADMIN_STATUS_OK.
+
+If a command fails, the device MUST set
+\field{status} to a value different from VIRTIO_ADMIN_STATUS_OK.
+
+If \field{status} is set to VIRTIO_ADMIN_STATUS_EINVAL, the
+device state MUST NOT change, that is the command MUST NOT have
+any side effects on the device, in particular the device MUST not
+enter an error state as a result of this command.
+
+If a command fails, the device state generally SHOULD NOT change,
+as far as possible.
+
+The device MAY enforce additional restrictions and dependencies on
+opcodes used by the driver and MAY fail the command
+VIRTIO_ADMIN_CMD_LIST_USE with \field{status} set to VIRTIO_ADMIN_STATUS_EINVAL
+and \field{status_qualifier} set to VIRTIO_ADMIN_STATUS_Q_INVALID_FIELD
+if the list of commands used violate internal device dependencies.
+
+If the device supports multiple group types, commands for each group
+type MUST operate independently of each other, in particular,
+the device MAY return different results for VIRTIO_ADMIN_CMD_LIST_QUERY
+for different group types.
+
+After reset, if the device supports a given group type
+and before receiving VIRTIO_ADMIN_CMD_LIST_USE for this group type
+the device MUST assume
+that the list of legal commands used by the driver consists of
+the two commands VIRTIO_ADMIN_CMD_LIST_QUERY and VIRTIO_ADMIN_CMD_LIST_USE.
+
+After completing VIRTIO_ADMIN_CMD_LIST_USE successfully,
+the device MUST set the list of legal commands used by the driver
+to the one supplied in \field{command_specific_data}.
+
+The device MUST set the list of legal commands used by the driver
+to the one supplied in VIRTIO_ADMIN_CMD_LIST_USE.
+

admin.tex \subsection{Group administration commands}\label{sec:Basic Facilities of a Virti · line 318

Quoted 10 lines

admin.tex +10 -0

+The device MUST validate commands against the list used by
+the driver and MUST fail any commands not in the list with
+\field{status} set to VIRTIO_ADMIN_STATUS_EINVAL
+and \field{status_qualifier} set to
+VIRTIO_ADMIN_STATUS_Q_INVALID_OPCODE.
+
+The list of supported commands MUST NOT shrink (but MAY expand):
+after reporting a given command as supported through
+VIRTIO_ADMIN_CMD_LIST_QUERY the device MUST NOT later report it
+as unsupported.  Further, after a given set of commands has been

Can the driver re-negotiate the command list through QUERY/USE in flight rather than upon a reset? If not, shall forbid this explicitly?

admin.tex \subsection{Group administration commands}\label{sec:Basic Facilities of a Virti · line 323

Quoted 4 lines

admin.tex +4 -0

+used (via a successful VIRTIO_ADMIN_CMD_LIST_USE), then after a
+device or system reset the device SHOULD complete successfully
+any following calls to VIRTIO_ADMIN_CMD_LIST_USE with the same
+list of commands; if this command VIRTIO_ADMIN_CMD_LIST_USE fails

I think this requires the device to remember what it had offered even after a power-cycled reset,

shall we say: The device should always offer all
supported
commands in the list?

Thanks,
Zhu Lingshan

Quoted 144 lines

admin.tex +137 -1

+after a device or system reset, the device MUST not fail it
+solely because of the command list used.  Failure to do so would
+interfere with resuming from suspend and error recovery.
+
+When processing a command with the SR-IOV group type,
+if the device does not have an SR-IOV Extended Capability or
+if \field{VF Enable} is clear
+then the device MUST fail all commands with
+\field{status} set to VIRTIO_ADMIN_STATUS_EINVAL and
+\field{status_qualifier} set to
+VIRTIO_ADMIN_STATUS_Q_INVALID_GROUP;
+otherwise, if \field{group_member_id} is not
+between $1$ and \field{NumVFs} inclusive,
+the device MUST fail all commands with
+\field{status} set to VIRTIO_ADMIN_STATUS_EINVAL and
+\field{status_qualifier} set to
+VIRTIO_ADMIN_STATUS_Q_INVALID_MEMBER;
+\field{NumVFs}, \field{VF Migration Capable}  and
+\field{VF Enable} refer to registers within the SR-IOV Extended
+Capability as specified by \hyperref[intro:PCIe]{[PCIe]}.
+
+\drivernormative{\paragraph}{Group administration commands}{Basic Facilities of a Virtio Device / Device groups / Group administration commands}
+
+The driver MAY discover whether device supports a specific group type
+by issuing VIRTIO_ADMIN_CMD_LIST_QUERY with the matching
+\field{group_type}.
+
+The driver MUST issue VIRTIO_ADMIN_CMD_LIST_USE
+and wait for it to be completed with status
+VIRTIO_ADMIN_STATUS_OK before issuing any commands
+(except for the initial VIRTIO_ADMIN_CMD_LIST_QUERY
+and VIRTIO_ADMIN_CMD_LIST_USE).
+
+The driver SHOULD NOT set bits in device_admin_cmds
+if it is not familiar with how the command opcode
+is used, as dependencies between command opcodes might exist.
+
+The driver MUST NOT request (via VIRTIO_ADMIN_CMD_LIST_USE)
+the use of any commands not previously reported as
+supported for the same group type by VIRTIO_ADMIN_CMD_LIST_QUERY.
+
+The driver MUST NOT use any commands for a given group type
+before sending VIRTIO_ADMIN_CMD_LIST_USE with the correct
+list of command opcodes and group type.
+
+The driver MAY block use of VIRTIO_ADMIN_CMD_LIST_QUERY and
+VIRTIO_ADMIN_CMD_LIST_USE by issuing VIRTIO_ADMIN_CMD_LIST_USE
+with respective bits cleared in \field{command_specific_data}.
+
+The driver MUST handle a command error with a reserved \field{status}
+value in the same way as \field{status} set to VIRTIO_ADMIN_STATUS_EINVAL
+(except possibly for different error reporting/diagnostic messages).
+
+The driver MUST handle a command error with a reserved
+\field{status_qualifier} value in the same way as
+\field{status_qualifier} set to
+VIRTIO_ADMIN_STATUS_Q_INVALID_COMMAND (except possibly for
+different error reporting/diagnostic messages).
+
+When sending commands with the SR-IOV group type,
+the driver specify a value for \field{group_member_id}
+between $1$ and \field{NumVFs} inclusive,
+the driver MUST also make sure that as long as any such command
+is outstanding, \field{VF Migration Capable} is clear and
+\field{VF Enable} is set;
+\field{NumVFs}, \field{VF Migration Capable}  and
+\field{VF Enable} refer to registers within the SR-IOV Extended
+Capability as specified by \hyperref[intro:PCIe]{[PCIe]}.
+
  \section{Administration Virtqueues}\label{sec:Basic Facilities of a Virtio Device / Administration Virtqueues}
  An administration virtqueue of an owner device is used to submit

@@ -323,4 +462,79 @@ \section{Administration Virtqueues}\label{sec:Basic Facilities of a Virtio Devic
  of the specification are designed, new fields can be added to the
  tail of a structure, with the driver/device using the full
  structure without concern for versioning.
->>>>>>> 0edc690... admin: introduce virtio admin virtqueues
+
+\devicenormative{\paragraph}{Group administration commands}{Basic Facilities of a Virtio Device / Administration virtqueues}
+
+The device MUST support device-readable and device-writeable buffers
+shorter than described in this specification, by
+\begin{enumerate}
+\item acting as if any data that would be read outside the
+device-readable buffers is set to zero, and
+\item discarding data that would be written outside the
+specified device-writeable buffers.
+\end{enumerate}
+
+The device MUST support device-readable and device-writeable buffers
+longer than described in this specification, by
+\begin{enumerate}
+\item ignoring any data in device-readable buffers outside
+the expected length, and
+\item only writing the expected structure to the device-writeable
+buffers, ignoring any extra buffers, and reporting the
+actual length of data written, in bytes,
+as buffer used length.
+\end{enumerate}
+
+The device SHOULD initialize the device-writeable buffer
+up to the length of the structure described by this specification or
+the length of the buffer supplied by the driver (even if the buffer is
+all set to zero), whichever is shorter.
+
+The device MUST NOT fail a command solely because the buffers
+provided are shorter or longer than described in this
+specification.
+
+The device MUST initialize the device-writeable part of
+\field{struct virtio_admin_cmd} that is a multiple of 64 bit in
+size.
+
+The device MUST initialize \field{status} in \field{struct
+virtio_admin_cmd}.
+
+The device MUST process commands on a given administration virtqueue
+in the order in which they are queued.
+
+If multiple administration virtqueues have been configured,
+device MAY process commands on distinct virtqueues with
+no order constraints.
+
+\drivernormative{\paragraph}{Group administration commands}{Basic Facilities of a Virtio Device / Administration virtqueues}
+
+The driver MAY supply device-readable or device-writeable parts
+of \field{struct virtio_admin_cmd} that are longer than described in
+this specification.
+
+The driver SHOULD supply device-readable part of
+\field{struct virtio_admin_cmd} that is at least as
+large as the structure described by this specification
+(even if the structure is all set to zero).
+
+The driver MUST supply both device-readable or device-writeable parts
+of \field{struct virtio_admin_cmd} that are a multiple of 64 bit
+in length.
+
+The device MUST supply both device-readable or device-writeable parts
+of \field{struct virtio_admin_cmd} that are larger than zero in
+length. However, \field{command_specific_data} and
+\field{command_specific_result} MAY be zero in length, unless
+specified otherwise for the command.
+
+The driver MUST NOT assume that the device will initialize the whole

admin.tex \section{Administration Virtqueues}\label{sec:Basic Facilities of a Virtio Devic · line 540

Quoted 8 lines

admin.tex +8 -0

+device-writeable part of \field{struct virtio_admin_cmd} as described in the specification; instead,
+the driver MUST act as if the structure
+outside the part of the buffer used by the device
+is set to zero.
+
+If multiple administration virtqueues have been configured,
+the driver MUST ensure ordering for commands
+placed on different administration virtqueues.

Quoted 84 lines · Lingshan

Quoted 70 lines · Michael S. Tsirkin

Add conformance clauses for admin commands and admin virtqueues.

Signed-off-by Michael S. Tsirkin <[email protected]>

---
   admin.tex | 216 +++++++++++++++++++++++++++++++++++++++++++++++++++++-
   1 file changed, 215 insertions(+), 1 deletion(-)

admin.tex +56 -0

index 1172054..6c4f79c 100644
@@ -251,6 +251,145 @@ \subsection{Group administration commands}\label{sec:Basic Facilities of a Virti
   supporting multiple group types, the list of supported commands
   might differ between different group types.
+\devicenormative{\paragraph}{Group administration commands}{Basic Facilities of a Virtio Device / Device groups / Group administration commands}
+
+The device MUST validate \field{opcode}, \field{group_type} and
+\field{group_member_id}, and if any of these has an invalid or
+unsupported value, set \field{status} to
+VIRTIO_ADMIN_STATUS_EINVAL and set \field{status_qualifier}
+accordingly:
+\begin{itemize}
+\item if \field{group_type} is invalid, \field{status_qualifier}
+	MUST be set to VIRTIO_ADMIN_STATUS_Q_INVALID_GROUP;
+\item otherwise, if \field{opcode} is invalid,
+	\field{status_qualifier} MUST be set to
+	VIRTIO_ADMIN_STATUS_Q_INVALID_OPCODE;
+\item otherwise, if \field{group_member_id} is used by the
+	specific command and is invalid, \field{status_qualifier} MUST be
+	set to VIRTIO_ADMIN_STATUS_Q_INVALID_MEMBER.
+\end{itemize}
+
+If a command completes successfully, the device MUST set
+\field{status} to VIRTIO_ADMIN_STATUS_OK.
+
+If a command fails, the device MUST set
+\field{status} to a value different from VIRTIO_ADMIN_STATUS_OK.
+
+If \field{status} is set to VIRTIO_ADMIN_STATUS_EINVAL, the
+device state MUST NOT change, that is the command MUST NOT have
+any side effects on the device, in particular the device MUST not
+enter an error state as a result of this command.
+
+If a command fails, the device state generally SHOULD NOT change,
+as far as possible.
+
+The device MAY enforce additional restrictions and dependencies on
+opcodes used by the driver and MAY fail the command
+VIRTIO_ADMIN_CMD_LIST_USE with \field{status} set to VIRTIO_ADMIN_STATUS_EINVAL
+and \field{status_qualifier} set to VIRTIO_ADMIN_STATUS_Q_INVALID_FIELD
+if the list of commands used violate internal device dependencies.
+
+If the device supports multiple group types, commands for each group
+type MUST operate independently of each other, in particular,
+the device MAY return different results for VIRTIO_ADMIN_CMD_LIST_QUERY
+for different group types.
+
+After reset, if the device supports a given group type
+and before receiving VIRTIO_ADMIN_CMD_LIST_USE for this group type
+the device MUST assume
+that the list of legal commands used by the driver consists of
+the two commands VIRTIO_ADMIN_CMD_LIST_QUERY and VIRTIO_ADMIN_CMD_LIST_USE.
+
+After completing VIRTIO_ADMIN_CMD_LIST_USE successfully,
+the device MUST set the list of legal commands used by the driver
+to the one supplied in \field{command_specific_data}.
+
+The device MUST set the list of legal commands used by the driver
+to the one supplied in VIRTIO_ADMIN_CMD_LIST_USE.
+

admin.tex \subsection{Group administration commands}\label{sec:Basic Facilities of a Virti · line 318

Quoted 10 lines

admin.tex +10 -0

+The device MUST validate commands against the list used by
+the driver and MUST fail any commands not in the list with
+\field{status} set to VIRTIO_ADMIN_STATUS_EINVAL
+and \field{status_qualifier} set to
+VIRTIO_ADMIN_STATUS_Q_INVALID_OPCODE.
+
+The list of supported commands MUST NOT shrink (but MAY expand):
+after reporting a given command as supported through
+VIRTIO_ADMIN_CMD_LIST_QUERY the device MUST NOT later report it
+as unsupported.  Further, after a given set of commands has been

Can the driver re-negotiate the command list through QUERY/USE in flight rather than upon a reset? If not, shall forbid this explicitly?

Quoted 6 lines

admin.tex \subsection{Group administration commands}\label{sec:Basic Facilities of a Virti · line 323

Quoted 4 lines

admin.tex +4 -0

+used (via a successful VIRTIO_ADMIN_CMD_LIST_USE), then after a
+device or system reset the device SHOULD complete successfully
+any following calls to VIRTIO_ADMIN_CMD_LIST_USE with the same
+list of commands; if this command VIRTIO_ADMIN_CMD_LIST_USE fails

I think this requires the device to remember what it had offered even after a power-cycled reset,

It just says always return the same.

Quoted 3 lines

shall we say: The device should always offer all
supported
commands in the list?

I don't see what will it mean.

Quoted 147 lines

Thanks,
Zhu Lingshan

Quoted 144 lines

admin.tex +137 -1

+after a device or system reset, the device MUST not fail it
+solely because of the command list used.  Failure to do so would
+interfere with resuming from suspend and error recovery.
+
+When processing a command with the SR-IOV group type,
+if the device does not have an SR-IOV Extended Capability or
+if \field{VF Enable} is clear
+then the device MUST fail all commands with
+\field{status} set to VIRTIO_ADMIN_STATUS_EINVAL and
+\field{status_qualifier} set to
+VIRTIO_ADMIN_STATUS_Q_INVALID_GROUP;
+otherwise, if \field{group_member_id} is not
+between $1$ and \field{NumVFs} inclusive,
+the device MUST fail all commands with
+\field{status} set to VIRTIO_ADMIN_STATUS_EINVAL and
+\field{status_qualifier} set to
+VIRTIO_ADMIN_STATUS_Q_INVALID_MEMBER;
+\field{NumVFs}, \field{VF Migration Capable}  and
+\field{VF Enable} refer to registers within the SR-IOV Extended
+Capability as specified by \hyperref[intro:PCIe]{[PCIe]}.
+
+\drivernormative{\paragraph}{Group administration commands}{Basic Facilities of a Virtio Device / Device groups / Group administration commands}
+
+The driver MAY discover whether device supports a specific group type
+by issuing VIRTIO_ADMIN_CMD_LIST_QUERY with the matching
+\field{group_type}.
+
+The driver MUST issue VIRTIO_ADMIN_CMD_LIST_USE
+and wait for it to be completed with status
+VIRTIO_ADMIN_STATUS_OK before issuing any commands
+(except for the initial VIRTIO_ADMIN_CMD_LIST_QUERY
+and VIRTIO_ADMIN_CMD_LIST_USE).
+
+The driver SHOULD NOT set bits in device_admin_cmds
+if it is not familiar with how the command opcode
+is used, as dependencies between command opcodes might exist.
+
+The driver MUST NOT request (via VIRTIO_ADMIN_CMD_LIST_USE)
+the use of any commands not previously reported as
+supported for the same group type by VIRTIO_ADMIN_CMD_LIST_QUERY.
+
+The driver MUST NOT use any commands for a given group type
+before sending VIRTIO_ADMIN_CMD_LIST_USE with the correct
+list of command opcodes and group type.
+
+The driver MAY block use of VIRTIO_ADMIN_CMD_LIST_QUERY and
+VIRTIO_ADMIN_CMD_LIST_USE by issuing VIRTIO_ADMIN_CMD_LIST_USE
+with respective bits cleared in \field{command_specific_data}.
+
+The driver MUST handle a command error with a reserved \field{status}
+value in the same way as \field{status} set to VIRTIO_ADMIN_STATUS_EINVAL
+(except possibly for different error reporting/diagnostic messages).
+
+The driver MUST handle a command error with a reserved
+\field{status_qualifier} value in the same way as
+\field{status_qualifier} set to
+VIRTIO_ADMIN_STATUS_Q_INVALID_COMMAND (except possibly for
+different error reporting/diagnostic messages).
+
+When sending commands with the SR-IOV group type,
+the driver specify a value for \field{group_member_id}
+between $1$ and \field{NumVFs} inclusive,
+the driver MUST also make sure that as long as any such command
+is outstanding, \field{VF Migration Capable} is clear and
+\field{VF Enable} is set;
+\field{NumVFs}, \field{VF Migration Capable}  and
+\field{VF Enable} refer to registers within the SR-IOV Extended
+Capability as specified by \hyperref[intro:PCIe]{[PCIe]}.
+
   \section{Administration Virtqueues}\label{sec:Basic Facilities of a Virtio Device / Administration Virtqueues}
   An administration virtqueue of an owner device is used to submit

@@ -323,4 +462,79 @@ \section{Administration Virtqueues}\label{sec:Basic Facilities of a Virtio Devic
   of the specification are designed, new fields can be added to the
   tail of a structure, with the driver/device using the full
   structure without concern for versioning.
->>>>>>> 0edc690... admin: introduce virtio admin virtqueues
+
+\devicenormative{\paragraph}{Group administration commands}{Basic Facilities of a Virtio Device / Administration virtqueues}
+
+The device MUST support device-readable and device-writeable buffers
+shorter than described in this specification, by
+\begin{enumerate}
+\item acting as if any data that would be read outside the
+device-readable buffers is set to zero, and
+\item discarding data that would be written outside the
+specified device-writeable buffers.
+\end{enumerate}
+
+The device MUST support device-readable and device-writeable buffers
+longer than described in this specification, by
+\begin{enumerate}
+\item ignoring any data in device-readable buffers outside
+the expected length, and
+\item only writing the expected structure to the device-writeable
+buffers, ignoring any extra buffers, and reporting the
+actual length of data written, in bytes,
+as buffer used length.
+\end{enumerate}
+
+The device SHOULD initialize the device-writeable buffer
+up to the length of the structure described by this specification or
+the length of the buffer supplied by the driver (even if the buffer is
+all set to zero), whichever is shorter.
+
+The device MUST NOT fail a command solely because the buffers
+provided are shorter or longer than described in this
+specification.
+
+The device MUST initialize the device-writeable part of
+\field{struct virtio_admin_cmd} that is a multiple of 64 bit in
+size.
+
+The device MUST initialize \field{status} in \field{struct
+virtio_admin_cmd}.
+
+The device MUST process commands on a given administration virtqueue
+in the order in which they are queued.
+
+If multiple administration virtqueues have been configured,
+device MAY process commands on distinct virtqueues with
+no order constraints.
+
+\drivernormative{\paragraph}{Group administration commands}{Basic Facilities of a Virtio Device / Administration virtqueues}
+
+The driver MAY supply device-readable or device-writeable parts
+of \field{struct virtio_admin_cmd} that are longer than described in
+this specification.
+
+The driver SHOULD supply device-readable part of
+\field{struct virtio_admin_cmd} that is at least as
+large as the structure described by this specification
+(even if the structure is all set to zero).
+
+The driver MUST supply both device-readable or device-writeable parts
+of \field{struct virtio_admin_cmd} that are a multiple of 64 bit
+in length.
+
+The device MUST supply both device-readable or device-writeable parts
+of \field{struct virtio_admin_cmd} that are larger than zero in
+length. However, \field{command_specific_data} and
+\field{command_specific_result} MAY be zero in length, unless
+specified otherwise for the command.
+
+The driver MUST NOT assume that the device will initialize the whole

Quoted 8 lines

admin.tex \section{Administration Virtqueues}\label{sec:Basic Facilities of a Virtio Devic · line 540

Quoted 8 lines

admin.tex +8 -0

+device-writeable part of \field{struct virtio_admin_cmd} as described in the specification; instead,
+the driver MUST act as if the structure
+outside the part of the buffer used by the device
+is set to zero.
+
+If multiple administration virtqueues have been configured,
+the driver MUST ensure ordering for commands
+placed on different administration virtqueues.

admin.tex \subsection{Group administration commands}\label{sec:Basic Facilities of a Virti · line 319

Quoted 4 lines

admin.tex +4 -0

+The list of supported commands MUST NOT shrink (but MAY expand):
+after reporting a given command as supported through
+VIRTIO_ADMIN_CMD_LIST_QUERY the device MUST NOT later report it
+as unsupported.  Further, after a given set of commands has been

Can the driver re-negotiate the command list through QUERY/USE in flight rather than upon a reset? If not, shall forbid this explicitly?

vq reset and shared memory are unsupported, too.

Signed-off-by Michael S. Tsirkin <[email protected]>

---
 content.tex | 2 ++
 1 file changed, 2 insertions(+)

From: Michael S. Tsirkin <[email protected]>
Sent: Thursday, March 2, 2023 8:05 AM
Change log:

since v9:

Quoted 6 lines

From: Michael S. Tsirkin <[email protected]>
Sent: Thursday, March 2, 2023 8:05 AM
Change log:

since v9:

This version is supposed to be v11 from last v10 [1].
https://lists.oasis-open.org/archives/virtio-dev/202302/msg00174.html

From: Michael S. Tsirkin <[email protected]>
Sent: Thursday, March 2, 2023 8:05 AM

My intent is to try and support both SR-IOV and SIOV usecases with the same structure and maybe even the same VQ.

For example, it might make sense to split creating/destroying SIOV devices from the transport passing data from the guest - the driver would then not negotiate VIRTIO_F_SR_IOV (which then means auto-provisioning).

This is out of RFC, since we have two commands which are useful to discover supported group types (ATM can be none or SR-IOV).

admin: introduce device group and related concepts
admin: introduce group administration commands
admin: introduce virtio admin virtqueues

Thu, Mar 02, 2023 at 02:04:48PM CET, [email protected] wrote:

[...]

Quoted 3 lines

admin: introduce device group and related concepts
admin: introduce group administration commands
admin: introduce virtio admin virtqueues

The ordering seems a bit odd to me. I think that at the beginning, the admin virtqueue should be introduced with generic concepts (including the generic command structure), only after that the section introducing device group stuff and related commands.

This publicly archived list offers a means to provide input to the OASIS Virtual I/O Device (VIRTIO) TC.

In order to verify user consent to the Feedback License terms and to minimize spam in the list archive, subscription is required before posting.

Subscribe: [email protected]
Unsubscribe: [email protected]
List help: [email protected]
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/

Quoted 5 lines · Jiri Pirko

Thu, Mar 02, 2023 at 02:04:48PM CET, [email protected] wrote:

[...]

Quoted 8 lines

Quoted 3 lines

admin: introduce device group and related concepts
admin: introduce group administration commands
admin: introduce virtio admin virtqueues

The ordering seems a bit odd to me. I think that at the beginning, the admin virtqueue should be introduced with generic concepts (including the generic command structure), only after that the section introducing device group stuff and related commands.

That's intentional.

Jason is working on additional ways to pass commands to devices, using memory mapped registers.

Thus admin commands stand by themselves, and the queue is just a way to issue them.

Quoted 10 lines

This publicly archived list offers a means to provide input to the OASIS Virtual I/O Device (VIRTIO) TC.

In order to verify user consent to the Feedback License terms and to minimize spam in the list archive, subscription is required before posting.

Quoted 8 lines

Subscribe: [email protected]
Unsubscribe: [email protected]
List help: [email protected]
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/

Quoted 20 lines · Mon, Mar 06, 2023 at 11:48:32PM CET, [email protected] wrote:

Quoted 5 lines · Jiri Pirko

Thu, Mar 02, 2023 at 02:04:48PM CET, [email protected] wrote:

[...]

Quoted 8 lines

Quoted 3 lines

admin: introduce device group and related concepts
admin: introduce group administration commands
admin: introduce virtio admin virtqueues

The ordering seems a bit odd to me. I think that at the beginning, the admin virtqueue should be introduced with generic concepts (including the generic command structure), only after that the section introducing device group stuff and related commands.

That's intentional.

Jason is working on additional ways to pass commands to devices, using memory mapped registers.

Quoted 2 lines

Thus admin commands stand by themselves, and the queue is just a way to issue them.

Okay, in that case, woudnlt it make sense if the generic CMD text (including the generic command structure) would go first, then the "group" related cmds?

Quoted 10 lines

Quoted 10 lines

This publicly archived list offers a means to provide input to the OASIS Virtual I/O Device (VIRTIO) TC.

In order to verify user consent to the Feedback License terms and to minimize spam in the list archive, subscription is required before posting.

Quoted 9 lines

Quoted 8 lines

Subscribe: [email protected]
Unsubscribe: [email protected]
List help: [email protected]
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/

Thu, Mar 02, 2023 at 02:04:48PM CET, [email protected] wrote:

[...]

Quoted 5 lines

TODO (maybe?) - probably ok to defer until this part is upstream:

Add "all members" member id.

Quoted 8 lines

Add commands for MSI, feature discovery.

Add commands for transport vq.

My intent is to try and support both SR-IOV and SIOV usecases with the same structure and maybe even the same VQ.

Sorry to be late to the party, I'm trying to catch up. Is it common to have cover letter for features this brief? I mean, from the cover letter, I'm totally unable to understand what you are introducing here.

Could you elaborate about what you are aiming to achive with this? Could you shed some usecases perhaps?

I have to be missing something obvious, but I don't get why any notion of SR-IOV could be beneficial for virtio.

Quoted 22 lines

For example, it might make sense to split creating/destroying SIOV devices from the transport passing data from the guest - the driver would then not negotiate VIRTIO_F_SR_IOV (which then means auto-provisioning).

This is out of RFC, since we have two commands which are useful to discover supported group types (ATM can be none or SR-IOV).

Michael S. Tsirkin (10):
virtio: document forward compatibility guarantees
admin: introduce device group and related concepts
admin: introduce group administration commands
admin: introduce virtio admin virtqueues
pci: add admin vq registers to virtio over pci
mmio: document ADMIN_VQ as reserved
ccw: document ADMIN_VQ as reserved
admin: command list discovery
admin: conformance clauses
ccw: document more reserved features

Quoted 26 lines

admin.tex        | 540 +++++++++++++++++++++++++++++++++++++++++++++++
content.tex      | 112 +++++++++-
introduction.tex |   3 +
3 files changed, 653 insertions(+), 2 deletions(-)

create mode 100644 admin.tex

Signature · 18 lines

MST


This publicly archived list offers a means to provide input to the
OASIS Virtual I/O Device (VIRTIO) TC.

In order to verify user consent to the Feedback License terms and
to minimize spam in the list archive, subscription is required
before posting.

Subscribe: [email protected]
Unsubscribe: [email protected]
List help: [email protected]
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/

This publicly archived list offers a means to provide input to the OASIS Virtual I/O Device (VIRTIO) TC.

In order to verify user consent to the Feedback License terms and to minimize spam in the list archive, subscription is required before posting.

Subscribe: [email protected]
Unsubscribe: [email protected]
List help: [email protected]
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/

Quoted 25 lines · Jiri Pirko

Thu, Mar 02, 2023 at 02:04:48PM CET, [email protected] wrote:

[...]

Quoted 5 lines

TODO (maybe?) - probably ok to defer until this part is upstream:

Add "all members" member id.

Quoted 8 lines

Add commands for MSI, feature discovery.

Add commands for transport vq.

My intent is to try and support both SR-IOV and SIOV usecases with the same structure and maybe even the same VQ.

Sorry to be late to the party, I'm trying to catch up. Is it common to have cover letter for features this brief? I mean, from the cover letter, I'm totally unable to understand what you are introducing here.

Could you elaborate about what you are aiming to achive with this? Could you shed some usecases perhaps?

Quoted 3 lines

I have to be missing something obvious, but I don't get why any notion of SR-IOV could be beneficial for virtio.

Good point, I'll add a bit of motivation.

For SR-IOV, it is not unusual for PFs to excercise control over VFs.

There is interest in the community to include an interface to allow this in the virtio spec, when the PF is a virtio device. This is what this patch does.

Quoted 60 lines

Quoted 26 lines

For example, it might make sense to split creating/destroying SIOV devices from the transport passing data from the guest - the driver would then not negotiate VIRTIO_F_SR_IOV (which then means auto-provisioning).

This is out of RFC, since we have two commands which are useful to discover supported group types (ATM can be none or SR-IOV).

Michael S. Tsirkin (10):
virtio: document forward compatibility guarantees
admin: introduce device group and related concepts
admin: introduce group administration commands
admin: introduce virtio admin virtqueues
pci: add admin vq registers to virtio over pci
mmio: document ADMIN_VQ as reserved
ccw: document ADMIN_VQ as reserved
admin: command list discovery
admin: conformance clauses
ccw: document more reserved features

Quoted 26 lines

admin.tex        | 540 +++++++++++++++++++++++++++++++++++++++++++++++
content.tex      | 112 +++++++++-
introduction.tex |   3 +
3 files changed, 653 insertions(+), 2 deletions(-)

create mode 100644 admin.tex

Signature · 18 lines

MST


This publicly archived list offers a means to provide input to the
OASIS Virtual I/O Device (VIRTIO) TC.

In order to verify user consent to the Feedback License terms and
to minimize spam in the list archive, subscription is required
before posting.

Subscribe: [email protected]
Unsubscribe: [email protected]
List help: [email protected]
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/

This publicly archived list offers a means to provide input to the OASIS Virtual I/O Device (VIRTIO) TC.

In order to verify user consent to the Feedback License terms and to minimize spam in the list archive, subscription is required before posting.

Quoted 8 lines

Subscribe: [email protected]
Unsubscribe: [email protected]
List help: [email protected]
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/

Quoted 26 lines · Mon, Mar 06, 2023 at 11:54:45PM CET, [email protected] wrote:

Quoted 25 lines · Jiri Pirko

Thu, Mar 02, 2023 at 02:04:48PM CET, [email protected] wrote:

[...]

Quoted 5 lines

TODO (maybe?) - probably ok to defer until this part is upstream:

Add "all members" member id.

Quoted 8 lines

Add commands for MSI, feature discovery.

Add commands for transport vq.

My intent is to try and support both SR-IOV and SIOV usecases with the same structure and maybe even the same VQ.

Sorry to be late to the party, I'm trying to catch up. Is it common to have cover letter for features this brief? I mean, from the cover letter, I'm totally unable to understand what you are introducing here.

Could you elaborate about what you are aiming to achive with this? Could you shed some usecases perhaps?

Quoted 7 lines

Quoted 3 lines

I have to be missing something obvious, but I don't get why any notion of SR-IOV could be beneficial for virtio.

Good point, I'll add a bit of motivation.

For SR-IOV, it is not unusual for PFs to excercise control over VFs.

I understand the concepts of SR-IOV. Yet I fail to see the need of such concept in virtio world. SR-IOV is very specific solution for PCI functions instantiation, and I believe that it is already considered quite limiting in many aspects. Does not make sense to me to introduce it for virtio. But again, I may be missing something crucial, I just would like to see the motivation, needs, usecases for this crystal clear, which is opposite to the current cover letter I'm afraid :/

Quoted 4 lines

There is interest in the community to include an interface to allow this in the virtio spec, when the PF is a virtio device. This is what this patch does.

Yeah, but why? As I asked before, what are the usecases? The fact there is interest in the community does not mean it makes sense to have it :)

Quoted 60 lines

Quoted 60 lines

Quoted 26 lines

For example, it might make sense to split creating/destroying SIOV devices from the transport passing data from the guest - the driver would then not negotiate VIRTIO_F_SR_IOV (which then means auto-provisioning).

This is out of RFC, since we have two commands which are useful to discover supported group types (ATM can be none or SR-IOV).

Michael S. Tsirkin (10):
virtio: document forward compatibility guarantees
admin: introduce device group and related concepts
admin: introduce group administration commands
admin: introduce virtio admin virtqueues
pci: add admin vq registers to virtio over pci
mmio: document ADMIN_VQ as reserved
ccw: document ADMIN_VQ as reserved
admin: command list discovery
admin: conformance clauses
ccw: document more reserved features

Quoted 26 lines

admin.tex        | 540 +++++++++++++++++++++++++++++++++++++++++++++++
content.tex      | 112 +++++++++-
introduction.tex |   3 +
3 files changed, 653 insertions(+), 2 deletions(-)

create mode 100644 admin.tex

Signature · 18 lines

MST


This publicly archived list offers a means to provide input to the
OASIS Virtual I/O Device (VIRTIO) TC.

In order to verify user consent to the Feedback License terms and
to minimize spam in the list archive, subscription is required
before posting.

Subscribe: [email protected]
Unsubscribe: [email protected]
List help: [email protected]
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/

This publicly archived list offers a means to provide input to the OASIS Virtual I/O Device (VIRTIO) TC.

In order to verify user consent to the Feedback License terms and to minimize spam in the list archive, subscription is required before posting.

Quoted 9 lines

Quoted 8 lines

Subscribe: [email protected]
Unsubscribe: [email protected]
List help: [email protected]
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/

From: Michael S. Tsirkin <[email protected]>
Sent: Tuesday, March 7, 2023 12:20 PM

Quoted 9 lines

Quoted 1 line

For SR-IOV, it is not unusual for PFs to excercise control over VFs.

I understand the concepts of SR-IOV. Yet I fail to see the need of such concept in virtio world. SR-IOV is very specific solution for PCI functions instantiation, and I believe that it is already considered quite limiting in many aspects. Does not make sense to me to introduce it for virtio. But again, I may be missing something crucial, I just would like to see the motivation, needs, usecases for this crystal clear, which is opposite to the current cover letter I'm afraid :/

First people are asking for it because it's out there, however limiting it is. In fact Nvidia is - why don't you talk to Parav here and tell him that SR-IOV is legacy and there's no need to support.

Quoted 10 lines

Quoted 5 lines

There is interest in the community to include an interface to allow this in the virtio spec, when the PF is a virtio device. This is what this patch does.

Yeah, but why? As I asked before, what are the usecases? The fact there is interest in the community does not mean it makes sense to have it :)

If people want to build such hardware it will need some interface. Far better to have it standard.

But generally e.g. intel already said they will reuse this same structure with a different group type for SIOV support.
I'll mention this in the cover letter.

Quoted 4 lines

From: Michael S. Tsirkin <[email protected]>
Sent: Tuesday, March 7, 2023 12:20 PM

[..]

Quoted 10 lines

Quoted 9 lines

Quoted 1 line

For SR-IOV, it is not unusual for PFs to excercise control over VFs.

I understand the concepts of SR-IOV. Yet I fail to see the need of such concept in virtio world. SR-IOV is very specific solution for PCI functions instantiation, and I believe that it is already considered quite limiting in many aspects. Does not make sense to me to introduce it for virtio. But again, I may be missing something crucial, I just would like to see the motivation, needs, usecases for this crystal clear, which is opposite to the current cover letter I'm afraid :/

Quoted 4 lines

First people are asking for it because it's out there, however limiting it is. In fact Nvidia is - why don't you talk to Parav here and tell him that SR-IOV is legacy and there's no need to support.

Jiri and I discussed offline today.
SR-IOV is widely used today and in the coming future up to thousands of devices. Many vendors are building it or already built.

SIOV as it stands today is still being crafted at various levels iommu, os, device, pci and more. Not there yet fully.

Jiri and I built the Linux kernel interfaces for SFs which are equivalent to SIOVs that overcome some of the SR-IOV limitations. And Jiri's suggestion was to leverage some of the vision and practice from there.

In other words, can AQ command be useful tomorrow for doing SIOV device add/remove, and provisioning from non-owning PF? The way AQ is crafted today is not there yet, but in the future, it can be extended.

Quoted 11 lines

Quoted 10 lines

Quoted 5 lines

There is interest in the community to include an interface to allow this in the virtio spec, when the PF is a virtio device. This is what this patch does.

Yeah, but why? As I asked before, what are the usecases? The fact there is interest in the community does not mean it makes sense to have it :)

Quoted 8 lines

If people want to build such hardware it will need some interface. Far better to have it standard.

But generally e.g. intel already said they will reuse this same structure with a different group type for SIOV support.
I'll mention this in the cover letter.

We have the following already identified use cases of a device agnostic VQ.

1. SR-IOV VF device provisioning at virtio layer (features and config)
2. SR IOV (SR-PCIM interface of the PCI spec) for VF provisioning, for example, MSI-X vectors

3. SR-IOV VF Live migration

4. Above #1 to #3 for SIOV devices instead of SRIOV devices

5. cmd q interface for PF/VF/SF/SIOV to transport their own features, config command over queue interface without any parent device

#1 to #3 are well described in past. So, It is worth covering them in the cover letter and things will be clear for new eyes. #4 is broad use case and like #1 to #3 that will eventually happen.
#5 likely needs more discussion, we can differ to a later point because AQ needs to progress in spec 1.3.

From: Jason Wang <[email protected]>
Sent: Wednesday, March 8, 2023 12:12 AM

Quoted 1 line

In other words, can AQ command be useful tomorrow for doing SIOV device

add/remove, and provisioning from non-owning PF?

Quoted 1 line

The way AQ is crafted today is not there yet, but in the future, it can be

extended.

There's already a proposal to use the queue as a transport. Any device that can't use a transport specific configuration or provisioning can benefit from that. SIOV is only one of the users for that.

Quoted 20 lines

Quoted 19 lines

Quoted 10 lines

Quoted 5 lines

There is interest in the community to include an interface to allow this in the virtio spec, when the PF is a virtio device. This is what this patch does.

Yeah, but why? As I asked before, what are the usecases? The fact there is interest in the community does not mean it makes sense to have it :)

If people want to build such hardware it will need some interface. Far better to have it standard.

But generally e.g. intel already said they will reuse this same structure with a different group type for SIOV support. I'll mention this in the cover letter.

Quoted 6 lines

We have the following already identified use cases of a device agnostic VQ.

1. SR-IOV VF device provisioning at virtio layer (features and config)
2. SR IOV (SR-PCIM interface of the PCI spec) for VF provisioning, for example, MSI-X vectors

The virtio layer device provisioning is transport independent.

I don't see
anything that is PCI specific there. Even the MSI-X could be reused by other transport.

Quoted 1 line

3. SR-IOV VF Live migration

I think we don't want a partial solution for live migration that only works for PCI VF devices and L1. Instead, there needs to be a general device facility section and the admin virtqueue could be one of the interfaces for the driver to use the facility.

Quoted 1 line

4. Above #1 to #3 for SIOV devices instead of SRIOV devices

If we make the above as a general facility, it would not be hard to re-use them for transport virtqueue or SIOV.

Quoted 2 lines

From: Michael S. Tsirkin <[email protected]>
Sent: Tuesday, March 7, 2023 12:20 PM

[..]

Quoted 8 lines

Quoted 8 lines

Quoted 1 line

For SR-IOV, it is not unusual for PFs to excercise control over VFs.

I understand the concepts of SR-IOV. Yet I fail to see the need of such concept in virtio world. SR-IOV is very specific solution for PCI functions instantiation, and I believe that it is already considered quite limiting in many aspects. Does not make sense to me to introduce it for virtio. But again, I may be missing something crucial, I just would like to see the motivation, needs, usecases for this crystal clear, which is opposite to the current cover letter I'm afraid :/

Quoted 4 lines

First people are asking for it because it's out there, however limiting it is. In fact Nvidia is - why don't you talk to Parav here and tell him that SR-IOV is legacy and there's no need to support.

Jiri and I discussed offline today.
SR-IOV is widely used today and in the coming future up to thousands of devices. Many vendors are building it or already built.

SIOV as it stands today is still being crafted at various levels iommu, os, device, pci and more. Not there yet fully.

Jiri and I built the Linux kernel interfaces for SFs which are equivalent to SIOVs that overcome some of the SR-IOV limitations. And Jiri's suggestion was to leverage some of the vision and practice from there.

In other words, can AQ command be useful tomorrow for doing SIOV device add/remove, and provisioning from non-owning PF? The way AQ is crafted today is not there yet, but in the future, it can be extended.

Quoted 7 lines

Quoted 7 lines

Quoted 3 lines

There is interest in the community to include an interface to allow this in the virtio spec, when the PF is a virtio device. This is what this patch does.

Yeah, but why? As I asked before, what are the usecases? The fact there is interest in the community does not mean it makes sense to have it :)

Quoted 8 lines

If people want to build such hardware it will need some interface. Far better to have it standard.

But generally e.g. intel already said they will reuse this same structure with a different group type for SIOV support.
I'll mention this in the cover letter.

We have the following already identified use cases of a device agnostic VQ.

1. SR-IOV VF device provisioning at virtio layer (features and config)
2. SR IOV (SR-PCIM interface of the PCI spec) for VF provisioning, for example, MSI-X vectors

3. SR-IOV VF Live migration
4. Above #1 to #3 for SIOV devices instead of SRIOV devices

5. cmd q interface for PF/VF/SF/SIOV to transport their own features, config command over queue interface without any parent device

#1 to #3 are well described in past. So, It is worth covering them in the cover letter and things will be clear for new eyes. #4 is broad use case and like #1 to #3 that will eventually happen.
#5 likely needs more discussion, we can differ to a later point because AQ needs to progress in spec 1.3.

Quoted 35 lines · Jiri Pirko

Quoted 26 lines · Mon, Mar 06, 2023 at 11:54:45PM CET, [email protected] wrote:

Quoted 25 lines · Jiri Pirko

Thu, Mar 02, 2023 at 02:04:48PM CET, [email protected] wrote:

[...]

Quoted 5 lines

TODO (maybe?) - probably ok to defer until this part is upstream:

Add "all members" member id.

Quoted 8 lines

Add commands for MSI, feature discovery.

Add commands for transport vq.

My intent is to try and support both SR-IOV and SIOV usecases with the same structure and maybe even the same VQ.

Sorry to be late to the party, I'm trying to catch up. Is it common to have cover letter for features this brief? I mean, from the cover letter, I'm totally unable to understand what you are introducing here.

Could you elaborate about what you are aiming to achive with this? Could you shed some usecases perhaps?

Quoted 7 lines

Quoted 3 lines

I have to be missing something obvious, but I don't get why any notion of SR-IOV could be beneficial for virtio.

Good point, I'll add a bit of motivation.

For SR-IOV, it is not unusual for PFs to excercise control over VFs.

Quoted 7 lines

I understand the concepts of SR-IOV. Yet I fail to see the need of such concept in virtio world. SR-IOV is very specific solution for PCI functions instantiation, and I believe that it is already considered quite limiting in many aspects. Does not make sense to me to introduce it for virtio. But again, I may be missing something crucial, I just would like to see the motivation, needs, usecases for this crystal clear, which is opposite to the current cover letter I'm afraid :/

First people are asking for it because it's out there, however limiting it is. In fact Nvidia is - why don't you talk to Parav here and tell him that SR-IOV is legacy and there's no need to support.

Quoted 5 lines

Quoted 7 lines

Quoted 3 lines

There is interest in the community to include an interface to allow this in the virtio spec, when the PF is a virtio device. This is what this patch does.

Yeah, but why? As I asked before, what are the usecases? The fact there is interest in the community does not mean it makes sense to have it :)

If people want to build such hardware it will need some interface. Far better to have it standard.

But generally e.g. intel already said they will reuse this same structure with a different group type for SIOV support. I'll mention this in the cover letter.

Quoted 61 lines

Quoted 61 lines

Quoted 61 lines

Quoted 27 lines

For example, it might make sense to split creating/destroying SIOV devices from the transport passing data from the guest - the driver would then not negotiate VIRTIO_F_SR_IOV (which then means auto-provisioning).

This is out of RFC, since we have two commands which are useful to discover supported group types (ATM can be none or SR-IOV).

Michael S. Tsirkin (10):
virtio: document forward compatibility guarantees
admin: introduce device group and related concepts
admin: introduce group administration commands
admin: introduce virtio admin virtqueues
pci: add admin vq registers to virtio over pci
mmio: document ADMIN_VQ as reserved
ccw: document ADMIN_VQ as reserved
admin: command list discovery
admin: conformance clauses
ccw: document more reserved features

Quoted 26 lines

admin.tex        | 540 +++++++++++++++++++++++++++++++++++++++++++++++
content.tex      | 112 +++++++++-
introduction.tex |   3 +
3 files changed, 653 insertions(+), 2 deletions(-)

create mode 100644 admin.tex

Signature · 18 lines

MST


This publicly archived list offers a means to provide input to the
OASIS Virtual I/O Device (VIRTIO) TC.

In order to verify user consent to the Feedback License terms and
to minimize spam in the list archive, subscription is required
before posting.

Subscribe: [email protected]
Unsubscribe: [email protected]
List help: [email protected]
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/

This publicly archived list offers a means to provide input to the OASIS Virtual I/O Device (VIRTIO) TC.

In order to verify user consent to the Feedback License terms and to minimize spam in the list archive, subscription is required before posting.

Quoted 9 lines

Quoted 9 lines

Quoted 8 lines

Subscribe: [email protected]
Unsubscribe: [email protected]
List help: [email protected]
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/

---------------------------------------------------------------------
To unsubscribe from this mail list, you must leave the OASIS TC that generates this mail. Follow this link to all your TCs in OASIS at:
https://www.oasis-open.org/apps/org/workgroup/portal/my_workgroups.php

Quoted 22 lines · Jiri Pirko

Thu, Mar 02, 2023 at 02:04:48PM CET, [email protected] wrote:

[...]

Quoted 5 lines

TODO (maybe?) - probably ok to defer until this part is upstream:

Add "all members" member id.

Quoted 8 lines

Add commands for MSI, feature discovery.

Add commands for transport vq.

My intent is to try and support both SR-IOV and SIOV usecases with the same structure and maybe even the same VQ.

Sorry to be late to the party, I'm trying to catch up. Is it common to have cover letter for features this brief? I mean, from the cover letter, I'm totally unable to understand what you are introducing here.

Quoted 6 lines

Could you elaborate about what you are aiming to achive with this? Could you shed some usecases perhaps?

I have to be missing something obvious, but I don't get why any notion of SR-IOV could be beneficial for virtio.

Good point, I'll add a bit of motivation.

For SR-IOV, it is not unusual for PFs to excercise control over VFs. There is interest in the community to include an interface to allow this in the virtio spec, when the PF is a virtio device. This is what this patch does.

Quoted 30 lines · Michael S. Tsirkin

Quoted 22 lines · Jiri Pirko

Thu, Mar 02, 2023 at 02:04:48PM CET, [email protected] wrote:

[...]

Quoted 5 lines

TODO (maybe?) - probably ok to defer until this part is upstream:

Add "all members" member id.

Quoted 8 lines

Add commands for MSI, feature discovery.

Add commands for transport vq.

My intent is to try and support both SR-IOV and SIOV usecases with the same structure and maybe even the same VQ.

Sorry to be late to the party, I'm trying to catch up. Is it common to have cover letter for features this brief? I mean, from the cover letter, I'm totally unable to understand what you are introducing here.

Quoted 6 lines

Could you elaborate about what you are aiming to achive with this? Could you shed some usecases perhaps?

I have to be missing something obvious, but I don't get why any notion of SR-IOV could be beneficial for virtio.

Quoted 6 lines

Good point, I'll add a bit of motivation.

For SR-IOV, it is not unusual for PFs to excercise control over VFs. There is interest in the community to include an interface to allow this in the virtio spec, when the PF is a virtio device. This is what this patch does.

Unfortunately, information in the cover letter does not make it into git -- should things like that go into the github issue (and into the ballot?) It's useful both for reviewing (cover letter) and understanding the rationale later (github/ballot).

Quoted 32 lines · Michael S. Tsirkin

Quoted 22 lines · Jiri Pirko

Thu, Mar 02, 2023 at 02:04:48PM CET, [email protected] wrote:

[...]

Quoted 5 lines

TODO (maybe?) - probably ok to defer until this part is upstream:

Add "all members" member id.

Quoted 8 lines

Add commands for MSI, feature discovery.

Add commands for transport vq.

My intent is to try and support both SR-IOV and SIOV usecases with the same structure and maybe even the same VQ.

Sorry to be late to the party, I'm trying to catch up. Is it common to have cover letter for features this brief? I mean, from the cover letter, I'm totally unable to understand what you are introducing here.

Quoted 6 lines

Could you elaborate about what you are aiming to achive with this? Could you shed some usecases perhaps?

I have to be missing something obvious, but I don't get why any notion of SR-IOV could be beneficial for virtio.

Good point, I'll add a bit of motivation.

Quoted 4 lines

For SR-IOV, it is not unusual for PFs to excercise control over VFs. There is interest in the community to include an interface to allow this in the virtio spec, when the PF is a virtio device. This is what this patch does.

Unfortunately, information in the cover letter does not make it into git -- should things like that go into the github issue (and into the

ballot?) It's useful both for reviewing (cover letter) and understanding the rationale later (github/ballot).