Introduce device group and device management
v13 cover Introduce device group and device management
Change log:
since 13:
command specific data is u8 again
exclude admin queues in blk's num_queues
minor other tweaks
since 11:
addressed lots of comments, all minor. consistency with outstanding number->index and queue->enqueue work i did not intentionally drop any reviewed-by tags as all changes are minor - if yours is missing it is because I forgot to record it, sorry
one "breaking" change in response to stefan's comment:
in patch 5, num_queues has been specified not to include admin
queues: just regular ones.
since v10:
addressed lots of comments by Jiri, Stefan. Cornelia, Lngshan, Parav, Max
since v9:
addressed comments by Parav, Max, Cornelia, David and Zhu Lingshan:
added link to errno header from Linux
rename _MEM to _MEMBER
admin vq num is zero based
clarify who sends commands where
minor english tweaks
clarify command length
specify interaction with sriov capability correct commit log - NumVFs can be 0
i could not decide what should happen when VFs are disabled. for now did not specify.
since v8:
addressed comments by Cornelia - as we agreed on list
since v7:
make high level error codes match linux, with virtio specific codes
in a separate field
renamed _ACCEPT to _USE since that's what it does clarified forward compatibility and non pci transports support multiple admin vqs
conformance statements
lots of changes all over the place to I changed author from Max to myself. Don't need to take credit but also don't want to blame Max for my mistakes.
since v6:
- removed some extentions intended for future use. We'll do them when we get there.
- brought back command list query from v5 in a simplified form - it's here to address the case where a single parent can address multiple groups, such as PF addressing transport vq and sriov vfs.
- attempt to make terminology more formal. In particular a term for whoever controls the group. I am still going back
and forth between "parent" and "owner" - owner might be better after all since it will work if we ever have a self group. For now it's parent.
TODO (maybe?) - probably ok to defer until this part is upstream:
Add "all members" member id.
Add commands for MSI, feature discovery.
Add commands for transport vq.
My intent is to try and support both SR-IOV and SIOV usecases with the same structure and maybe even the same VQ.
For example, it might make sense to split creating/destroying SIOV devices from the transport passing data from the guest - the driver would then not negotiate VIRTIO_F_SR_IOV (which then means auto-provisioning).
More ideas for use-cases:
virtio VF features query and configuration space provisioning virtio VF resource (queues, msix vectors count) provisioning
Future directions (shouldn't block this patch)
- aborting commands - left for later. or is vq reset enough?
- should we rename structures from admin to group admin?
Michael S. Tsirkin (10):
virtio: document forward compatibility guarantees
admin: introduce device group and related concepts
admin: introduce group administration commands
admin: introduce virtio admin virtqueues
pci: add admin vq registers to virtio over pci
mmio: document ADMIN_VQ as reserved
ccw: document ADMIN_VQ as reserved
admin: command list discovery
admin: conformance clauses
ccw: document more reserved features
admin.tex | 584 +++++++++++++++++++++++++++++++ content.tex | 62 +++- device-types/blk/description.tex | 2 +- introduction.tex | 3 + transport-ccw.tex | 14 + transport-mmio.tex | 12 + transport-pci.tex | 33 ++ 7 files changed, 707 insertions(+), 3 deletions(-) create mode 100644 admin.tex
Signature · 6 lines
MST --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
PATCH v13 2/10 admin: introduce device group and related concepts
Each device group has a type. For now, define one initial group type:
SR-IOV type - PCI SR-IOV virtual functions (VFs) of a given PCI SR-IOV physical function (PF). This group may contain zero or more virtio devices according to NumVFs configured.
Each device within a group has a unique identifier. This identifier is the group member identifier.
Note: one can argue both ways whether the new device group handling functionality (this and following patches) is closer
to a new device type or a new transport type.
However, it's expected that we will add more features in the near future. To facilitate this as much as possible of the text is located in the new admin chapter.
Effort was made to minimize transport-specific text.
There's a bit of duplication with 0x1 repeated twice and no special section for group type identifiers.
It seems ok to defer adding these until we have more group types.
---
changes in v12:
stefan's ack
minor commit log tweaks, Parav
use neutral tone, Parav
changes in v11:
dropped Max's S.O.B.
dropped an unmatched ) as reported by Parav
document that member id is 1 to numvfs
document that vf enable must be set (will also be reflected in
the conformance section)
document that we deferred generalizing text as requested by Parav - I think we can do it later
minor wording fixes to address comments by David
add concepts of owner and member driver. unused currently but will come in handy later, as suggested by Stefan
--- admin.tex | 63 +++++++++++++++++++++++++++++++++++++++++++++++++++++ content.tex | 2 ++ 2 files changed, 65 insertions(+) create mode 100644 admin.tex
admin.tex new file +63 -0
@@ -0,0 +1,63 @@ +\section{Device groups}\label{sec:Basic Facilities of a Virtio Device / Device groups} + +It is occasionally useful to have a device control a group of +other devices. Terminology used in such cases: + +\begin{description} +\item[Device group] + or just group, includes zero or more devices. +\item[Owner device] + or owner, the device controlling the group. +\item[Member device] + a device within a group. The owner device itself is not + a member of the group. +\item[Member identifier] + each member has this identifier, unique within the group + and used to address it through the owner device. +\item[Group type identifier] + specifies what kind of member devices there are in a + group, how the member identifier is interpreted + and what kind of control the owner has. + A given owner can control multiple groups + of different types but only a single group of a given type, + thus the type and the owner together identify the group. + \footnote{Even though some group types only support + specific transports, group type identifiers + are global rather than transport-specific - + a flood of new group types is not expected.} +\end{description} + +\begin{note} +Each device only has a single driver, thus for the purposes of +this section, "the driver" is usually unambiguous and refers to +the driver of the owner device. When there's ambiguity, "owner +driver" refers to the driver of the owner device, while "member +driver" refers to the driver of a member device. +\end{note} + +The following group types, and their identifiers, are currently specified: +\begin{description} +\item[SR-IOV group type (0x1)] +This device group has a PCI Single Root I/O Virtualization +(SR-IOV) physical function (PF) device as the owner and includes +all its SR-IOV virtual functions (VFs) as members (see +\hyperref[intro:PCIe]{[PCIe]}). + +The PF device itself is not a member of the group. + +The group type identifier for this group is 0x1. + +A member identifier for this group can have a value from 0x1 to +\field{NumVFs} as specified in the +SR-IOV Extended Capability of the owner device +and equals the SR-IOV VF number of the member device; +the group only exists when the \field{VF Enable} bit +in the SR-IOV Control Register within the +SR-IOV Extended Capability of the owner device is set +(see \hyperref[intro:PCIe]{[PCIe]}). + +Both owner and member devices for this group type use the Virtio +PCI transport (see \ref{sec:Virtio Transport Options / Virtio Over PCI Bus}). +\end{description} + +
content.tex +2 -0
@@ -493,6 +493,8 @@ \section{Exporting Objects}\label{sec:Basic Facilities of a Virtio Device / Expo types. It is RECOMMENDED that devices generate version 4 UUIDs as specified by \hyperref[intro:rfc4122]{[RFC4122]}. +\input{admin.tex} + \chapter{General Initialization And Device Operation}\label{sec:General Initialization And Device Operation} We start with an overview of device initialization, then expand on the
Signature · 6 lines
MST --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
PATCH v13 3/10 admin: introduce group administration commands
In reply to Michael S. TsirkinThis introduces a general structure for group administration commands, used to control device groups through their owner.
Following patches will introduce specific commands and an interface for submitting these commands to the owner.
Note that the commands are focused on controlling device groups:
this is why group related fields are in the generic part of the structure.
Without this the admin vq would become a "whatever" vq which does not do anything specific at all, just a general transport like thing. I feel going this way opens the design space to the point where we no longer know what belongs in e.g. config space
what in the control q and what in the admin q.
As it is, whatever deals with groups is in the admin q; other things not in the admin q.
This is also why I am using the term administration and not management:
administration as in hypervisor (admin) access through e.g. PF while guest has access through e.g. VF.
For example for SRIOV, anything where we access VF through a PF would be in scope for admin commands.
There are specific exceptions such as query but that's an exception that proves the rule ;)
---
changes since v12:
squashed patch by Parav:
switch command_specific_data to u8. useful for example for legacy aq commands, to make sure length is byte granularity.
changes since v11:
acks by Stefan and Lingshan
changes since v10:
explain the role of status vs status_qualifier, addresses
multiple comments
add more status values and appropriate qualifiers,
as suggested by Parav
clarify reserved command opcodes as suggested by Stefan better formatting for ranges, comment by Jiri make sure command-specific-data is a multiple of qword, so things are aligned, suggested by Jiri
add Q_OK qualifier for success
--- admin.tex | 121 +++++++++++++++++++++++++++++++++++++++++++++++ introduction.tex | 3 ++ 2 files changed, 124 insertions(+)
admin.tex +121 -0
@@ -60,4 +60,125 @@ \section{Device groups}\label{sec:Basic Facilities of a Virtio Device / Device g PCI transport (see \ref{sec:Virtio Transport Options / Virtio Over PCI Bus}). \end{description} +\subsection{Group administration commands}\label{sec:Basic Facilities of a Virtio Device / Device groups / Group administration commands} +The driver sends group administration commands to the owner device of +a group to control member devices of the group. +This mechanism can +be used, for example, to configure a member device before it is +initialized by its driver. +\footnote{The term "administration" is intended to be interpreted +widely to include any kind of control. See specific commands +for detail.} + +All the group administration commands are of the following form: + +\begin{lstlisting} +struct virtio_admin_cmd { + /* Device-readable part */ + le16 opcode; + /* + * 1 - SR-IOV + * 2-65535 - reserved + */ + le16 group_type; + /* unused, reserved for future extensions */ + u8 reserved1[12]; + le64 group_member_id; + u8 command_specific_data[]; + + /* Device-writable part */ + le16 status; + le16 status_qualifier; + /* unused, reserved for future extensions */ + u8 reserved2[4]; + u8 command_specific_result[]; +}; +\end{lstlisting} + +For all commands, \field{opcode}, \field{group_type} and if +necessary \field{group_member_id} and \field{command_specific_data} are +set by the driver, and the owner device sets \field{status} and if +needed \field{status_qualifier} and +\field{command_specific_result}. + +Generally, any unused device-readable fields are set to zero by the driver +and ignored by the device. Any unused device-writeable fields are set to zero +by the device and ignored by the driver. + +\field{opcode} specifies the command. The valid +values for \field{opcode} can be found in the following table: + +\begin{tabular}{|l|l|} +\hline +opcode & Name & Command Description \\ +\hline \hline +0x0000 - 0x7FFF & - & Commands using \field{struct virtio_admin_cmd} \\ +\hline +0x8000 - 0xFFFF & - & Reserved for future commands (possibly using a different structure) \\ +\hline +\end{tabular} + +The \field{group_type} specifies the group type identifier. +The \field{group_member_id} specifies the member identifier within the group. +See section \ref{sec:Introduction / Terminology / Device group} +for the definition of the group type identifier and group member +identifier. + +The \field{status} describes the command result and possibly +failure reason at an abstract level, this is appropriate for +forwarding to applications. The \field{status_qualifier} describes +failures at a low virtio specific level, as appropriate for debugging. +The following table describes possible \field{status} values; +to simplify common implementations, they are intentionally +matching common \hyperref[intro:errno]{Linux error names and numbers}: + +\begin{tabular}{|l|l|l|} +\hline +Status (decimal) & Name & Description \\ +\hline \hline +00 & VIRTIO_ADMIN_STATUS_OK & successful completion \\ +\hline +11 & VIRTIO_ADMIN_STATUS_EAGAIN & try again \\ +\hline +12 & VIRTIO_ADMIN_STATUS_ENOMEM & insufficient resources \\ +\hline +22 & VIRTIO_ADMIN_STATUS_EINVAL & invalid command \\ +\hline +other & - & group administration command error \\ +\hline +\end{tabular} + +When \field{status} is VIRTIO_ADMIN_STATUS_OK, \field{status_qualifier} +is reserved and set to zero by the device. + +The following table describes possible \field{status_qualifier} values: +\begin{tabular}{|l|l|l|} +\hline +Status & Name & Description \\ +\hline \hline +0x00 & VIRTIO_ADMIN_STATUS_Q_OK & used with VIRTIO_ADMIN_STATUS_OK \\ +\hline +0x01 & VIRTIO_ADMIN_STATUS_Q_INVALID_COMMAND & command error: no additional information \\ +\hline +0x02 & VIRTIO_ADMIN_STATUS_Q_INVALID_OPCODE & unsupported or invalid \field{opcode} \\ +\hline +0x03 & VIRTIO_ADMIN_STATUS_Q_INVALID_FIELD & unsupported or invalid field within \field{command_specific_data} \\ +\hline +0x04 & VIRTIO_ADMIN_STATUS_Q_INVALID_GROUP & unsupported or invalid \field{group_type} \\ +\hline +0x05 & VIRTIO_ADMIN_STATUS_Q_INVALID_MEMBER & unsupported or invalid \field{group_member_id} \\ +\hline +0x06 & VIRTIO_ADMIN_STATUS_Q_NORESOURCE & out of internal resources: ok to retry \\ +\hline +0x07 & VIRTIO_ADMIN_STATUS_Q_TRYAGAIN & command blocks for too long: should retry \\ +\hline +0x08-0xFFFF & - & reserved for future use \\ +\hline +\end{tabular} + +Each command uses a different \field{command_specific_data} and +\field{command_specific_result} structures and the length of +\field{command_specific_data} and \field{command_specific_result} +depends on these structures and is described separately or is +implicit in the structure description.
introduction.tex +3 -0
@@ -68,6 +68,9 @@ \section{Normative References}\label{sec:Normative References} \phantomsection\label{intro:FUSE}\textbf{[FUSE]} & Linux FUSE interface, \newline\url{https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/include/uapi/linux/fuse.h}\\ + \phantomsection\label{intro:errno}\textbf{[errno]} & + Linux error names and numbers, + \newline\url{https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/include/uapi/asm-generic/errno-base.h}\\ \phantomsection\label{intro:eMMC}\textbf{[eMMC]} & eMMC Electrical Standard (5.1), JESD84-B51, \newline\url{http://www.jedec.org/sites/default/files/docs/JESD84-B51.pdf}\\
Signature · 6 lines
MST --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
PATCH v13 1/10 virtio: document forward compatibility guarantees
In reply to Michael S. TsirkinFeature negotiation forms the basis of forward compatibility guarantees of virtio but has never been properly documented. Do it now.
---
Changes in v11:
make explanation about not using any unspecified bits more detailed as suggested by Cornelia and David
--- content.tex | 44 ++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 44 insertions(+)
content.tex +44 -0
@@ -114,21 +114,65 @@ \section{Feature Bits}\label{sec:Basic Facilities of a Virtio Device / Feature B In particular, new fields in the device configuration space are indicated by offering a new feature bit. +To keep the feature negotiation mechanism extensible, it is +important that devices \em{do not} offer any feature bits that +they would not be able to handle if the driver accepted them +(even though drivers are not supposed to accept any unspecified, +reserved, or unsupported features even if offered, according to +the specification.) Likewise, it is important that drivers \em{do +not} accept feature bits they do not know how to handle (even +though devices are not supposed to offer any unspecified, +reserved, or unsupported features in the first place, +according to the specification.) The preferred +way for handling reserved and unexpected features is that the +driver ignores them. + +In particular, this is +especially important for features limited to specific transports, +as enabling these for more transports in future versions of the +specification is highly likely to require changing the behaviour +from drivers and devices. Drivers and devices supporting +multiple transports need to carefully maintain per-transport +lists of allowed features. + \drivernormative{\subsection}{Feature Bits}{Basic Facilities of a Virtio Device / Feature Bits} The driver MUST NOT accept a feature which the device did not offer, and MUST NOT accept a feature which requires another feature which was not accepted. +The driver MUST validate the feature bits offered by the device. +The driver MUST ignore and MUST NOT accept any feature bit that is +\begin{itemize} +\item not described in this specification, +\item marked as reserved, +\item not supported for the specific transport, +\item not defined for the device type. +\end{itemize} + The driver SHOULD go into backwards compatibility mode if the device does not offer a feature it understands, otherwise MUST set the FAILED \field{device status} bit and cease initialization. +By contrast, the driver MUST NOT fail solely because a feature +it does not understand has been offered by the device. + \devicenormative{\subsection}{Feature Bits}{Basic Facilities of a Virtio Device / Feature Bits} The device MUST NOT offer a feature which requires another feature which was not offered. The device SHOULD accept any valid subset of features the driver accepts, otherwise it MUST fail to set the FEATURES_OK \field{device status} bit when the driver writes it. +The device MUST NOT offer feature bits corresponding to features +it would not support if accepted by the driver (even if the +driver is prohibited from accepting the feature bits by the +specification); for the sake of clarity, this refers to feature +bits not described in this specification, reserved feature bits +and feature bits reserved or not supported for the specific +transport or the specific device type, but this does not preclude +devices written to a future version of this specification from +offering such feature bits should such a specification have a +provision for devices to support the corresponding features. + If a device has successfully negotiated a set of features at least once (by accepting the FEATURES_OK \field{device status} bit during device initialization), then it SHOULD
Signature · 6 lines
MST --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
PATCH v13 4/10 admin: introduce virtio admin virtqueues
In reply to Michael S. TsirkinThe admin virtqueues will be the first interface used to issue admin commands.
Currently the virtio specification defines control virtqueue to manipulate features and configuration of the device it operates on:
virtio-net, virtio-scsi, etc all have existing control virtqueues. However, control virtqueue commands are device type specific, which makes it very difficult to extend for device agnostic commands.
Keeping the device-specific virtqueue separate from the admin virtqueue is simpler and has fewer potential problems. I don't think creating common infrastructure for device-specific control virtqueues across device types worthwhile or within the scope of this patch series.
To support this requirement in a more generic way, this patch introduces a new admin virtqueue interface.
The admin virtqueue can be seen as the virtqueue analog to a transport. The admin queue thus does nothing device type-specific (net, scsi, etc) and instead focuses on transporting the admin commands.
We also support more than one admin virtqueue, for QoS and scalability requirements.
---
since v12:
document that num_queues in blk excludes admin queues reported by Parav
changes since v11:
ack by stefan queues->enqueues to address comment by parav
changes since v10:
explain ordering of commands as suggested by Stefan dropped Max's S.O.B
reword commit log as suggested by David minor wording fixes suggested by David
--- admin.tex | 75 ++++++++++++++++++++++++++++++++ content.tex | 7 ++- device-types/blk/description.tex | 2 +- 3 files changed, 81 insertions(+), 3 deletions(-)
admin.tex +75 -0
@@ -182,3 +182,78 @@ \subsection{Group administration commands}\label{sec:Basic Facilities of a Virti \field{command_specific_data} and \field{command_specific_result} depends on these structures and is described separately or is implicit in the structure description. + +\section{Administration Virtqueues}\label{sec:Basic Facilities of a Virtio Device / Administration Virtqueues} + +An administration virtqueue of an owner device is used to submit +group administration commands. An owner device can have more +than one administration virtqueue. + +If VIRTIO_F_ADMIN_VQ has been negotiated, an owner device exposes one +or more adminstration virtqueues. The number and locations of the +administration virtqueues are exposed by the owner device in a transport +specific manner. + +The driver enqueues requests to an arbitrary administration +virtqueue, and they are used by the device on that same +virtqueue. It is the responsibility of the driver to ensure +strict request ordering for commands, because they will be +consumed with no order constraints. For example, if consistency +is required then the driver can wait for the processing of a +first command by the device to be completed before submitting +another command depending on the first one. + +Administration virtqueues are used as follows: +\begin{itemize} +\item The driver submits the command using the \field{struct virtio_admin_cmd} +structure using a buffer consisting of two parts: a device-readable one followed by a +device-writable one. +\item the device-readable part includes fields from \field{opcode} +through \field{command_specific_data}. +\item the device-writeable buffer includes fields from \field{status} +through \field{command_specific_result} inclusive. +\end{itemize} + +For each command, this specification describes a distinct +format structure used for \field{command_specific_data} and +\field{command_specific_result}, the length of these fields +depends on the command. + +However, to ensure forward compatibility +\begin{itemize} +\item drivers are allowed to submit buffers that are longer +than the device expects +(that is, longer than the length of +\field{opcode} through \field{command_specific_data}). +This allows the driver to maintain +a single format structure even if some structure fields are +unused by the device. +\item drivers are allowed to submit buffers that are shorter +than what the device expects +(that is, shorter than the length of \field{status} through +\field{command_specific_result}). This allows the device to maintain +a single format structure even if some structure fields are +unused by the driver. +\end{itemize} + +The device compares the length of each part (device-readable and +device-writeable) of the buffer as submitted by driver to what it +expects and then silently truncates the structures to either the +length submitted by the driver, or the length described in this +specification, whichever is shorter. The device silently ignores +any data falling outside the shorter of the two lengths. Any +missing fields are interpreted as set to zero. + +Similarly, the driver compares the used buffer length +of the buffer to what it expects and then silently +truncates the structure to the used buffer length. +The driver silently ignores any data falling outside +the used buffer length reported by the device. Any missing +fields are interpreted as set to zero. + +This simplifies driver and device implementations since the +driver/device can simply maintain a single large structure (such +as a C structure) for a command and its result. As new versions +of the specification are designed, new fields can be added to the +tail of a structure, with the driver/device using the full +structure without concern for versioning.
content.tex +5 -2
@@ -99,10 +99,10 @@ \section{Feature Bits}\label{sec:Basic Facilities of a Virtio Device / Feature B \begin{description} \item[0 to 23, and 50 to 127] Feature bits for the specific device type -\item[24 to 40] Feature bits reserved for extensions to the queue and +\item[24 to 41] Feature bits reserved for extensions to the queue and feature negotiation mechanisms -\item[41 to 49, and 128 and above] Feature bits reserved for future extensions. +\item[42 to 49, and 128 and above] Feature bits reserved for future extensions. \end{description} \begin{note}
@@ -849,6 +849,9 @@ \chapter{Reserved Feature Bits}\label{sec:Reserved Feature Bits} that the driver can reset a queue individually. See \ref{sec:Basic Facilities of a Virtio Device / Virtqueues / Virtqueue Reset}. + \item[VIRTIO_F_ADMIN_VQ(41)] This feature indicates that the device exposes one or more + administration virtqueues. + \end{description} \drivernormative{\section}{Reserved Feature Bits}{Reserved Feature Bits}
device-types/blk/description.tex +1 -1
@@ -136,7 +136,7 @@ \subsection{Device configuration layout}\label{sec:Device Types / Block Device / bits as indicated above. The field \field{num_queues} only exists if VIRTIO_BLK_F_MQ is set. This field specifies -the number of queues. +the number of queues\footnote{This excludes administration virtqueues if any are supported. }. The parameters in the configuration space of the device \field{max_discard_sectors} \field{discard_sector_alignment} are expressed in 512-byte units if the
Signature · 18 lines
MST This publicly archived list offers a means to provide input to the OASIS Virtual I/O Device (VIRTIO) TC. In order to verify user consent to the Feedback License terms and to minimize spam in the list archive, subscription is required before posting. Subscribe: [email protected] Unsubscribe: [email protected] List help: [email protected] List archive: https://lists.oasis-open.org/archives/virtio-comment/ Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists Committee: https://www.oasis-open.org/committees/virtio/ Join OASIS: https://www.oasis-open.org/join/
PATCH v13 5/10 pci: add admin vq registers to virtio over pci
In reply to Michael S. TsirkinAdd new registers to the PCI common configuration structure.
These registers will be used for querying the indices of the admin virtqueues of the owner device. To configure, reset or enable the admin virtqueues, the driver should follow existing queue configuration/setup sequence.
---
since v11:
document that admin vqs are not counted with regular vqs
dropped Max's S.O.B
make queue_num not 0 based
--- content.tex | 9 +++++++++ transport-pci.tex | 26 ++++++++++++++++++++++++++ 2 files changed, 35 insertions(+)
content.tex +9 -0
@@ -851,6 +851,15 @@ \chapter{Reserved Feature Bits}\label{sec:Reserved Feature Bits} \item[VIRTIO_F_ADMIN_VQ(41)] This feature indicates that the device exposes one or more administration virtqueues. + At the moment this feature is only supported for devices using + \ref{sec:Virtio Transport Options / Virtio Over PCI + Bus}~\nameref{sec:Virtio Transport Options / Virtio Over PCI Bus} + as the transport and is reserved for future use for + devices using other transports (see + \ref{drivernormative:Basic Facilities of a Virtio Device / Feature Bits} + and + \ref{devicenormative:Basic Facilities of a Virtio Device / Feature Bits} for + handling features reserved for future use. \end{description}
transport-pci.tex +26 -0
@@ -321,6 +321,10 @@ \subsubsection{Common configuration structure layout}\label{sec:Virtio Transport le64 queue_device; /* read-write */ le16 queue_notify_data; /* read-only for driver */ le16 queue_reset; /* read-write */ + + /* About the administration virtqueue. */ + le16 admin_queue_index; /* read-only for driver */ + le16 admin_queue_num; /* read-only for driver */ }; \end{lstlisting}
@@ -347,6 +351,7 @@ \subsubsection{Common configuration structure layout}\label{sec:Virtio Transport \item[\field{num_queues}] The device specifies the maximum number of virtqueues supported here. + This excludes administration virtqueues if any are supported. \item[\field{device_status}] The driver writes the device status here (see \ref{sec:Basic Facilities of a Virtio Device / Device Status Field}). Writing 0 into this
@@ -406,6 +411,19 @@ \subsubsection{Common configuration structure layout}\label{sec:Virtio Transport This field exists only if VIRTIO_F_RING_RESET has been negotiated. (see \ref{sec:Basic Facilities of a Virtio Device / Virtqueues / Virtqueue Reset}). +\item[\field{admin_queue_index}] + The device uses this to report the index of the first administration virtqueue. + This field is valid only if VIRTIO_F_ADMIN_VQ has been negotiated. +\item[\field{admin_queue_num}] + The device uses this to report the number of the + supported administration virtqueues. + Virtqueues with index + between \field{admin_queue_index} and (\field{admin_queue_index} + + \field{admin_queue_num} - 1) inclusive serve as administration + virtqueues. + The value 0 indicates no supported administration virtqueues. + This field is valid only if VIRTIO_F_ADMIN_VQ has been + negotiated. \end{description} \devicenormative{\paragraph}{Common configuration structure layout}{Virtio Transport Options / Virtio Over PCI Bus / PCI Device Layout / Common configuration structure layout}
@@ -492,6 +510,14 @@ \subsubsection{Common configuration structure layout}\label{sec:Virtio Transport were used before the queue reset. (see \ref{sec:Basic Facilities of a Virtio Device / Virtqueues / Virtqueue Reset}). +If VIRTIO_F_ADMIN_VQ has been negotiated, and if the driver +configures any administration virtqueues, the driver MUST +configure the administration virtqueues using the index +in the range \field{admin_queue_index} to +\field{admin_queue_index} + \field{admin_queue_num} - 1 inclusive. +The driver MAY configure less administration virtqueues than +supported by the device. + \subsubsection{Notification structure layout}\label{sec:Virtio Transport Options / Virtio Over PCI Bus / PCI Device Layout / Notification capability} The notification location is found using the VIRTIO_PCI_CAP_NOTIFY_CFG
Signature · 18 lines
MST This publicly archived list offers a means to provide input to the OASIS Virtual I/O Device (VIRTIO) TC. In order to verify user consent to the Feedback License terms and to minimize spam in the list archive, subscription is required before posting. Subscribe: [email protected] Unsubscribe: [email protected] List help: [email protected] List archive: https://lists.oasis-open.org/archives/virtio-comment/ Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists Committee: https://www.oasis-open.org/committees/virtio/ Join OASIS: https://www.oasis-open.org/join/
PATCH v13 6/10 mmio: document ADMIN_VQ as reserved
In reply to Michael S. TsirkinAdding relevant registers needs more work and it's not clear what the use-case will be as currently only the PCI transport is supported. But let's keep the door open on this.
We already say it's reserved in a central place, but it does not hurt to remind implementers to mask it.
---
changes in v12:
drop "as of now" to address comment by parav
--- transport-mmio.tex | 12 ++++++++++++ 1 file changed, 12 insertions(+)
transport-mmio.tex +12 -0
@@ -562,3 +562,15 @@ \subsection{Legacy interface}\label{sec:Virtio Transport Options / Virtio Over M \end{enumerate} Notification mechanisms did not change. + +\subsection{Features reserved for future use}\label{sec:Virtio Transport Options / Virtio Over MMIO / Features reserved for future use} + +Devices and drivers utilizing Virtio Over MMIO +do not support the following features: +\begin{itemize} + +\item VIRTIO_F_ADMIN_VQ + +\end{itemize} + +These features are reserved for future use.
Signature · 18 lines
MST This publicly archived list offers a means to provide input to the OASIS Virtual I/O Device (VIRTIO) TC. In order to verify user consent to the Feedback License terms and to minimize spam in the list archive, subscription is required before posting. Subscribe: [email protected] Unsubscribe: [email protected] List help: [email protected] List archive: https://lists.oasis-open.org/archives/virtio-comment/ Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists Committee: https://www.oasis-open.org/committees/virtio/ Join OASIS: https://www.oasis-open.org/join/
PATCH v13 8/10 admin: command list discovery
In reply to Michael S. TsirkinAdd commands to find out which commands does each group support, as well as enable their use by driver.
This will be especially useful once we have multiple group types.
An alternative is per-type VQs. This is possible but will require more per-transport work. Discovery through the vq helps keep things contained.
e.g. lack of support for some command can switch to a legacy mode
note that commands are expected to be avolved by adding new fields to command specific data at the tail, so
we generally do not need feature bits for compatibility.
---
changes in v12:
devices->device address comment by parav
changes in v11:
dropped S.O.B by Max
explain in commit log how commands will evolve, comment by Stefan replace will use with is capable of use, comment by Stefan typo fix reported by David and Lingshan
rename cmds to cmd_opcodes suggested by Jiri
list group_type explicitly in command description suggested by Jiri clarify how can device not support some commands
always say group administration commands, comment by Jiri
--- admin.tex | 102 +++++++++++++++++++++++++++++++++++++++++++++++++++++- 1 file changed, 101 insertions(+), 1 deletion(-)
admin.tex +101 -1
@@ -113,7 +113,9 @@ \subsection{Group administration commands}\label{sec:Basic Facilities of a Virti \hline opcode & Name & Command Description \\ \hline \hline -0x0000 - 0x7FFF & - & Commands using \field{struct virtio_admin_cmd} \\ +0x0000 & VIRTIO_ADMIN_CMD_LIST_QUERY & Provides to driver list of commands supported for this group type \\ +0x0001 & VIRTIO_ADMIN_CMD_LIST_USE & Provides to device list of commands used for this group type \\ +0x0002 - 0x7FFF & - & Commands using \field{struct virtio_admin_cmd} \\ \hline 0x8000 - 0xFFFF & - & Reserved for future commands (possibly using a different structure) \\ \hline
@@ -183,6 +185,104 @@ \subsection{Group administration commands}\label{sec:Basic Facilities of a Virti depends on these structures and is described separately or is implicit in the structure description. +Before sending any group administration commands to the device, the driver +needs to communicate to the device which commands it is going to +use. Initially (after reset), only two commands are assumed to be used: +VIRTIO_ADMIN_CMD_LIST_QUERY and VIRTIO_ADMIN_CMD_LIST_USE. + +Before sending any other commands for any member of a specific group to +the device, the driver queries the supported commands via +VIRTIO_ADMIN_CMD_LIST_QUERY and sends the commands it is +capable of using via VIRTIO_ADMIN_CMD_LIST_USE. + +Commands VIRTIO_ADMIN_CMD_LIST_QUERY and +VIRTIO_ADMIN_CMD_LIST_USE both use the following structure for +\field{command_specific_result} and \field{command_specific_data} +respectively describing the command opcodes: + +\begin{lstlisting} +struct virtio_admin_cmd_list { + /* Indicates which of the below fields were returned + le64 device_admin_cmd_opcodes[]; +}; +\end{lstlisting} + +This structure is an array of 64 bit values in little-endian byte +order, in which a bit is set if the specific command opcode +is supported. Thus, \field{device_admin_cmd_opcodes[0]} refers to the +first 64-bit value in this array corresponding to opcodes 0 to +63, \field{device_admin_cmd_opcodes[1]} is the second 64-bit value +corresponding to opcodes 64 to 127, etc. +For example, the array of size 2 including +the values 0x3 in \field{device_admin_cmd_opcodes[0]} +and 0x1 in \field{device_admin_cmd_opcodes[1]} indicates that only +opcodes 0, 1 and 64 are supported. +The length of the array depends on the supported opcodes - it is +large enough to include bits set for all supported opcodes, +that is the length can be calculated by starting with the largest +supported opcode adding one, dividing by 64 and rounding up. +In other words, for +VIRTIO_ADMIN_CMD_LIST_QUERY and VIRTIO_ADMIN_CMD_LIST_USE the +length of \field{command_specific_result} and +\field{command_specific_data} respectively will be +$DIV_ROUND_UP(max_cmd, 64) * 8$ where DIV_ROUND_UP is integer division +with round up and \field{max_cmd} is the largest available command opcode. + +The array is also allowed to be larger and to additionally +include an arbitrary number of all-zero entries. + +Accordingly, bits 0 and 1 corresponding to opcode 0 +(VIRTIO_ADMIN_CMD_LIST_QUERY) and 1 +(VIRTIO_ADMIN_CMD_LIST_USE) are +always set in \field{device_admin_cmd_opcodes[0]} returned by VIRTIO_ADMIN_CMD_LIST_QUERY. + +For the command VIRTIO_ADMIN_CMD_LIST_QUERY, \field{opcode} is set to 0x0. +The \field{group_member_id} is unused. It is set to zero by driver. +This command has no command specific data. +The device, upon success, returns a result in +\field{command_specific_result} in the format +\field{struct virtio_admin_cmd_list} describing the +list of group administration commands supported for the group type +specified by \field{group_type}. + +For the command VIRTIO_ADMIN_CMD_LIST_USE, \field{opcode} +is set to 0x1. +The \field{group_member_id} is unused. It is set to zero by driver. +The \field{command_specific_data} is in the format +\field{struct virtio_admin_cmd_list} describing the +list of group administration commands used by the driver +with the group type specified by \field{group_type}. + +This command has no command specific result. + +The driver issues the command VIRTIO_ADMIN_CMD_LIST_QUERY to +query the list of commands valid for this group and before sending +any commands for any member of a group. + +The driver then enables use of some of the opcodes by sending to +the device the command VIRTIO_ADMIN_CMD_LIST_USE with a subset +of the list returned by VIRTIO_ADMIN_CMD_LIST_QUERY that is +both understood and used by the driver. + +If the device supports the command list used by the driver, the +device completes the command with status VIRTIO_ADMIN_STATUS_OK. +If the device does not support the command list +(for example, if the driver is not capable to use +some required commands), the device +completes the command with status +VIRTIO_ADMIN_STATUS_INVALID_FIELD. + +Note: the driver is assumed not to set bits in +device_admin_cmd_opcodes +if it is not familiar with how the command opcode +is used, since the device could have dependencies between +command opcodes. + +It is assumed that all members in a group support and are used +with the same list of commands. However, for owner devices +supporting multiple group types, the list of supported commands +might differ between different group types. + \section{Administration Virtqueues}\label{sec:Basic Facilities of a Virtio Device / Administration Virtqueues} An administration virtqueue of an owner device is used to submit
Signature · 18 lines
MST This publicly archived list offers a means to provide input to the OASIS Virtual I/O Device (VIRTIO) TC. In order to verify user consent to the Feedback License terms and to minimize spam in the list archive, subscription is required before posting. Subscribe: [email protected] Unsubscribe: [email protected] List help: [email protected] List archive: https://lists.oasis-open.org/archives/virtio-comment/ Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists Committee: https://www.oasis-open.org/committees/virtio/ Join OASIS: https://www.oasis-open.org/join/
PATCH v13 10/10 ccw: document more reserved features
In reply to Michael S. Tsirkinvq reset and shared memory are unsupported, too.
---
changes in v12:
address comment by parav
--- transport-ccw.tex | 2 ++ 1 file changed, 2 insertions(+)
transport-ccw.tex +2 -0
@@ -607,6 +607,8 @@ \subsection{Features reserved for future use}\label{sec:Virtio Transport Options \begin{itemize} \item VIRTIO_F_ADMIN_VQ +\item VIRTIO_F_RING_RESET +\item Shared memory regions including VIRTIO_PMEM_F_SHMEM_REGION \end{itemize}
Signature · 18 lines
MST This publicly archived list offers a means to provide input to the OASIS Virtual I/O Device (VIRTIO) TC. In order to verify user consent to the Feedback License terms and to minimize spam in the list archive, subscription is required before posting. Subscribe: [email protected] Unsubscribe: [email protected] List help: [email protected] List archive: https://lists.oasis-open.org/archives/virtio-comment/ Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists Committee: https://www.oasis-open.org/committees/virtio/ Join OASIS: https://www.oasis-open.org/join/
PATCH v13 9/10 admin: conformance clauses
In reply to Michael S. TsirkinAdd conformance clauses for admin commands and admin virtqueues.
---
changes in v13:
squashed patch by Parav:
admin: Remove dependency on the deprecated register
'VF Migration Capable' is already deprecated as per PCI base specification 6.0.1 section 9.3.3.2.1. It was long ago deprecated which was meant for MR-IOV.
Hence, remove virtio PCI specification to not depend on such deprecated bits.
changes in v12:
add device conformance statements to address comment by Stefan wording tweak to address comment by Stefan
clarify: is being processed -> has been submitted and not yet completed. comment by lingshan
changes in v11:
typo and wording fixes reported by David
clarify cmd list use can be repeated but not in parallel with other commands, and returning same value across uses - comment by Lingshan add conformance clauses for new error codes
--- admin.tex | 225 ++++++++++++++++++++++++++++++++++++++++++++++ transport-pci.tex | 9 +- 2 files changed, 233 insertions(+), 1 deletion(-)
admin.tex +225 -0
@@ -283,6 +283,148 @@ \subsection{Group administration commands}\label{sec:Basic Facilities of a Virti supporting multiple group types, the list of supported commands might differ between different group types. +\devicenormative{\paragraph}{Group administration commands}{Basic Facilities of a Virtio Device / Device groups / Group administration commands} + +The device MUST validate \field{opcode}, \field{group_type} and +\field{group_member_id}, and if any of these has an invalid or +unsupported value, set \field{status} to +VIRTIO_ADMIN_STATUS_EINVAL and set \field{status_qualifier} +accordingly: +\begin{itemize} +\item if \field{group_type} is invalid, \field{status_qualifier} + MUST be set to VIRTIO_ADMIN_STATUS_Q_INVALID_GROUP; +\item otherwise, if \field{opcode} is invalid, + \field{status_qualifier} MUST be set to + VIRTIO_ADMIN_STATUS_Q_INVALID_OPCODE; +\item otherwise, if \field{group_member_id} is used by the + specific command and is invalid, \field{status_qualifier} MUST be + set to VIRTIO_ADMIN_STATUS_Q_INVALID_MEMBER. +\end{itemize} + +If a command completes successfully, the device MUST set +\field{status} to VIRTIO_ADMIN_STATUS_OK. + +If a command fails, the device MUST set +\field{status} to a value different from VIRTIO_ADMIN_STATUS_OK. + +If \field{status} is set to VIRTIO_ADMIN_STATUS_EINVAL, the +device state MUST NOT change, that is the command MUST NOT have +any side effects on the device, in particular the device MUST NOT +enter an error state as a result of this command. + +If a command fails, the device state generally SHOULD NOT change, +as far as possible. + +The device MAY enforce additional restrictions and dependencies on +opcodes used by the driver and MAY fail the command +VIRTIO_ADMIN_CMD_LIST_USE with \field{status} set to VIRTIO_ADMIN_STATUS_EINVAL +and \field{status_qualifier} set to VIRTIO_ADMIN_STATUS_Q_INVALID_FIELD +if the list of commands used violate internal device dependencies. + +If the device supports multiple group types, commands for each group +type MUST operate independently of each other, in particular, +the device MAY return different results for VIRTIO_ADMIN_CMD_LIST_QUERY +for different group types. + +After reset, if the device supports a given group type +and before receiving VIRTIO_ADMIN_CMD_LIST_USE for this group type +the device MUST assume +that the list of legal commands used by the driver consists of +the two commands VIRTIO_ADMIN_CMD_LIST_QUERY and VIRTIO_ADMIN_CMD_LIST_USE. + +After completing VIRTIO_ADMIN_CMD_LIST_USE successfully, +the device MUST set the list of legal commands used by the driver +to the one supplied in \field{command_specific_data}. + +The device MUST validate commands against the list used by +the driver and MUST fail any commands not in the list with +\field{status} set to VIRTIO_ADMIN_STATUS_EINVAL +and \field{status_qualifier} set to +VIRTIO_ADMIN_STATUS_Q_INVALID_OPCODE. + +The list of supported commands reported by the device MUST NOT +shrink (but MAY expand): after reporting a given command as +supported through VIRTIO_ADMIN_CMD_LIST_QUERY the device MUST NOT +later report it as unsupported. Further, after a given set of +commands has been used (via a successful +VIRTIO_ADMIN_CMD_LIST_USE), then after a device or system reset +the device SHOULD complete successfully any following calls to +VIRTIO_ADMIN_CMD_LIST_USE with the same list of commands; if this +command VIRTIO_ADMIN_CMD_LIST_USE fails after a device or system +reset, the device MUST not fail it solely because of the command +list used. Failure to do so would interfere with resuming from +suspend and error recovery. Exceptions MAY apply if the system +configuration assures, in some way, that the driver does not +cache the previous value of VIRTIO_ADMIN_CMD_LIST_USE, +such as in the case of a firmware upgrade or downgrade. + +When processing a command with the SR-IOV group type, +if the device does not have an SR-IOV Extended Capability or +if \field{VF Enable} is clear +then the device MUST fail all commands with +\field{status} set to VIRTIO_ADMIN_STATUS_EINVAL and +\field{status_qualifier} set to +VIRTIO_ADMIN_STATUS_Q_INVALID_GROUP; +otherwise, if \field{group_member_id} is not +between $1$ and \field{NumVFs} inclusive, +the device MUST fail all commands with +\field{status} set to VIRTIO_ADMIN_STATUS_EINVAL and +\field{status_qualifier} set to +VIRTIO_ADMIN_STATUS_Q_INVALID_MEMBER; +\field{NumVFs} and \field{VF Enable} refer to registers within the +SR-IOV Extended Capability as specified by \hyperref[intro:PCIe]{[PCIe]}. + +\drivernormative{\paragraph}{Group administration commands}{Basic Facilities of a Virtio Device / Device groups / Group administration commands} + +The driver MAY discover whether device supports a specific group type +by issuing VIRTIO_ADMIN_CMD_LIST_QUERY with the matching +\field{group_type}. + +The driver MUST issue VIRTIO_ADMIN_CMD_LIST_USE +and wait for it to be completed with status +VIRTIO_ADMIN_STATUS_OK before issuing any commands +(except for the initial VIRTIO_ADMIN_CMD_LIST_QUERY +and VIRTIO_ADMIN_CMD_LIST_USE). + +The driver MAY issue VIRTIO_ADMIN_CMD_LIST_USE any number +of times but MUST NOT issue VIRTIO_ADMIN_CMD_LIST_USE commands +if any other command has been submitted to the +device and has not yet completed processing by the device. + +The driver SHOULD NOT set bits in device_admin_cmd_opcodes +if it is not familiar with how the command opcode +is used, as dependencies between command opcodes might exist. + +The driver MUST NOT request (via VIRTIO_ADMIN_CMD_LIST_USE) +the use of any commands not previously reported as +supported for the same group type by VIRTIO_ADMIN_CMD_LIST_QUERY. + +The driver MUST NOT use any commands for a given group type +before sending VIRTIO_ADMIN_CMD_LIST_USE with the correct +list of command opcodes and group type. + +The driver MAY block use of VIRTIO_ADMIN_CMD_LIST_QUERY and +VIRTIO_ADMIN_CMD_LIST_USE by issuing VIRTIO_ADMIN_CMD_LIST_USE +with respective bits cleared in \field{command_specific_data}. + +The driver MUST handle a command error with a reserved \field{status} +value in the same way as \field{status} set to VIRTIO_ADMIN_STATUS_EINVAL +(except possibly for different error reporting/diagnostic messages). + +The driver MUST handle a command error with a reserved +\field{status_qualifier} value in the same way as +\field{status_qualifier} set to +VIRTIO_ADMIN_STATUS_Q_INVALID_COMMAND (except possibly for +different error reporting/diagnostic messages). + +When sending commands with the SR-IOV group type, +the driver specify a value for \field{group_member_id} +between $1$ and \field{NumVFs} inclusive, +the driver MUST also make sure that as long as any such command +is outstanding, is clear and \field{VF Enable} is set; +\field{NumVFs} and \field{VF Enable} refer to registers within the +SR-IOV Extended Capability as specified by \hyperref[intro:PCIe]{[PCIe]}. + \section{Administration Virtqueues}\label{sec:Basic Facilities of a Virtio Device / Administration Virtqueues} An administration virtqueue of an owner device is used to submit
@@ -357,3 +499,86 @@ \section{Administration Virtqueues}\label{sec:Basic Facilities of a Virtio Devic of the specification are designed, new fields can be added to the tail of a structure, with the driver/device using the full structure without concern for versioning. + +\devicenormative{\paragraph}{Group administration commands}{Basic Facilities of a Virtio Device / Administration virtqueues} + +The device MUST support device-readable and device-writeable buffers +shorter than described in this specification, by +\begin{enumerate} +\item acting as if any data that would be read outside the +device-readable buffers is set to zero, and +\item discarding data that would be written outside the +specified device-writeable buffers. +\end{enumerate} + +The device MUST support device-readable and device-writeable buffers +longer than described in this specification, by +\begin{enumerate} +\item ignoring any data in device-readable buffers outside +the expected length, and +\item only writing the expected structure to the device-writeable +buffers, ignoring any extra buffers, and reporting the +actual length of data written, in bytes, +as buffer used length. +\end{enumerate} + +The device SHOULD initialize the device-writeable buffer +up to the length of the structure described by this specification or +the length of the buffer supplied by the driver (even if the buffer is +all set to zero), whichever is shorter. + +The device MUST NOT fail a command solely because the buffers +provided are shorter or longer than described in this +specification. + +The device MUST initialize the device-writeable part of +\field{struct virtio_admin_cmd} that is a multiple of 64 bit in +size. + +The device MUST initialize \field{status} and +\field{status_qualifier} in \field{struct virtio_admin_cmd}. + +The device MUST process commands on a given administration virtqueue +in the order in which they are queued. + +If multiple administration virtqueues have been configured, +device MAY process commands on distinct virtqueues with +no order constraints. + +If the device sets \field{status} to either VIRTIO_ADMIN_STATUS_EAGAIN +or VIRTIO_ADMIN_STATUS_ENOMEM, then the command MUST NOT +have any side effects, making it safe to retry. + +\drivernormative{\paragraph}{Group administration commands}{Basic Facilities of a Virtio Device / Administration virtqueues} + +The driver MAY supply device-readable or device-writeable parts +of \field{struct virtio_admin_cmd} that are longer than described in +this specification. + +The driver SHOULD supply device-readable part of +\field{struct virtio_admin_cmd} that is at least as +large as the structure described by this specification +(even if the structure is all set to zero). + +The driver MUST supply both device-readable or device-writeable parts +of \field{struct virtio_admin_cmd} that are a multiple of 64 bit +in length. + +The device MUST supply both device-readable or device-writeable parts +of \field{struct virtio_admin_cmd} that are larger than zero in +length. However, \field{command_specific_data} and +\field{command_specific_result} MAY be zero in length, unless +specified otherwise for the command. + +The driver MUST NOT assume that the device will initialize the whole +device-writeable part of \field{struct virtio_admin_cmd} as described in the specification; instead, +the driver MUST act as if the structure +outside the part of the buffer used by the device +is set to zero. + +If multiple administration virtqueues have been configured, +the driver MUST ensure ordering for commands +placed on different administration virtqueues. + +The driver SHOULD retry a command that completed with +\field{status} VIRTIO_ADMIN_STATUS_EAGAIN.
transport-pci.tex +8 -1
@@ -484,6 +484,13 @@ \subsubsection{Common configuration structure layout}\label{sec:Virtio Transport present either a value of 0 or a power of 2 in \field{queue_size}. +If VIRTIO_F_ADMIN_VQ has been negotiated, the value +\field{admin_queue_index} MUST be equal to, or bigger than +\field{num_queues}; also, \field{admin_queue_num} MUST be +smaller than, or equal to 0x10000 - \field{admin_queue_index}, +to ensure that indices of valid admin queues fit into +a 16 bit range beyond all other virtqueues. + \drivernormative{\paragraph}{Common configuration structure layout}{Virtio Transport Options / Virtio Over PCI Bus / PCI Device Layout / Common configuration structure layout} The driver MUST NOT write to \field{device_feature}, \field{num_queues}, \field{config_generation}, \field{queue_notify_off} or \field{queue_notify_data}.
@@ -515,7 +522,7 @@ \subsubsection{Common configuration structure layout}\label{sec:Virtio Transport configure the administration virtqueues using the index in the range \field{admin_queue_index} to \field{admin_queue_index} + \field{admin_queue_num} - 1 inclusive. -The driver MAY configure less administration virtqueues than +The driver MAY configure fewer administration virtqueues than supported by the device. \subsubsection{Notification structure layout}\label{sec:Virtio Transport Options / Virtio Over PCI Bus / PCI Device Layout / Notification capability}
Signature · 18 lines
MST This publicly archived list offers a means to provide input to the OASIS Virtual I/O Device (VIRTIO) TC. In order to verify user consent to the Feedback License terms and to minimize spam in the list archive, subscription is required before posting. Subscribe: [email protected] Unsubscribe: [email protected] List help: [email protected] List archive: https://lists.oasis-open.org/archives/virtio-comment/ Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists Committee: https://www.oasis-open.org/committees/virtio/ Join OASIS: https://www.oasis-open.org/join/
PATCH v13 7/10 ccw: document ADMIN_VQ as reserved
In reply to Michael S. TsirkinAdding relevant registers needs more work and it's not clear what the use-case will be as currently only the PCI transport is supported. But let's keep the door open on this.
We already say it's reserved in a central place, but it does not hurt to remind implementers to mask it.
Note: there are more features to add to this list. Will be done later with a patch on top.
since v12:
drop "at this time" comment by parav
--- transport-ccw.tex | 12 ++++++++++++ 1 file changed, 12 insertions(+)
transport-ccw.tex +12 -0
@@ -599,3 +599,15 @@ \subsubsection{Resetting Devices}\label{sec:Virtio Transport Options / Virtio ov successful conclusion of the CCW_CMD_VDEV_RESET channel command, although it MAY also choose to verify reset completion by reading \field{device status} via CCW_CMD_READ_STATUS and checking whether it is 0 afterwards. + +\subsection{Features reserved for future use}\label{sec:Virtio Transport Options / Virtio over channel I/O / Features reserved for future use} + +Devices and drivers utilizing Virtio over channel I/O +do not support the following features: +\begin{itemize} + +\item VIRTIO_F_ADMIN_VQ + +\end{itemize} + +These features are reserved for future use.
Signature · 18 lines
MST This publicly archived list offers a means to provide input to the OASIS Virtual I/O Device (VIRTIO) TC. In order to verify user consent to the Feedback License terms and to minimize spam in the list archive, subscription is required before posting. Subscribe: [email protected] Unsubscribe: [email protected] List help: [email protected] List archive: https://lists.oasis-open.org/archives/virtio-comment/ Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists Committee: https://www.oasis-open.org/committees/virtio/ Join OASIS: https://www.oasis-open.org/join/
Quoted 69 lines · Michael S. Tsirkin
Change log:
since 13:
command specific data is u8 again
exclude admin queues in blk'snum_queues
minor other tweakssince 11:
addressed lots of comments, all minor. consistency with outstandingnumber->indexandqueue->enqueuework i did not intentionally drop any reviewed-by tags as all changes are minor - if yours is missing it is because I forgot to record it, sorryone "breaking" change in response to stefan's comment:
in patch 5,num_queueshas been specified not to include admin
queues: just regular ones.since v10:
addressed lots of comments by Jiri, Stefan. Cornelia, Lngshan, Parav, Maxsince v9:
addressed comments by Parav, Max, Cornelia, David and Zhu Lingshan:
added link to errno header from Linux
rename_MEMto_MEMBER
admin vq num is zero based
clarify who sends commands where
minor english tweaks
clarify command length
specify interaction with sriov capability correct commit log - NumVFs can be 0i could not decide what should happen when VFs are disabled. for now did not specify.
since v8:
addressed comments by Cornelia - as we agreed on listsince v7:
make high level error codes match linux, with virtio specific codes
in a separate field
renamed_ACCEPTto_USEsince that's what it does clarified forward compatibility and non pci transports support multiple admin vqs
conformance statements
lots of changes all over the place to I changed author from Max to myself. Don't need to take credit but also don't want to blame Max for my mistakes.since v6:
- removed some extentions intended for future use. We'll do them when we get there.
- brought back command list query from v5 in a simplified form - it's here to address the case where a single parent can address multiple groups, such as PF addressing transport vq and sriov vfs.
- attempt to make terminology more formal. In particular a term for whoever controls the group. I am still going back
and forth between "parent" and "owner" - owner might be better after all since it will work if we ever have a self group. For now it's parent.
Quoted 7 lines
TODO (maybe?) - probably ok to defer until this part is upstream:
Add "all members" member id.
Add commands for MSI, feature discovery.
Add commands for transport vq.
As there have only been tiny tweaks since v12, I'm starting a vote now.
Quoted 32 lines
My intent is to try and support both SR-IOV and SIOV usecases with the same structure and maybe even the same VQ.
For example, it might make sense to split creating/destroying SIOV devices from the transport passing data from the guest - the driver would then not negotiate
VIRTIO_F_SR_IOV(which then means auto-provisioning).More ideas for use-cases:
virtio VF features query and configuration space provisioning virtio VF resource (queues, msix vectors count) provisioningFuture directions (shouldn't block this patch)
- aborting commands - left for later. or is vq reset enough?
- should we rename structures from admin to group admin?Michael S. Tsirkin (10):
virtio: document forward compatibility guarantees
admin: introduce device group and related concepts
admin: introduce group administration commands
admin: introduce virtio admin virtqueues
pci: add admin vq registers to virtio over pci
mmio: documentADMIN_VQas reserved
ccw: documentADMIN_VQas reserved
admin: command list discovery
admin: conformance clauses
ccw: document more reserved features
Quoted 12 lines
admin.tex | 584 +++++++++++++++++++++++++++++++ content.tex | 62 +++- device-types/blk/description.tex | 2 +- introduction.tex | 3 + transport-ccw.tex | 14 + transport-mmio.tex | 12 + transport-pci.tex | 33 ++ 7 files changed, 707 insertions(+), 3 deletions(-) create mode 100644 admin.texSignature · 1 line
MST
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
[virtio-comment] Re: [PATCH v13 03/10] admin: introduce group administration commands
In reply to Michael S. TsirkinHi Michael,
Quoted 60 lines · Michael S. Tsirkin
+\field{opcode} specifies the command. The valid
admin.tex +59
-0+values for \field{opcode} can be found in the following table: + +\begin{tabular}{|l|l|} +\hline +opcode & Name & Command Description \\ +\hline \hline +0x0000 - 0x7FFF & - & Commands using \field{struct virtio_admin_cmd} \\ +\hline +0x8000 - 0xFFFF & - & Reserved for future commands (possibly using a different structure) \\ +\hline +\end{tabular} + +The \field{group_type} specifies the group type identifier. +The \field{group_member_id} specifies the member identifier within the group. +See section \ref{sec:Introduction / Terminology / Device group} +for the definition of the group type identifier and group member +identifier. + +The \field{status} describes the command result and possibly +failure reason at an abstract level, this is appropriate for +forwarding to applications. The \field{status_qualifier} describes +failures at a low virtio specific level, as appropriate for debugging. +The following table describes possible \field{status} values; +to simplify common implementations, they are intentionally +matching common \hyperref[intro:errno]{Linux error names and numbers}: + +\begin{tabular}{|l|l|l|} +\hline +Status (decimal) & Name & Description \\ +\hline \hline +00 & VIRTIO_ADMIN_STATUS_OK & successful completion \\ +\hline +11 & VIRTIO_ADMIN_STATUS_EAGAIN & try again \\ +\hline +12 & VIRTIO_ADMIN_STATUS_ENOMEM & insufficient resources \\ +\hline +22 & VIRTIO_ADMIN_STATUS_EINVAL & invalid command \\ +\hline +other & - & group administration command error \\ +\hline +\end{tabular} + +When \field{status} is VIRTIO_ADMIN_STATUS_OK, \field{status_qualifier} +is reserved and set to zero by the device. + +The following table describes possible \field{status_qualifier} values: +\begin{tabular}{|l|l|l|} +\hline +Status & Name & Description \\ +\hline \hline +0x00 & VIRTIO_ADMIN_STATUS_Q_OK & used with VIRTIO_ADMIN_STATUS_OK \\ +\hline +0x01 & VIRTIO_ADMIN_STATUS_Q_INVALID_COMMAND & command error: no additional information \\ +\hline +0x02 & VIRTIO_ADMIN_STATUS_Q_INVALID_OPCODE & unsupported or invalid \field{opcode} \\ +\hline +0x03 & VIRTIO_ADMIN_STATUS_Q_INVALID_FIELD & unsupported or invalid field within \field{command_specific_data} \\ +\hline +0x04 & VIRTIO_ADMIN_STATUS_Q_INVALID_GROUP & unsupported or invalid \field{group_type} \\
admin.tex \section{Device groups}\label{sec:Basic Facilities of a Virtio Device / Device g · line 178
Quoted 10 lines
admin.tex +10
-0+\hline +0x05 & VIRTIO_ADMIN_STATUS_Q_INVALID_MEMBER & unsupported or invalid \field{group_member_id} \\ +\hline +0x06 & VIRTIO_ADMIN_STATUS_Q_NORESOURCE & out of internal resources: ok to retry \\ +\hline +0x07 & VIRTIO_ADMIN_STATUS_Q_TRYAGAIN & command blocks for too long: should retry \\ +\hline +0x08-0xFFFF & - & reserved for future use \\ +\hline +\end{tabular}
Did you miss taking my 3 changes here I posted at [1]?
https://lists.oasis-open.org/archives/virtio-comment/202304/msg00532.html
Or you sent some older version by mistake?
or they got fixed in some later hunk in this series?
Without this changes pdf generation is broken and manual intervention of script generates pdf table which is messed up.
It is not too late, to fix and resend quickly.
Or probably list down the known issues and I will create the follow up patches as these are mandatory non cosmetic editorial changes.
Please let me know how shall we proceed.
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
[virtio-comment] RE: [virtio] Re: [PATCH v13 03/10] admin: introduce group administration commands
Quoted 16 lines
From: [email protected] <[email protected]> On Behalf Of Parav Pandit
Sent: Friday, May 5, 2023 12:03 PMDid you miss taking my 3 changes here I posted at [1]?
https://lists.oasis-open.org/archives/virtio-comment/202304/msg00532.htmlOr you sent some older version by mistake?
or they got fixed in some later hunk in this series?
Without this changes pdf generation is broken and manual intervention of script generates pdf table which is messed up.
Quoted 6 lines
It is not too late, to fix and resend quickly.
Or probably list down the known issues and I will create the follow up patches as these are mandatory non cosmetic editorial changes.
Please let me know how shall we proceed.
I see you already started voting. Ok.
We can do below editorial follow up change as posted [2].
[2] https://lore.kernel.org/virtio-comment/[email protected]/T/#u
[virtio-comment] Re: [PATCH v13 02/10] admin: introduce device group and related concepts
In reply to Michael S. TsirkinQuoted 19 lines · Michael S. Tsirkin
Each device group has a type. For now, define one initial group type:
SR-IOV type - PCI SR-IOV virtual functions (VFs) of a given PCI SR-IOV physical function (PF). This group may contain zero or more virtio devices according to NumVFs configured.
Each device within a group has a unique identifier. This identifier is the group member identifier.
Note: one can argue both ways whether the new device group handling functionality (this and following patches) is closer
to a new device type or a new transport type.However, it's expected that we will add more features in the near future. To facilitate this as much as possible of the text is located in the new admin chapter.
Effort was made to minimize transport-specific text.
Quoted 8 lines
There's a bit of duplication with
0x1repeated twice and no special section for group type identifiers.
It seems ok to defer adding these until we have more group types.Based-on-patch-by Max Gurtovoy <[email protected]>Signed-off-by Michael S. Tsirkin <[email protected]>Reviewed-by Stefan Hajnoczi <[email protected]>
This publicly archived list offers a means to provide input to the OASIS Virtual I/O Device (VIRTIO) TC.
In order to verify user consent to the Feedback License terms and to minimize spam in the list archive, subscription is required before posting.
Subscribe: [email protected]
Unsubscribe: [email protected]
List help: [email protected]
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/
[virtio-comment] Re: [PATCH v13 04/10] admin: introduce virtio admin virtqueues
In reply to Michael S. TsirkinQuoted 19 lines · Michael S. Tsirkin
The admin virtqueues will be the first interface used to issue admin commands.
Currently the virtio specification defines control virtqueue to manipulate features and configuration of the device it operates on:
virtio-net, virtio-scsi, etc all have existing control virtqueues. However, control virtqueue commands are device type specific, which makes it very difficult to extend for device agnostic commands.Keeping the device-specific virtqueue separate from the admin virtqueue is simpler and has fewer potential problems. I don't think creating common infrastructure for device-specific control virtqueues across device types worthwhile or within the scope of this patch series.
To support this requirement in a more generic way, this patch introduces a new admin virtqueue interface.
The admin virtqueue can be seen as the virtqueue analog to a transport. The admin queue thus does nothing device type-specific (net, scsi, etc) and instead focuses on transporting the admin commands.
Quoted 6 lines
We also support more than one admin virtqueue, for QoS and scalability requirements.
Based-on-patch-by Max Gurtovoy <[email protected]>Signed-off-by Michael S. Tsirkin <[email protected]>Reviewed-by Stefan Hajnoczi <[email protected]>
This publicly archived list offers a means to provide input to the OASIS Virtual I/O Device (VIRTIO) TC.
In order to verify user consent to the Feedback License terms and to minimize spam in the list archive, subscription is required before posting.
Subscribe: [email protected]
Unsubscribe: [email protected]
List help: [email protected]
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/
[virtio-comment] Re: [PATCH v13 05/10] pci: add admin vq registers to virtio over pci
In reply to Michael S. TsirkinQuoted 9 lines · Michael S. Tsirkin
Add new registers to the PCI common configuration structure.
These registers will be used for querying the indices of the admin virtqueues of the owner device. To configure, reset or enable the admin virtqueues, the driver should follow existing queue configuration/setup sequence.
Based-on-patch-by Max Gurtovoy <[email protected]>Signed-off-by Michael S. Tsirkin <[email protected]>
Quoted 83 lines
---
since v11:
document that admin vqs are not counted with regular vqsReviewed-by Parav Pandit <[email protected]>Reviewed-by Zhu Lingshan <[email protected]>dropped Max's S.O.B
makequeue_numnot 0 based--- content.tex | 9 +++++++++ transport-pci.tex | 26 ++++++++++++++++++++++++++ 2 files changed, 35 insertions(+)content.tex +9
-0@@ -851,6 +851,15 @@ \chapter{Reserved Feature Bits}\label{sec:Reserved Feature Bits} \item[VIRTIO_F_ADMIN_VQ(41)] This feature indicates that the device exposes one or more administration virtqueues. + At the moment this feature is only supported for devices using + \ref{sec:Virtio Transport Options / Virtio Over PCI + Bus}~\nameref{sec:Virtio Transport Options / Virtio Over PCI Bus} + as the transport and is reserved for future use for + devices using other transports (see + \ref{drivernormative:Basic Facilities of a Virtio Device / Feature Bits} + and + \ref{devicenormative:Basic Facilities of a Virtio Device / Feature Bits} for + handling features reserved for future use. \end{description}transport-pci.tex +19
-0@@ -321,6 +321,10 @@ \subsubsection{Common configuration structure layout}\label{sec:Virtio Transport le64 queue_device; /* read-write */ le16 queue_notify_data; /* read-only for driver */ le16 queue_reset; /* read-write */ + + /* About the administration virtqueue. */ + le16 admin_queue_index; /* read-only for driver */ + le16 admin_queue_num; /* read-only for driver */ }; \end{lstlisting}@@ -347,6 +351,7 @@ \subsubsection{Common configuration structure layout}\label{sec:Virtio Transport \item[\field{num_queues}] The device specifies the maximum number of virtqueues supported here. + This excludes administration virtqueues if any are supported. \item[\field{device_status}] The driver writes the device status here (see \ref{sec:Basic Facilities of a Virtio Device / Device Status Field}). Writing 0 into this@@ -406,6 +411,19 @@ \subsubsection{Common configuration structure layout}\label{sec:Virtio Transport This field exists only if VIRTIO_F_RING_RESET has been negotiated. (see \ref{sec:Basic Facilities of a Virtio Device / Virtqueues / Virtqueue Reset}). +\item[\field{admin_queue_index}] + The device uses this to report the index of the first administration virtqueue. + This field is valid only if VIRTIO_F_ADMIN_VQ has been negotiated. +\item[\field{admin_queue_num}] + The device uses this to report the number of the + supported administration virtqueues. + Virtqueues with index + between \field{admin_queue_index} and (\field{admin_queue_index} + + \field{admin_queue_num} - 1) inclusive serve as administration + virtqueues. + The value 0 indicates no supported administration virtqueues. + This field is valid only if VIRTIO_F_ADMIN_VQ has been + negotiated. \end{description} \devicenormative{\paragraph}{Common configuration structure layout}{Virtio Transport Options / Virtio Over PCI Bus / PCI Device Layout / Common configuration structure layout}@@ -492,6 +510,14 @@ \subsubsection{Common configuration structure layout}\label{sec:Virtio Transport were used before the queue reset. (see \ref{sec:Basic Facilities of a Virtio Device / Virtqueues / Virtqueue Reset}). +If VIRTIO_F_ADMIN_VQ has been negotiated, and if the driver
transport-pci.tex \subsubsection{Common configuration structure layout}\label{sec:Virtio Transport · line 523
Quoted 10 lines
transport-pci.tex +7
-0+configures any administration virtqueues, the driver MUST +configure the administration virtqueues using the index +in the range \field{admin_queue_index} to +\field{admin_queue_index} + \field{admin_queue_num} - 1 inclusive. +The driver MAY configure less administration virtqueues than +supported by the device. + \subsubsection{Notification structure layout}\label{sec:Virtio Transport Options / Virtio Over PCI Bus / PCI Device Layout / Notification capability} The notification location is found using the VIRTIO_PCI_CAP_NOTIFY_CFG
This publicly archived list offers a means to provide input to the OASIS Virtual I/O Device (VIRTIO) TC.
In order to verify user consent to the Feedback License terms and to minimize spam in the list archive, subscription is required before posting.
Subscribe: [email protected]
Unsubscribe: [email protected]
List help: [email protected]
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/
[virtio-comment] Re: [PATCH v13 10/10] ccw: document more reserved features
In reply to Michael S. TsirkinQuoted 6 lines · Michael S. Tsirkin
vq reset and shared memory are unsupported, too.
Signed-off-by Michael S. Tsirkin <[email protected]>Reviewed-by Stefan Hajnoczi <[email protected]>Reviewed-by Zhu Lingshan <[email protected]>
This publicly archived list offers a means to provide input to the OASIS Virtual I/O Device (VIRTIO) TC.
In order to verify user consent to the Feedback License terms and to minimize spam in the list archive, subscription is required before posting.
Subscribe: [email protected]
Unsubscribe: [email protected]
List help: [email protected]
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/
[virtio-comment] Re: [PATCH v13 08/10] admin: command list discovery
In reply to Michael S. TsirkinQuoted 10 lines · Michael S. Tsirkin
Add commands to find out which commands does each group support, as well as enable their use by driver.
This will be especially useful once we have multiple group types.An alternative is per-type VQs. This is possible but will require more per-transport work. Discovery through the vq helps keep things contained.
e.g. lack of support for some command can switch to a legacy mode
Quoted 9 lines
note that commands are expected to be avolved by adding new fields to command specific data at the tail, so
we generally do not need feature bits for compatibility.Based-on-patch-by Max Gurtovoy <[email protected]>Signed-off-by Michael S. Tsirkin <[email protected]>Reviewed-by Stefan Hajnoczi <[email protected]>Reviewed-by Zhu Lingshan <[email protected]>
This publicly archived list offers a means to provide input to the OASIS Virtual I/O Device (VIRTIO) TC.
In order to verify user consent to the Feedback License terms and to minimize spam in the list archive, subscription is required before posting.
Subscribe: [email protected]
Unsubscribe: [email protected]
List help: [email protected]
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/
Quoted 109 lines · Fri, May 05, 2023 at 05:40:33PM CEST, [email protected] wrote:
Change log:
since 13:
command specific data is u8 again
exclude admin queues in blk'snum_queues
minor other tweakssince 11:
addressed lots of comments, all minor. consistency with outstandingnumber->indexandqueue->enqueuework i did not intentionally drop any reviewed-by tags as all changes are minor - if yours is missing it is because I forgot to record it, sorryone "breaking" change in response to stefan's comment:
in patch 5,num_queueshas been specified not to include admin
queues: just regular ones.since v10:
addressed lots of comments by Jiri, Stefan. Cornelia, Lngshan, Parav, Maxsince v9:
addressed comments by Parav, Max, Cornelia, David and Zhu Lingshan:
added link to errno header from Linux
rename_MEMto_MEMBER
admin vq num is zero based
clarify who sends commands where
minor english tweaks
clarify command length
specify interaction with sriov capability correct commit log - NumVFs can be 0i could not decide what should happen when VFs are disabled. for now did not specify.
since v8:
addressed comments by Cornelia - as we agreed on listsince v7:
make high level error codes match linux, with virtio specific codes
in a separate field
renamed_ACCEPTto_USEsince that's what it does clarified forward compatibility and non pci transports support multiple admin vqs
conformance statements
lots of changes all over the place to I changed author from Max to myself. Don't need to take credit but also don't want to blame Max for my mistakes.since v6:
- removed some extentions intended for future use. We'll do them when we get there.
- brought back command list query from v5 in a simplified form - it's here to address the case where a single parent can address multiple groups, such as PF addressing transport vq and sriov vfs.
- attempt to make terminology more formal. In particular a term for whoever controls the group. I am still going back
and forth between "parent" and "owner" - owner might be better after all since it will work if we ever have a self group. For now it's parent.TODO (maybe?) - probably ok to defer until this part is upstream:
Add "all members" member id.
Add commands for MSI, feature discovery.
Add commands for transport vq.
My intent is to try and support both SR-IOV and SIOV usecases with the same structure and maybe even the same VQ.
For example, it might make sense to split creating/destroying SIOV devices from the transport passing data from the guest - the driver would then not negotiate
VIRTIO_F_SR_IOV(which then means auto-provisioning).More ideas for use-cases:
virtio VF features query and configuration space provisioning virtio VF resource (queues, msix vectors count) provisioningFuture directions (shouldn't block this patch)
- aborting commands - left for later. or is vq reset enough?
- should we rename structures from admin to group admin?Michael S. Tsirkin (10):
virtio: document forward compatibility guarantees
admin: introduce device group and related concepts
admin: introduce group administration commands
admin: introduce virtio admin virtqueues
pci: add admin vq registers to virtio over pci
mmio: documentADMIN_VQas reserved
ccw: documentADMIN_VQas reserved
admin: command list discovery
admin: conformance clauses
ccw: document more reserved features
Quoted 7 lines
admin.tex | 584 +++++++++++++++++++++++++++++++ content.tex | 62 +++- device-types/blk/description.tex | 2 +- introduction.tex | 3 + transport-ccw.tex | 14 + transport-mmio.tex | 12 + transport-pci.tex | 33 ++
Michael, not sure if it is problem of this patchset or on my side but makepdf fails to build this. With master branch, works fine.
This publicly archived list offers a means to provide input to the OASIS Virtual I/O Device (VIRTIO) TC.
In order to verify user consent to the Feedback License terms and to minimize spam in the list archive, subscription is required before posting.
Subscribe: [email protected]
Unsubscribe: [email protected]
List help: [email protected]
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/
Quoted 7 lines
From: Jiri Pirko <[email protected]>
Sent: Wednesday, May 10, 2023 10:30 AMMichael, not sure if it is problem of this patchset or on my side but makepdf fails to build this. With master branch, works fine.
This patch fixes it.
https://lore.kernel.org/virtio-comment/[email protected]/
You can use b4 now to get this patch.
This publicly archived list offers a means to provide input to the OASIS Virtual I/O Device (VIRTIO) TC.
In order to verify user consent to the Feedback License terms and to minimize spam in the list archive, subscription is required before posting.
Subscribe: [email protected]
Unsubscribe: [email protected]
List help: [email protected]
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/
Quoted 5 lines · Wed, May 10, 2023 at 04:33:52PM CEST, [email protected] wrote:
Quoted 5 lines
From: Jiri Pirko <[email protected]>
Sent: Wednesday, May 10, 2023 10:30 AM
Quoted 7 lines
Quoted 2 lines
Michael, not sure if it is problem of this patchset or on my side but makepdf fails to build this. With master branch, works fine.
This patch fixes it.
https://lore.kernel.org/virtio-comment/[email protected]/You can use b4 now to get this patch.
Indeed, that helped. Thanks!
This publicly archived list offers a means to provide input to the OASIS Virtual I/O Device (VIRTIO) TC.
In order to verify user consent to the Feedback License terms and to minimize spam in the list archive, subscription is required before posting.
Subscribe: [email protected]
Unsubscribe: [email protected]
List help: [email protected]
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/
Quoted 5 lines · Parav Pandit
Quoted 5 lines
From: Jiri Pirko <[email protected]>
Sent: Wednesday, May 10, 2023 10:30 AM
Quoted 7 lines
Quoted 2 lines
Michael, not sure if it is problem of this patchset or on my side but makepdf fails to build this. With master branch, works fine.
This patch fixes it.
https://lore.kernel.org/virtio-comment/[email protected]/You can use b4 now to get this patch.
Thanks a lot, I will fold your patchset in when applying.
Signature · 18 lines
MST This publicly archived list offers a means to provide input to the OASIS Virtual I/O Device (VIRTIO) TC. In order to verify user consent to the Feedback License terms and to minimize spam in the list archive, subscription is required before posting. Subscribe: [email protected] Unsubscribe: [email protected] List help: [email protected] List archive: https://lists.oasis-open.org/archives/virtio-comment/ Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists Committee: https://www.oasis-open.org/committees/virtio/ Join OASIS: https://www.oasis-open.org/join/