Introduce virtio virtual device and transport vq
PATCH 1/1 Introduce virtio virtual device and transport vq
This patch introduces a new device type: virtio virtual device. An virtio virtual device example is virtio based Scalable I/O Virtualization devices.
This commit also introduces a new transport other than PCI and MMIO - the transport virtqueue.
This transport is useful for implementing virtual devices with a limited transport specific resources or presenting the virtual device in a transport independent way.
This means, all the basic device facilities are provided solely via the the transport virtqueue. Additionally, the transport virtqueue is also in charge of the creating and destroying of the virtual device.
With the help of the transport virtqueue, the presenting of the virtual device is done via the co-operation between the management device and its driver.
--- content.tex | 621 ++++++++++++++++++++++++++++++++++++++++++++++- introduction.tex | 3 + 2 files changed, 622 insertions(+), 2 deletions(-)
content.tex +619 -2
@@ -2784,6 +2784,619 @@ \subsubsection{Resetting Devices}\label{sec:Virtio Transport Options / Virtio ov In order to reset a device, a driver sends the CCW_CMD_VDEV_RESET command. +\section{Virtio Over Transport Virtqueue}\label{sec:Virtio Transport Options Virtio Over Transport Virtqueue} +Sometimes it's hard to implement the device in a transport specific method. +One example is that a physical device may try to present multiple virtual devices +with limited transport specific resources. +Another example is to implement virtual devices which are transport independent. +In those cases, the transport virtqueue provided by the management device could be used to replace +the transport specific method to implement the virtual device. +Then the presenting of the virtual device is done through the cooperation between +the transport virtqueue and the management device's driver. + +Feature bit VIRTIO_F_TRNSPT_VQ indicates that the device offers a transport virtqueue. + +\subsection{Basic Concepts}\label{sec:Virtio Transport Options / Virtio over Transport Virtqueue / Basic Concepts} + +All transport virtqueue commands are of the following form: + +\begin{lstlisting} +struct virtio_transportq_ctrl { + u64 device_id; + u16 class; + u16 command; + u32 length_out; + u8 command-out-data[]; + u32 length_in; + u32 ack; + u8 command-in-data[]; +}; + +/* ack values */ +#define VIRTIO_TRANSPTQ_OK 0 +#define VIRTIO_TRANSPTQ_EIO 1 +#define VIRTIO_TRANSPTQ_EAGAIN 11 +#define VIRTIO_TRANSPTQ_EBUSY 16 +#define VIRTIO_TRANSPTQ_EINVAL 22 +#define VIRTIO_TRANSPTQ_ERR 255 +\end{lstlisting} + +The \field{device_id}, \field{class}, \field{command}, \field{length_out} and +\field{command-out-data} are set by the management driver, +and the device sets the \field{ack}, \field{length_in} and \field{command-in-data}. + +\field{device_id} is an UUID to address a virtio virtual device, +The \field{device_id} value 0 is used for identify the management device itself + +\subsubsection{The managment devices}\label{sec:Virtio Transport Options / Virtio over Transport Virtqueue / Basic Concepts / The management devices} +A device that offers a transport virtqueue (via feature VIRTIO_F_TRNSPT_VQ) +is the management device of the virtual devices which are the managed devices. + +For example, a PCIe PF with a transport virtqueue is a management device. + +\subsubsection{The virtual devices}\label{sec:Virtio Transport Options / Virtio over Transport Virtqueue / Basic Concepts / The virtual devices} +A device that is created through a transport virtqueue which is offered by a management +device is a virtual device. Virtual devices are logical devices, managed by the management devices, +they usually don't have a dedicated physical transport. + +For example, a Scalable I/O Virtualization \hyperref[intro:SIOV]{[SIOV]} device which is created and managed through a transport virtqueue of +a management device is a virtual device. + +\devicenormative{\subsubsection}{Basic Concepts}{Virtio Transport Options / Virtio Over Transport Virtqueue / Basic Concepts} +The management device MUST offer feature bit VIRTIO_F_TRNSPT_VQ and a transport virtqueue as a device specific virtqueue.\\ +The virtual devices MUST not offer VIRTIO_F_TRNSPT_VQ feature.\\ +The management device MUST fail all commands against the transport virtqueue if VIRTIO_F_TRNSPT_VQ is not negotiated. + +\drivernormative{\subsubsection}{Basic Concepts}{Virtio Transport Options / Virtio Over Transport Virtqueue / Basic Concepts} +The management driver MUST not send any commands against the transport virtqueue if VIRTIO_F_TRNSPT_VQ is not negotiated. + +\subsection{Virtual Devices Discovery}\label{sec:Virtio Transport Options /Virtio Over Transport Virtqueue / Virtual Devices Discovery} + +The management device is discovered through its own transport and device specific method. +Virtual devices are created and discovered via the transport virtqueue. + +\subsection{Transport Virtqueue Features}\label{sec:Virtio Transport Options /Virtio Over Transport Virtqueue / Transport Virtqueue Features} + +Feature bit VIRTIO_F_TRNSPT_VQ_VDEV indicates that the virtual devices are created, configured +and destroyed through the transport virtqueue. +This means the transport virtqueue is the transport for the virtual devices. + +Feature bit VIRTIO_F_TRNSPT_VQ_VDEV depends on VIRTIO_F_TRNSPT_VQ + +\devicenormative{\subsubsection}{Transport Virtqueue Features}{Virtio Transport Options / Virtio Over Transport Virtqueue / Transport Virtqueue Features} +The management device MUST not accept VIRTIO_F_TRNSPT_VQ_VDEV if VIRTIO_F_TRNSPT_VQ is not negotiated. + +\subsection{Device Management}\label{sec:Virtio Transport Options / Virtio Over Transport Virtqueue / Device Management} + +When VIRTIO_F_TRNSPT_VQ_VDEV is negotiated, +virtual devices must be created and destroyed through the transport virtqueue. + +\begin{lstlisting} +struct virtio_transportq_ctrl_vdev_attribute { + u32 virtio_device_id; + u32 asid; + u8 config[]; +}; + +#define virtio_transportq_ctrl_vdev 1 (class) + #define virtio_transportq_ctrl_vdev_CREATE 0 (command) + #define virtio_transportq_ctrl_vdev_DESTROY 1 (command) +\end{lstlisting} + +The virtio_transportq_ctrl_vdev_CREATE command is used to create a virtual device. +The command-out-data for VIRTIO_TRANSPTQ_CTRL_CREATE is the virtio device id (\field{virtio_device_id}), +the address space id (\field{asid}) +and device specific configuration (\field{config}) for creating the virtual device. +When succeed, the device returns an u64 as a unique identifier of the created virtual device in command-in-data. + +The field \field{config} is device type specific. E.g., for virtio-net, it should follow the struct virtio_net_config +in section \ref{sec:Device Types / Network Device / Device configuration layout} + +The virtio_transportq_ctrl_vdev_DESTROY command is used to destroy a +virtual device which is identified by its 64bit identifier +\field{virtual_device_id}. There's no command-in-data for +VIRTIO_TRANSPTQ_CTRL_DESTROY command. + +\devicenormative{\subsubsection}{Device Management}{Virtio Transport Options / Virtio Over Transport Virtqueue / Device Management} + +The management device MUST fail the virtio_transportq_ctrl_vdev_CREATE if +\field{device_id} is not 0. + +The management device MUST fail the virtio_transportq_ctrl_vdev_DESTROY if +\field{device_id} is 0. + +All virtual devices MUST be created via the transport virtqueue if the management +device offers VIRTIO_F_TRNSPT_VQ_VDEV. + +\drivernormative{\subsubsection}{Driver Management}{Virtio Transport Options / Virtio Over Transport Virtqueue / Device Management} + +The management driver MUST use 0 as \field{device_id} for +virtio_transportq_ctrl_vdev_CREATE command. + +\subsection{Available resource of the management device}\label{sec:Virtio Transport Options / Virtio Over Transport Virtqueue / Available resource of the management devic} +When VIRTIO_F_TRNSPT_VQ_VDEV is negotiated, +statistical information of available resource of the management device could be fetched +by the following command: + +\begin{lstlisting} +#define VIRTIO_TRANSPTQ_CTRL_AVAIL_RES 2 + #define VIRTIO_TRANSPTQ_CTRL_AVAIL_RES_GET 0 +\end{lstlisting} + +VIRTIO_TRANSPTQ_CTRL_AVAIL_RES_GET command is used to get the statstical information of available +resource of the management device. There is no command-out-data for VIRTIO_TRANSPTQ_CTRL_AVAIL_RES_GET +command. The management device fills command-in-data in the following format: + +\begin{lstlisting} +struct mgmt_dev_avail_res{ + /* The number of total remaining virtqueus of the management device */ + u16 num_avail_vqs; + /* The minimal number of virtqueues of the virtual devices */ + u16 min_vdev_vqs; + /* The maximum number of virtqueues of the virtual devices */ + u16 max_vdev_vqs; + /* The number of virtual devices that can be created with min_vdev_vqs virtqueues. + * This number may not equal to num_avail_vqs divided by min_vdev_vqs due to + * the limitations of other resource + */ + u16 num_avail_vdev; +}; +\end{lstlisting} + +If \field{min_vdev_vqs} equals to \field{min_vdev_vqs}, that means the management device +can only create virtual devices with a fixed number of virtqueues + +\devicenormative{\subsubsection}{Available resource of the management device}{Virtio Transport Options / +Virtio Over Transport Virtqueue / Available resource of the management device} +The management device MUST fail the VIRTIO_TRANSPTQ_CTRL_AVAIL_RES_GET command if \field{device_id} is not 0 + +\drivernormative{\subsubsection}{Available resource of the management device}{Virtio Transport Options / +Virtio Over Transport Virtqueue / Available resource of the management device} +The management driver MUST use 0 as \field{device_id} for +VIRTIO_TRANSPTQ_CTRL_AVAIL_RES_GET command. + +\subsection{Features Negotiation}\label{sec:Virtio Transport Options / Virtio +Over Transport Virtqueue / Features Negotiation} + +When VIRTIO_F_TRNSPT_VQ_VDEV is negotiated, +the feature negotiation of virtual devices is done by the +following commands: + +\begin{lstlisting} +#define VIRTIO_TRANSPTQ_CTRL_FEAT 3 + #define VIRTIO_TRANSPTQ_CTRL_FEAT_DEVICE_GET 0 + #define VIRTIO_TRANSPTQ_CTRL_FEAT_DRIVER_SET 1 + #define VIRTIO_TRANSPTQ_CTRL_FEAT_DRIVER_GET 2 +\end{lstlisting} + +The VIRTIO_TRANSPTQ_CTRL_FEAT_DEVICE_GET is to get the features offered +by a virtual device. + +The VIRTIO_TRANSPTQ_CTRL_FEAT_DRIVER_SET is for driver to accept feature +bits offered by the virtual device. + +The VIRTIO_TRANSPTQ_CTRL_FEAT_DRIVER_GET is to get the features accepted +by both the virtual driver and the device. + +The features is 64 bits mask of the virtio features bit. For +VIRTIO_TRANSPTQ_CTRL_DRIVER_SET, the feature is passed to the device +through command-out-data. For VIRTIO_TRANSPTQ_CTRL_FEAT_DEVICE_GET and +VIRTIO_TRANSPTQ_CTRL_DRIVER_GET the feature is returned for the device +through command-in-data. + +\devicenormative{\subsubsection}{Features Negotiation}{Virtio Transport Options / +Virtio Over Transport Virtqueue / Features Negotiation} + +The management device MUST fail VIRTIO_TRANSPTQ_F_CTRL_FEAT class for the +command that use 0 as its \field{device_id}. + +\drivernormative{\subsubsection}{Features Negotiation}{Virtio Transport Options / +Virtio Over Transport Virtqueue / Features Negotiation} + +The management driver MAY mediate between the feature negotiation +request of the virtual devices and the transport virtqueue. E.g when +offering features to the virtual device, the management driver MAY +exclude some features in order to limit the behaviour of the virtual +device. + +\subsection{Device Status}\label{sec:Virtio Transport Options / Virtio Over +Transport Virtqueue / Device Status} + +When VIRTIO_F_TRNSPT_VQ_VDEV is negotiated, +the status of virtual device is accessed by the following +commands: + +\begin{lstlisting} +#define VIRTIO_TRANSPTQ_CTRL_STATUS 4 + #define VIRTIO_TRANSPTQ_CTRL_STATUS_SET 0 + #define VIRTIO_TRANSPTQ_CTRL_STATUS_GET 1 +\end{lstlisting} + +The VIRTIO_TRANSPTQ_CTRL_STATUS_SET is used to set the device status of +a virtual device here. The command-out-data is the one byte status +to set to the device. There's no command-in-data for this command. + +The VIRTIO_TRANSPTQ_CTRL_STATUS_GET is used to get the device status of +a virtual device. The command-in-data is the one byte status +returned from the device. There's no command-out-data for this +command. + +\devicenormative{\subsubsection}{Device Status}{Virtio Transport Options / Virtio +Over Transport Virtqueue / Device Status} + +The management device MUST reset the virtual device when 0 +is written via VIRTIO_TRANSPTQ_CTRL_STATUS_SET, the success of this +command demonstrate the success of the reset. + +The management device MUST present 0 through +VIRTIO_TRANSPTQ_CTRL_STATUS_GET once the reset is done. + +The management device MUST fail the device status access if +\field{device_id} is 0. + +\drivernormative{\subsubsection}{Device Status}{Virtio Transport Options / Virtio +Over Transport Virtqueue / Device Status} + +After writing 0 via VIRTIO_TRANSPTQ_CTRL_STATUS_SET, the driver MUST wait +for the success of the command before re-initializing the device. + +\subsection{Device Generation}\label{sec:Virtio Transport Options / Virtio Over Transport Virtqueue / Device Generation} + +When VIRTIO_F_TRNSPT_VQ_VDEV is negotiated, +the device generation count is read from the following commands: + +\begin{lstlisting} +#define VIRTIO_TRANSPTQ_CTRL_GENERATION 5 + #define VIRTIO_TRANSPTQ_CTRL_GENERATION_GET 0 +\end{lstlisting} + +The VIRTIO_TRANSPTQ_CTRL_GENERATION_GET is used to get the device configuration generation field +of the virtual device. The command-in-data is the u32 device +generation returned from the device. There's no command-out-data for +this command. + +\devicenormative{\subsubsection}{Device Generation}{Virtio Transport Options / Virtio Over Transport Virtqueue / Device Generation} + +The management device MUST present a changed generation count after the driver +has read a device-specific configuration value which has changed since +any part of the device-specific configuration was last read. + +The management device MUST fail the device generation access if \field{device_id} is 0. + +\subsection{Device Specific Configuration}\label{sec:Virtio Transport Options / +Virtio Over Transport Virtqueue / Device Specific Configuration} + +When VIRTIO_F_TRNSPT_VQ_VDEV is negotiated, +the device config space contents of a virtual device are accessed through +VIRTIO_TRANSPTQ_CTRL_CONFIG_GET and VIRTIO_TRANSPTQ_CTRL_CONFIG_SET. + +\begin{lstlisting} +#define VIRTIO_TRANSPTQ_CTRL_CONFIG 6 + #define VIRTIO_TRANSPTQ_CTRL_CONFIG_GET 0 + #define VIRTIO_TRANSPTQ_CTRL_CONFIG_SET 1 + +struct virtio_transportq_ctrl_vdev_config_get { + u32 offset; + u32 size; +}; + +struct virtio_transportq_ctrl_vdev_config_set { + u32 offset; + u32 size; + u8 data[]; +}; +\end{lstlisting} + +The VIRTIO_TRANSPTQ_CTRL_CONFIG_GET is used to read data from the +device configuration space. As described in struct +virtio_transportq_ctrl_vdev_config_get, The command-out-data is the offset +from the start of the config space and the size of the data. The +command-in-data is an array of u8 data that read from the config +space. + +The VIRTIO_TRANSPTQ_CTRL_CONFIG_SET is used to write data to the device +configuration space. As described in struct +virtio_transportq_ctrl_vdev_config_set, the command-out-data contains the +offset from the start of the config space, the size of the data and +the data that will be written. There's no command-in-data for this +command. + +\devicenormative{\subsubsection}{Device Specific Configuration}{Virtio Transport +Options / Virtio Over Transport Virtqueue / Device Specific Configuration} + +The management device MUST fail the device configuration space access +if the driver wants to access the range which is outside the config space. + +The management device MUST fail the device configuration space access +if \field{device_id} is 0. + +\subsection{MSI Configuration}\label{sec:Virtio Transport Options / Virtio Over +dmin Virtqueue / MSI Configuration} + +When VIRTIO_F_TRNSPT_VQ_VDEV is negotiated, +the MSI entry for a specific virtqueue is set through following command: + +\begin{lstlisting} +#define VIRTIO_TRANSPTQ_CTRL_MSI 7 + #define VIRTIO_TRANSPTQ_CTRL_MSI_VQ_SET 0 + #define VIRTIO_TRANSPTQ_CTRL_MSI_VQ_ENABLE 1 + #define VIRTIO_TRANSPTQ_CTRL_MSI_VQ_MASK 2 + #define VIRTIO_TRANSPTQ_CTRL_MSI_CONFIG_SET 3 + #define VIRTIO_TRANSPTQ_CTRL_MSI_CONFIG_ENABLE 4 + #define VIRTIO_TRANSPTQ_CTRL_MSI_CONFIG_MASK 5 + +struct virtio_transportq_ctrl_vdev_msi_vq_set { + u16 queue_index; + u64 addr; + u32 data; +}; + +struct virtio_transportq_ctrl_vdev_msi_vq_enable { + u16 queue_index; + u8 enable; +}; + +struct virtio_transportq_ctrl_vdev_msi_vq_mask { + u16 queue_index; + u8 mask; +}; + +struct virtio_transportq_ctrl_vdev_msi_config { + u64 addr; + u32 data; +}; +\end{lstlisting} + +The VIRTIO_TRANSPTQ_CTRL_MSI_VQ_SET command is used to set the MSI entry for a +specific virtqueue. The command-out-data is the virtqueue index and +the MSI address and data (as described in struct +virtio_transportq_ctrl_vdev_msix_vq_set). + +The VIRTIO_TRANSPTQ_CTRL_MSI_VQ_ENABLE command is used to enable or disable +the MSI interrupt for a specific virtqueue. The command-out-data is the +virtqueue index and whether to enable the MSI: 0 means to enable and 1 +means to disable (as described in struct +virtio_transportq_ctrl_vdev_msi_vq_enable). + +The VIRTIO_TRANSPTQ_CTRL_MSI_VQ_MASK command is used to mask or unmask the +MSI interrupt for a specific virtqueue. The command-out-data is the +virtqueue index and the mask status: 0 means unmak and 1 means mask +(as described in struct virtio_transportq_ctrl_vdev_msi_vq_mask). + +The VIRTIO_TRANSPTQ_CTRL_MSI_CONFIG_SET command is used to set the MSI entry +for the config interrupt. The command-out-data is the MSI address and +data (as described in struct virtio_transportq_ctrl_vdev_msi_config). + +The VIRTIO_TRANSPTQ_CTRL_MSI_CONFIG_ENABLE command is used to enable and disable +MSI for config space. The command-out-data is an u8: 0 means to +disable and 1 means to enable. + +The VIRTIO_TRANSPTQ_CTRL_MSI_CONFIG_MASK command is used to mask and unmask MSI +interrupt for config space. The command-out-data is an u8: 0 means to +mask and 1 means to unmask. + +There's no command-in-data for all the above MSI commands. + +\devicenormative{\subsubsection}{MSI Configuration}{Virtio Transport Options / Virtio +ver Transport Virtqueue / MSI Configuration} + +The virtual device MUST record the pending MSI interrupts and +re-generate the pending MSI interrupts when unmasking. + +The virtual device MUST disable the MSI for both virtqueue and config space +upon reset. + +\drivernormative{\subsubsection}{MSI Configuration}{Virtio Transport Options / Virtio +Over Transport Virtqueue / MSI Configuration} + +The driver MUST allocate transport or platform specific MSI entries +for both virtqueue and config space if it wants to use interrupt. + +The driver MAY choose disable the MSI if polling mode is used. + +\subsection{Virtqueue Address}\label{sec:Virtio Transport Options / Virtio Over +Transport Virtqueue / Virtqueue Address} + +When VIRTIO_F_TRNSPT_VQ_VDEV is negotiated, +the addresses of a specific virtqueue are accessed through the following command: + +\begin{lstlisting} +#define VIRTIO_TRANSPTQ_CTRL_VQ_ADDR 8 + #define VIRTIO_TRANSPTQ_CTRL_VQ_ADDR_SET 0 + +struct virtio_transportq_ctrl_vdev_vq_addr { + u16 queue_index; + u64 device_area; + u64 descriptor_area; + u64 driver_area; +}; +\end{lstlisting} + +The command-out-data is the queue index, the addresses of device area, +descriptor area and driver area (as described in struct +virtio_transportq_ctrl_vdev_vq_addr); There's no command-in-data. + +\devicenormative{\subsubsection}{Virtqueue Address}{Virtio Transport Options / Virtio +Over Transport Virtqueue / Virtqueeu Address} + +The management device MUST fail the commands of class +VIRTIO_TRANSPTQ_CTRL_VQ_ADDR if \field{device_id} is 0. + +\subsection{Virtqueue Status}\label{sec:Virtio Transport Options / Virtio Over +Transport Virtqueue / Virtqueue Status} + +When VIRTIO_F_TRNSPT_VQ_VDEV is negotiated, +virtqueue status is set and get through the following command: + +\begin{lstlisting} +#define VIRTIO_TRANSPTQ_CTRL_VQ_ENABLE 9 + #define VIRTIO_TRANSPTQ_CTRL_VQ_ENABLE_SET 0 + #define VIRTIO_TRANSPTQ_CTRL_VQ_ENABLE_GET 1 + +struct virtio_transportq_ctrl_vq_status_set { + u16 queue_index; + u8 status; +}; + +\end{lstlisting} + +The VIRTIO_TRANSPTQ_CTRL_VQ_ENABLE_SET command is used to set the status of a +specific virtqueue. The command-out-data is the queue index and the +status that is set to the virtqueue (0 disabled, 1 enabled), +as described in struct virtio_transportq_ctrl_vq_status_set. +There's no command-in-data. + +The VIRTIO_TRANSPTQ_CTRL_VQ_ENABLE_GET is used to get the status of a +specific virtqueue. The command-out-data is an u16 of the queue +index. The command-in-data is the virtqueue status (0 disabled, 1 +enabled). + +\devicenormative{\subsubsection}{Virtqueue Status}{Virtio Transport Options / Virtio +Over Transport Virtqueue / Virtqueue Status} + +When disabled, the virtual device MUST stop processing requests from +this virtqueue. + +The management device MUST present a 0 via +VIRTIO_TRANSPTQ_CTRL_VQ_ENABLE_GET upon a reset of the virtual device. + +The management device MUST fail the virtqueue status access if +\field{device_id} is 0. + +\drivernormative{\subsubsection}{Virtqueue Status}{Virtio Transport Options / Virtio +Over Transport Virtqueue / Virtqueue Status} + +The driver MUST configure other virtqueue fields before enabling +the virtqueue with VIRTIO_TRANSPTQ_CTRL_VQ_ENABLE_SET. + +\subsection{Virtqueue Size}\label{sec:Virtio Transport Options / Virtio Over +Transport Virtqueue / Virtqueue Size} + +When VIRTIO_F_TRNSPT_VQ_VDEV is negotiated, +virtqueue size is accessed through the following command: + +\begin{lstlisting} +#define VIRTIO_TRANSPTQ_CTRL_VQ_SIZE 10 + #define VIRTIO_TRANSPTQ_CTRL_VQ_SIZE_SET 0 + #define VIRTIO_TRANSPTQ_CTRL_VQ_SIZE_GET 1 + +struct virtio_transportq_ctrl_vdev_vq_size_set { + u16 queue_index; + u16 size; +}; +\end{lstlisting} + +The VIRTIO_TRANSPTQ_CTRL_VQ_SIZE_SET command is used to set the virtqueue +size. The command-out-data is the queue index and the size of the +virtqueue (as described in struct +virtio_transportq_ctrl_vdev_vq_size_set). There's no command-in-data. + +The VIRTIO_TRANSPTQ_CTRL_VQ_SIZE_GET command is used to get the virtqueue +size. On reset, the maximum queue size supported by the device is +returned. The command-out-data is an u16 of the virtqueue index. The +command-in-data is an u16 of queue size for the virtqueue. + +\devicenormative{\subsubsection}{Virtqueue Status}{Virtio Transport Options / Virtio +Over Transport Virtqueue / Virtqueue Size} + +The management device MUST fail the virtqueue size access if +\field{device_id} is 0. + +\subsection{Virtqueue Notification}\label{sec:Virtio Transport Options / Virtio +Over Transport Virtqueue / Virtqueue Notification} + +When VIRTIO_F_TRNSPT_VQ_VDEV is negotiated, +the virtqueue notification is done through the following commands: + +\begin{lstlisting} +#define VIRTIO_TRANSPTQ_CTRL_VQ_NOTIFY 11 + #define VIRTIO_TRANSPTQ_CTRL_VQ_NOTIFY_GET 1 + +struct virtio_transportq_ctrl_vdev_vq_notification_area { + le64 addr + le64 size; +}; +\end{lstlisting} + +The VIRTIO_TRANSPTQ_CTRL_VQ_NOTIFY_GET is used to get the transport +specific address area that can be used to notify a virtqueue. The +command-out-data is an u16 of the virtqueue index. The command-in-data +contains the address and the size of the notification area (as +described in struct virtio_transportq_ctrl_vdev_vq_notification_area). + +\devicenormative{\subsubsection}{Virtqueue Notification}{Virtio Transport Options / +Virtio Over Transport Virtqueue / Virtqueue Notification} + +The management device MUST fail the VIRTIO_TRANSPTQ_CTRL_VQ_NOTIFY_GET command +if there's no transport specific notification address for a virtqueue of +its virtual device. + +The management device MUST fail the virtqueue notification access if +\field{device_id} is 0. + +The management device MUST forbid the notification area of a specific +virtual device to be accessed from another virtual device. + +\drivernormative{\subsubsection}{Virtqueue Notification}{Virtio Transport Options / +Virtio Over Transport Virtqueue / Virtqueue Notification} + +The driver MAY choose to notify the virtqueue by writing the queue +index at address \field{addr} which is fetched from the +VIRTIO_TRANSPTQ_CTRL_VQ_NOTIFY_GET command. + +\subsection{Virtqueue Index}\label{sec:Virtio Transport Options / Virtio +Over Transport Virtqueue / Virtqueue Index} + +When VIRTIO_F_TRNSPT_VQ_VDEV is negotiated, +the virtqueue available index is accessed through the following commands: + +\begin{lstlisting} +#define VIRTIO_TRANSPTQ_CTRL_VQ_AVAIL_INDEX 12 + #define VIRTIO_TRANSPTQ_CTRL_VQ_AVAIL_INDEX_SET 0 + #define VIRTIO_TRANSPTQ_CTRL_VQ_ AVAIL_INDEX_GET 1 + +struct virtio_transportq_ctrl_vdev_vq_avail_idx_set { + u16 queue_index; + u16 avail_idx; +}; +\end{lstlisting} + +The VIRTIO_TRANSPTQ_CTRL_VQ_AVAIL_INDEX_SET commmand is used to set the available index of a specific virtqueue. +The command-out-data is the virtqueue index and the available index (as described in struct virtio_transportq_ctrl_vdev_vq_avail_idx). +There is no command-in-data. + +The VIRTIO_TRANSPTQ_CTRL_VQ_ AVAIL_INDEX_GET command is used to get the available index of a specific virtqueue. +The command-out-data is the queue index, the command-in-data is the available index of the queue. + +\devicenormative{\subsubsection}{Virtqueue Index}{Virtio Transport Options / +Virtio Over Transport Virtqueue / Virtqueue Index} + +The management device MUST fail the virtqueue index access if \field{device_id} is 0 +The management device MUST fail the virtqueue index access if \field{queue_id} is invalid. + +\subsection{Presenting Virtual Devices}\label{sec:Virtio Transport Options / +Virtio Over Transport Virtqueue / Presenting Virtual Device} + +If VIRTIO_TRANSPTQ_F_VDEV is offered by a management device. The presenting of +the virtual devices requires co-operation between the management +driver and the transport virtqueue. This means, from the view of the +virtual device driver, the transport is done via the communication +with the management device driver. It's up to the software to decide +what kind of method that is needed be used for those communications. + +The management driver typically takes the following steps for creating a +virtual device: + +\begin{enumerate} +\item Determine the virtio id and device specific configuration. +\item Create the virtual devices using the virtio_transportq_ctrl_vdev_CREATE command. +\item Optionally, configure the MSI. +\item Perform device specific setup. +\item Let the virtual device to be probed by the virtual device +driver. The management driver will then use the transport virtqueue to +implement the requests of basic facility from the virtual device +driver. +\end{enumerate} \chapter{Device Types}\label{sec:Device Types}
@@ -2911,12 +3524,14 @@ \subsection{Virtqueues}\label{sec:Device Types / Network Device / Virtqueues} \item[2(N-1)] receiveqN \item[2(N-1)+1] transmitqN \item[2N] controlq +\item[2N+1] transportq \end{description} N=1 if neither VIRTIO_NET_F_MQ nor VIRTIO_NET_F_RSS are negotiated, otherwise N is set by \field{max_virtqueue_pairs}. - controlq only exists if VIRTIO_NET_F_CTRL_VQ set. + controlq only exists if VIRTIO_NET_F_CTRL_VQ set.\\ + transportq only exists if VIRTIO_F_TRNSPT_VQ is offered. \subsection{Feature bits}\label{sec:Device Types / Network Device / Feature bits}
@@ -4399,10 +5014,12 @@ \subsection{Virtqueues}\label{sec:Device Types / Block Device / Virtqueues} \item[0] requestq1 \item[\ldots] \item[N-1] requestqN +\item[N] transportq \end{description} N=1 if VIRTIO_BLK_F_MQ is not negotiated, otherwise N is set by - \field{num_queues}. + \field{num_queues}.\\ + transportq only exists if VIRTIO_F_TRNSPT_VQ is offered. \subsection{Feature bits}\label{sec:Device Types / Block Device / Feature bits}
introduction.tex +3 -0
@@ -79,6 +79,9 @@ \section{Normative References}\label{sec:Normative References} \phantomsection\label{intro:SCMI}\textbf{[SCMI]} & Arm System Control and Management Interface, DEN0056, \newline\url{https://developer.arm.com/docs/den0056/c}, version C and any future revisions\\ + \phantomsection\label{intro:SIOV}\textbf{[SIOV]} & + Scalable I/O Virtualization, + \newline\url{https://www.opencompute.org/documents/ocp-scalable-io-virtualization-technical-specification-revision-1-v1-2-pdf}\\ \end{longtable}
Signature · 18 lines
2.35.3 This publicly archived list offers a means to provide input to the OASIS Virtual I/O Device (VIRTIO) TC. In order to verify user consent to the Feedback License terms and to minimize spam in the list archive, subscription is required before posting. Subscribe: [email protected] Unsubscribe: [email protected] List help: [email protected] List archive: https://lists.oasis-open.org/archives/virtio-comment/ Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists Committee: https://www.oasis-open.org/committees/virtio/ Join OASIS: https://www.oasis-open.org/join/
Quoted 8 lines · Zhu Lingshan
This patch introduces a new device type: virtio virtual device. An virtio virtual device example is virtio based Scalable I/O Virtualization devices.
This commit also introduces a new transport other than PCI and MMIO - the transport virtqueue.
Quoted 6 lines
This transport is useful for implementing virtual devices with a limited transport specific resources or presenting the virtual device in a transport independent way.
This means, all the basic device facilities are provided solely via the the transport virtqueue. Additionally, the transport virtqueue is also in
Two "the".
Quoted 5 lines
charge of the creating and destroying of the virtual device.
With the help of the transport virtqueue, the presenting of the virtual device is done via the co-operation between the management device and its driver.
Having a dedicated virtqueue is probably fine but there is some overlap at the description of management/managed device with the admin virtqueue proposal from Max. Need to consider a way to unify the definition at least.
Quoted 32 lines
Signed-off-by Jason Wang <[email protected]>Signed-off-by Zhu Lingshan <[email protected]>--- content.tex | 621 ++++++++++++++++++++++++++++++++++++++++++++++- introduction.tex | 3 + 2 files changed, 622 insertions(+), 2 deletions(-)content.tex +16
-0@@ -2784,6 +2784,619 @@ \subsubsection{Resetting Devices}\label{sec:Virtio Transport Options / Virtio ov In order to reset a device, a driver sends the CCW_CMD_VDEV_RESET command. +\section{Virtio Over Transport Virtqueue}\label{sec:Virtio Transport Options Virtio Over Transport Virtqueue} +Sometimes it's hard to implement the device in a transport specific method. +One example is that a physical device may try to present multiple virtual devices +with limited transport specific resources. +Another example is to implement virtual devices which are transport independent. +In those cases, the transport virtqueue provided by the management device could be used to replace +the transport specific method to implement the virtual device. +Then the presenting of the virtual device is done through the cooperation between +the transport virtqueue and the management device's driver. + +Feature bit VIRTIO_F_TRNSPT_VQ indicates that the device offers a transport virtqueue. + +\subsection{Basic Concepts}\label{sec:Virtio Transport Options / Virtio over Transport Virtqueue / Basic Concepts} + +All transport virtqueue commands are of the following form: +
content.tex \subsubsection{Resetting Devices}\label{sec:Virtio Transport Options / Virtio ov · line 2812
Quoted 10 lines
content.tex +10
-0+\begin{lstlisting} +struct virtio_transportq_ctrl { + u64 device_id; + u16 class; + u16 command; + u32 length_out; + u8 command-out-data[]; + u32 length_in; + u32 ack; + u8 command-in-data[];
Any reason we need length_in/length_out (we have length in descriptors)? FYI, we don't have them for ctrl virtqueue of networking devices.
Quoted 7 lines
content.tex +7
-0+}; + +/* ack values */ +#define VIRTIO_TRANSPTQ_OK 0 +#define VIRTIO_TRANSPTQ_EIO 1 +#define VIRTIO_TRANSPTQ_EAGAIN 11 +#define VIRTIO_TRANSPTQ_EBUSY 16
content.tex \subsubsection{Resetting Devices}\label{sec:Virtio Transport Options / Virtio ov · line 2829
Quoted 10 lines
content.tex +10
-0+#define VIRTIO_TRANSPTQ_EINVAL 22 +#define VIRTIO_TRANSPTQ_ERR 255 +\end{lstlisting} + +The \field{device_id}, \field{class}, \field{command}, \field{length_out} and +\field{command-out-data} are set by the management driver, +and the device sets the \field{ack}, \field{length_in} and \field{command-in-data}. + +\field{device_id} is an UUID to address a virtio virtual device, +The \field{device_id} value 0 is used for identify the management device itself
identifying?
content.tex \subsubsection{Resetting Devices}\label{sec:Virtio Transport Options / Virtio ov · line 2831
Quoted 2 lines
content.tex +2
-0+ +\subsubsection{The managment devices}\label{sec:Virtio Transport Options / Virtio over Transport Virtqueue / Basic Concepts / The management devices}
s/managment/management/
content.tex \subsubsection{Resetting Devices}\label{sec:Virtio Transport Options / Virtio ov · line 2833
Quoted 2 lines
content.tex +2
-0+A device that offers a transport virtqueue (via feature VIRTIO_F_TRNSPT_VQ) +is the management device of the virtual devices which are the managed devices.
Not a native speaker but I couldn't parse this sentence.
content.tex \subsubsection{Resetting Devices}\label{sec:Virtio Transport Options / Virtio ov · line 2837
Quoted 4 lines
content.tex +4
-0+ +For example, a PCIe PF with a transport virtqueue is a management device. + +\subsubsection{The virtual devices}\label{sec:Virtio Transport Options / Virtio over Transport Virtqueue / Basic Concepts / The virtual devices}
I think to be consistent, per Michael's suggestion, we should use "managed" here which is better than "virtual".
content.tex \subsubsection{Resetting Devices}\label{sec:Virtio Transport Options / Virtio ov · line 2843
Quoted 6 lines
content.tex +6
-0+A device that is created through a transport virtqueue which is offered by a management +device is a virtual device. Virtual devices are logical devices, managed by the management devices, +they usually don't have a dedicated physical transport. + +For example, a Scalable I/O Virtualization \hyperref[intro:SIOV]{[SIOV]} device which is created and managed through a transport virtqueue of +a management device is a virtual device.
Maybe we can remove the "is a virtual device" here, since we use "For example" in the subsection of "The virtual device"
content.tex \subsubsection{Resetting Devices}\label{sec:Virtio Transport Options / Virtio ov · line 2847
Quoted 4 lines
content.tex +4
-0+ +\devicenormative{\subsubsection}{Basic Concepts}{Virtio Transport Options / Virtio Over Transport Virtqueue / Basic Concepts} +The management device MUST offer feature bit VIRTIO_F_TRNSPT_VQ and a transport virtqueue as a device specific virtqueue.\\ +The virtual devices MUST not offer VIRTIO_F_TRNSPT_VQ feature.\\
Is this a must? What's the value of prohibiting the nesting of the management device?
Quoted 31 lines
content.tex +31
-0+The management device MUST fail all commands against the transport virtqueue if VIRTIO_F_TRNSPT_VQ is not negotiated. + +\drivernormative{\subsubsection}{Basic Concepts}{Virtio Transport Options / Virtio Over Transport Virtqueue / Basic Concepts} +The management driver MUST not send any commands against the transport virtqueue if VIRTIO_F_TRNSPT_VQ is not negotiated. + +\subsection{Virtual Devices Discovery}\label{sec:Virtio Transport Options /Virtio Over Transport Virtqueue / Virtual Devices Discovery} + +The management device is discovered through its own transport and device specific method. +Virtual devices are created and discovered via the transport virtqueue. + +\subsection{Transport Virtqueue Features}\label{sec:Virtio Transport Options /Virtio Over Transport Virtqueue / Transport Virtqueue Features} + +Feature bit VIRTIO_F_TRNSPT_VQ_VDEV indicates that the virtual devices are created, configured +and destroyed through the transport virtqueue. +This means the transport virtqueue is the transport for the virtual devices. + +Feature bit VIRTIO_F_TRNSPT_VQ_VDEV depends on VIRTIO_F_TRNSPT_VQ + +\devicenormative{\subsubsection}{Transport Virtqueue Features}{Virtio Transport Options / Virtio Over Transport Virtqueue / Transport Virtqueue Features} +The management device MUST not accept VIRTIO_F_TRNSPT_VQ_VDEV if VIRTIO_F_TRNSPT_VQ is not negotiated. + +\subsection{Device Management}\label{sec:Virtio Transport Options / Virtio Over Transport Virtqueue / Device Management} + +When VIRTIO_F_TRNSPT_VQ_VDEV is negotiated, +virtual devices must be created and destroyed through the transport virtqueue. + +\begin{lstlisting} +struct virtio_transportq_ctrl_vdev_attribute { + u32 virtio_device_id; + u32 asid; + u8 config[];
content.tex \subsubsection{Resetting Devices}\label{sec:Virtio Transport Options / Virtio ov · line 2888
Quoted 10 lines
content.tex +10
-0+}; + +#define virtio_transportq_ctrl_vdev 1 (class) + #define virtio_transportq_ctrl_vdev_CREATE 0 (command) + #define virtio_transportq_ctrl_vdev_DESTROY 1 (command) +\end{lstlisting} + +The virtio_transportq_ctrl_vdev_CREATE command is used to create a virtual device. +The command-out-data for VIRTIO_TRANSPTQ_CTRL_CREATE is the virtio device id (\field{virtio_device_id}), +the address space id (\field{asid})
We need to be verbose here. Is this a transport specific identifier or a device specific identifier?
I also wonder if this is the best place for this. To me, it looks more suitable to have a dedicated command to configure it before DRIVER_OK like virtqueue address. Then we can configure it per virtqueue per device with more flexibility.
content.tex \subsubsection{Resetting Devices}\label{sec:Virtio Transport Options / Virtio ov · line 2892
Quoted 4 lines
content.tex +4
-0+and device specific configuration (\field{config}) for creating the virtual device. +When succeed, the device returns an u64 as a unique identifier of the created virtual device in command-in-data. + +The field \field{config} is device type specific. E.g., for virtio-net, it should follow the struct virtio_net_config
I think we need a dedicated section to describe the transport virtqueue support for virtio-net.
content.tex \subsubsection{Resetting Devices}\label{sec:Virtio Transport Options / Virtio ov · line 2903
Quoted 11 lines
content.tex +11
-0+in section \ref{sec:Device Types / Network Device / Device configuration layout} + +The virtio_transportq_ctrl_vdev_DESTROY command is used to destroy a +virtual device which is identified by its 64bit identifier +\field{virtual_device_id}. There's no command-in-data for +VIRTIO_TRANSPTQ_CTRL_DESTROY command. + +\devicenormative{\subsubsection}{Device Management}{Virtio Transport Options / Virtio Over Transport Virtqueue / Device Management} + +The management device MUST fail the virtio_transportq_ctrl_vdev_CREATE if +\field{device_id} is not 0.
Let's use VIRTIO_TRANSPTQ_OK instead of the magic number here.
content.tex \subsubsection{Resetting Devices}\label{sec:Virtio Transport Options / Virtio ov · line 2906
Quoted 3 lines
content.tex +3
-0+ +The management device MUST fail the virtio_transportq_ctrl_vdev_DESTROY if +\field{device_id} is 0.
Same here.
content.tex \subsubsection{Resetting Devices}\label{sec:Virtio Transport Options / Virtio ov · line 2909
Quoted 3 lines
content.tex +3
-0+ +All virtual devices MUST be created via the transport virtqueue if the management +device offers VIRTIO_F_TRNSPT_VQ_VDEV.
Does this imply no pre-created virtual devices? If this is true, we don't need a dedicated feature bit (VIRTIO_F_TRNSPT_VQ_VDEV).
The bit only makes sense for the case when some managed(virtual) devices are pre-created like initial_vfs in SR-IOV. If we allow pre-creation, we need commands to discover them.
Quoted 4 lines
content.tex +4
-0+ +\drivernormative{\subsubsection}{Driver Management}{Virtio Transport Options / Virtio Over Transport Virtqueue / Device Management} + +The management driver MUST use 0 as \field{device_id} for
content.tex \subsubsection{Resetting Devices}\label{sec:Virtio Transport Options / Virtio ov · line 2923
Quoted 10 lines
content.tex +10
-0+virtio_transportq_ctrl_vdev_CREATE command. + +\subsection{Available resource of the management device}\label{sec:Virtio Transport Options / Virtio Over Transport Virtqueue / Available resource of the management devic} +When VIRTIO_F_TRNSPT_VQ_VDEV is negotiated, +statistical information of available resource of the management device could be fetched +by the following command: + +\begin{lstlisting} +#define VIRTIO_TRANSPTQ_CTRL_AVAIL_RES 2 + #define VIRTIO_TRANSPTQ_CTRL_AVAIL_RES_GET 0
Nit, I think this command should come first, mgmt need to query this before creating devices.
content.tex \subsubsection{Resetting Devices}\label{sec:Virtio Transport Options / Virtio ov · line 2926
Quoted 3 lines
content.tex +3
-0+\end{lstlisting} + +VIRTIO_TRANSPTQ_CTRL_AVAIL_RES_GET command is used to get the statstical information of available
s/statstical/statistical/
content.tex \subsubsection{Resetting Devices}\label{sec:Virtio Transport Options / Virtio ov · line 2932
Quoted 6 lines
content.tex +6
-0+resource of the management device. There is no command-out-data for VIRTIO_TRANSPTQ_CTRL_AVAIL_RES_GET +command. The management device fills command-in-data in the following format: + +\begin{lstlisting} +struct mgmt_dev_avail_res{ + /* The number of total remaining virtqueus of the management device */
Should be virtqueues.
Actually the description here is confusing, something like this might be better
"The number of total remaining virtqueues for the managed(virtual) devices */
content.tex \subsubsection{Resetting Devices}\label{sec:Virtio Transport Options / Virtio ov · line 2934
Quoted 2 lines
content.tex +2
-0+ u16 num_avail_vqs; + /* The minimal number of virtqueues of the virtual devices */
"of a virtual device"?
Quoted 44 lines
content.tex +44
-0+ u16 min_vdev_vqs; + /* The maximum number of virtqueues of the virtual devices */ + u16 max_vdev_vqs; + /* The number of virtual devices that can be created with min_vdev_vqs virtqueues. + * This number may not equal to num_avail_vqs divided by min_vdev_vqs due to + * the limitations of other resource + */ + u16 num_avail_vdev; +}; +\end{lstlisting} + +If \field{min_vdev_vqs} equals to \field{min_vdev_vqs}, that means the management device +can only create virtual devices with a fixed number of virtqueues + +\devicenormative{\subsubsection}{Available resource of the management device}{Virtio Transport Options / +Virtio Over Transport Virtqueue / Available resource of the management device} +The management device MUST fail the VIRTIO_TRANSPTQ_CTRL_AVAIL_RES_GET command if \field{device_id} is not 0 + +\drivernormative{\subsubsection}{Available resource of the management device}{Virtio Transport Options / +Virtio Over Transport Virtqueue / Available resource of the management device} +The management driver MUST use 0 as \field{device_id} for +VIRTIO_TRANSPTQ_CTRL_AVAIL_RES_GET command. + +\subsection{Features Negotiation}\label{sec:Virtio Transport Options / Virtio +Over Transport Virtqueue / Features Negotiation} + +When VIRTIO_F_TRNSPT_VQ_VDEV is negotiated, +the feature negotiation of virtual devices is done by the +following commands: + +\begin{lstlisting} +#define VIRTIO_TRANSPTQ_CTRL_FEAT 3 + #define VIRTIO_TRANSPTQ_CTRL_FEAT_DEVICE_GET 0 + #define VIRTIO_TRANSPTQ_CTRL_FEAT_DRIVER_SET 1 + #define VIRTIO_TRANSPTQ_CTRL_FEAT_DRIVER_GET 2 +\end{lstlisting} + +The VIRTIO_TRANSPTQ_CTRL_FEAT_DEVICE_GET is to get the features offered +by a virtual device. + +The VIRTIO_TRANSPTQ_CTRL_FEAT_DRIVER_SET is for driver to accept feature +bits offered by the virtual device. + +The VIRTIO_TRANSPTQ_CTRL_FEAT_DRIVER_GET is to get the features accepted
content.tex \subsubsection{Resetting Devices}\label{sec:Virtio Transport Options / Virtio ov · line 2988
Quoted 10 lines
content.tex +10
-0+by both the virtual driver and the device. + +The features is 64 bits mask of the virtio features bit. For +VIRTIO_TRANSPTQ_CTRL_DRIVER_SET, the feature is passed to the device +through command-out-data. For VIRTIO_TRANSPTQ_CTRL_FEAT_DEVICE_GET and +VIRTIO_TRANSPTQ_CTRL_DRIVER_GET the feature is returned for the device +through command-in-data. + +\devicenormative{\subsubsection}{Features Negotiation}{Virtio Transport Options / +Virtio Over Transport Virtqueue / Features Negotiation}
Can one of the above three commands fail? If yes, we need to document the behaviour.
Quoted 43 lines
content.tex +43
-0+ +The management device MUST fail VIRTIO_TRANSPTQ_F_CTRL_FEAT class for the +command that use 0 as its \field{device_id}. + +\drivernormative{\subsubsection}{Features Negotiation}{Virtio Transport Options / +Virtio Over Transport Virtqueue / Features Negotiation} + +The management driver MAY mediate between the feature negotiation +request of the virtual devices and the transport virtqueue. E.g when +offering features to the virtual device, the management driver MAY +exclude some features in order to limit the behaviour of the virtual +device. + +\subsection{Device Status}\label{sec:Virtio Transport Options / Virtio Over +Transport Virtqueue / Device Status} + +When VIRTIO_F_TRNSPT_VQ_VDEV is negotiated, +the status of virtual device is accessed by the following +commands: + +\begin{lstlisting} +#define VIRTIO_TRANSPTQ_CTRL_STATUS 4 + #define VIRTIO_TRANSPTQ_CTRL_STATUS_SET 0 + #define VIRTIO_TRANSPTQ_CTRL_STATUS_GET 1 +\end{lstlisting} + +The VIRTIO_TRANSPTQ_CTRL_STATUS_SET is used to set the device status of +a virtual device here. The command-out-data is the one byte status +to set to the device. There's no command-in-data for this command. + +The VIRTIO_TRANSPTQ_CTRL_STATUS_GET is used to get the device status of +a virtual device. The command-in-data is the one byte status +returned from the device. There's no command-out-data for this +command. + +\devicenormative{\subsubsection}{Device Status}{Virtio Transport Options / Virtio +Over Transport Virtqueue / Device Status} + +The management device MUST reset the virtual device when 0 +is written via VIRTIO_TRANSPTQ_CTRL_STATUS_SET, the success of this +command demonstrate the success of the reset. + +The management device MUST present 0 through
content.tex \subsubsection{Resetting Devices}\label{sec:Virtio Transport Options / Virtio ov · line 3041
Quoted 10 lines
content.tex +10
-0+VIRTIO_TRANSPTQ_CTRL_STATUS_GET once the reset is done. + +The management device MUST fail the device status access if +\field{device_id} is 0. + +\drivernormative{\subsubsection}{Device Status}{Virtio Transport Options / Virtio +Over Transport Virtqueue / Device Status} + +After writing 0 via VIRTIO_TRANSPTQ_CTRL_STATUS_SET, the driver MUST wait +for the success of the command before re-initializing the device.
So we need to clarify which is the condition for the succeed of a reset:
1) the succeed of STATUS_SET (this is how ccw did)
2) 0 is read from STATUS_GET (this is how pci did)
Quoted 77 lines
content.tex +77
-0+ +\subsection{Device Generation}\label{sec:Virtio Transport Options / Virtio Over Transport Virtqueue / Device Generation} + +When VIRTIO_F_TRNSPT_VQ_VDEV is negotiated, +the device generation count is read from the following commands: + +\begin{lstlisting} +#define VIRTIO_TRANSPTQ_CTRL_GENERATION 5 + #define VIRTIO_TRANSPTQ_CTRL_GENERATION_GET 0 +\end{lstlisting} + +The VIRTIO_TRANSPTQ_CTRL_GENERATION_GET is used to get the device configuration generation field +of the virtual device. The command-in-data is the u32 device +generation returned from the device. There's no command-out-data for +this command. + +\devicenormative{\subsubsection}{Device Generation}{Virtio Transport Options / Virtio Over Transport Virtqueue / Device Generation} + +The management device MUST present a changed generation count after the driver +has read a device-specific configuration value which has changed since +any part of the device-specific configuration was last read. + +The management device MUST fail the device generation access if \field{device_id} is 0. + +\subsection{Device Specific Configuration}\label{sec:Virtio Transport Options / +Virtio Over Transport Virtqueue / Device Specific Configuration} + +When VIRTIO_F_TRNSPT_VQ_VDEV is negotiated, +the device config space contents of a virtual device are accessed through +VIRTIO_TRANSPTQ_CTRL_CONFIG_GET and VIRTIO_TRANSPTQ_CTRL_CONFIG_SET. + +\begin{lstlisting} +#define VIRTIO_TRANSPTQ_CTRL_CONFIG 6 + #define VIRTIO_TRANSPTQ_CTRL_CONFIG_GET 0 + #define VIRTIO_TRANSPTQ_CTRL_CONFIG_SET 1 + +struct virtio_transportq_ctrl_vdev_config_get { + u32 offset; + u32 size; +}; + +struct virtio_transportq_ctrl_vdev_config_set { + u32 offset; + u32 size; + u8 data[]; +}; +\end{lstlisting} + +The VIRTIO_TRANSPTQ_CTRL_CONFIG_GET is used to read data from the +device configuration space. As described in struct +virtio_transportq_ctrl_vdev_config_get, The command-out-data is the offset +from the start of the config space and the size of the data. The +command-in-data is an array of u8 data that read from the config +space. + +The VIRTIO_TRANSPTQ_CTRL_CONFIG_SET is used to write data to the device +configuration space. As described in struct +virtio_transportq_ctrl_vdev_config_set, the command-out-data contains the +offset from the start of the config space, the size of the data and +the data that will be written. There's no command-in-data for this +command. + +\devicenormative{\subsubsection}{Device Specific Configuration}{Virtio Transport +Options / Virtio Over Transport Virtqueue / Device Specific Configuration} + +The management device MUST fail the device configuration space access +if the driver wants to access the range which is outside the config space. + +The management device MUST fail the device configuration space access +if \field{device_id} is 0. + +\subsection{MSI Configuration}\label{sec:Virtio Transport Options / Virtio Over +dmin Virtqueue / MSI Configuration} + +When VIRTIO_F_TRNSPT_VQ_VDEV is negotiated, +the MSI entry for a specific virtqueue is set through following command: +
content.tex \subsubsection{Resetting Devices}\label{sec:Virtio Transport Options / Virtio ov · line 3128
Quoted 10 lines
content.tex +10
-0+\begin{lstlisting} +#define VIRTIO_TRANSPTQ_CTRL_MSI 7 + #define VIRTIO_TRANSPTQ_CTRL_MSI_VQ_SET 0 + #define VIRTIO_TRANSPTQ_CTRL_MSI_VQ_ENABLE 1 + #define VIRTIO_TRANSPTQ_CTRL_MSI_VQ_MASK 2 + #define VIRTIO_TRANSPTQ_CTRL_MSI_CONFIG_SET 3 + #define VIRTIO_TRANSPTQ_CTRL_MSI_CONFIG_ENABLE 4 + #define VIRTIO_TRANSPTQ_CTRL_MSI_CONFIG_MASK 5 + +struct virtio_transportq_ctrl_vdev_msi_vq_set {
To be consistent with msi, let's use _vdev_mst_vq_config?
Quoted 19 lines
content.tex +19
-0+ u16 queue_index; + u64 addr; + u32 data; +}; + +struct virtio_transportq_ctrl_vdev_msi_vq_enable { + u16 queue_index; + u8 enable; +}; + +struct virtio_transportq_ctrl_vdev_msi_vq_mask { + u16 queue_index; + u8 mask; +}; + +struct virtio_transportq_ctrl_vdev_msi_config { + u64 addr; + u32 data; +};
content.tex \subsubsection{Resetting Devices}\label{sec:Virtio Transport Options / Virtio ov · line 3157
Quoted 10 lines
content.tex +10
-0+\end{lstlisting} + +The VIRTIO_TRANSPTQ_CTRL_MSI_VQ_SET command is used to set the MSI entry for a +specific virtqueue. The command-out-data is the virtqueue index and +the MSI address and data (as described in struct +virtio_transportq_ctrl_vdev_msix_vq_set). + +The VIRTIO_TRANSPTQ_CTRL_MSI_VQ_ENABLE command is used to enable or disable +the MSI interrupt for a specific virtqueue. The command-out-data is the +virtqueue index and whether to enable the MSI: 0 means to enable and 1
It might be better to use macros instead of magic numbers.
Quoted 28 lines
content.tex +28
-0+means to disable (as described in struct +virtio_transportq_ctrl_vdev_msi_vq_enable). + +The VIRTIO_TRANSPTQ_CTRL_MSI_VQ_MASK command is used to mask or unmask the +MSI interrupt for a specific virtqueue. The command-out-data is the +virtqueue index and the mask status: 0 means unmak and 1 means mask +(as described in struct virtio_transportq_ctrl_vdev_msi_vq_mask). + +The VIRTIO_TRANSPTQ_CTRL_MSI_CONFIG_SET command is used to set the MSI entry +for the config interrupt. The command-out-data is the MSI address and +data (as described in struct virtio_transportq_ctrl_vdev_msi_config). + +The VIRTIO_TRANSPTQ_CTRL_MSI_CONFIG_ENABLE command is used to enable and disable +MSI for config space. The command-out-data is an u8: 0 means to +disable and 1 means to enable. + +The VIRTIO_TRANSPTQ_CTRL_MSI_CONFIG_MASK command is used to mask and unmask MSI +interrupt for config space. The command-out-data is an u8: 0 means to +mask and 1 means to unmask. + +There's no command-in-data for all the above MSI commands. + +\devicenormative{\subsubsection}{MSI Configuration}{Virtio Transport Options / Virtio +ver Transport Virtqueue / MSI Configuration} + +The virtual device MUST record the pending MSI interrupts and +re-generate the pending MSI interrupts when unmasking. +
content.tex \subsubsection{Resetting Devices}\label{sec:Virtio Transport Options / Virtio ov · line 3195
Quoted 10 lines
content.tex +10
-0+The virtual device MUST disable the MSI for both virtqueue and config space +upon reset. + +\drivernormative{\subsubsection}{MSI Configuration}{Virtio Transport Options / Virtio +Over Transport Virtqueue / MSI Configuration} + +The driver MUST allocate transport or platform specific MSI entries +for both virtqueue and config space if it wants to use interrupt. + +The driver MAY choose disable the MSI if polling mode is used.
"choose to disable" ?
Quoted 127 lines
content.tex +127
-0+ +\subsection{Virtqueue Address}\label{sec:Virtio Transport Options / Virtio Over +Transport Virtqueue / Virtqueue Address} + +When VIRTIO_F_TRNSPT_VQ_VDEV is negotiated, +the addresses of a specific virtqueue are accessed through the following command: + +\begin{lstlisting} +#define VIRTIO_TRANSPTQ_CTRL_VQ_ADDR 8 + #define VIRTIO_TRANSPTQ_CTRL_VQ_ADDR_SET 0 + +struct virtio_transportq_ctrl_vdev_vq_addr { + u16 queue_index; + u64 device_area; + u64 descriptor_area; + u64 driver_area; +}; +\end{lstlisting} + +The command-out-data is the queue index, the addresses of device area, +descriptor area and driver area (as described in struct +virtio_transportq_ctrl_vdev_vq_addr); There's no command-in-data. + +\devicenormative{\subsubsection}{Virtqueue Address}{Virtio Transport Options / Virtio +Over Transport Virtqueue / Virtqueeu Address} + +The management device MUST fail the commands of class +VIRTIO_TRANSPTQ_CTRL_VQ_ADDR if \field{device_id} is 0. + +\subsection{Virtqueue Status}\label{sec:Virtio Transport Options / Virtio Over +Transport Virtqueue / Virtqueue Status} + +When VIRTIO_F_TRNSPT_VQ_VDEV is negotiated, +virtqueue status is set and get through the following command: + +\begin{lstlisting} +#define VIRTIO_TRANSPTQ_CTRL_VQ_ENABLE 9 + #define VIRTIO_TRANSPTQ_CTRL_VQ_ENABLE_SET 0 + #define VIRTIO_TRANSPTQ_CTRL_VQ_ENABLE_GET 1 + +struct virtio_transportq_ctrl_vq_status_set { + u16 queue_index; + u8 status; +}; + +\end{lstlisting} + +The VIRTIO_TRANSPTQ_CTRL_VQ_ENABLE_SET command is used to set the status of a +specific virtqueue. The command-out-data is the queue index and the +status that is set to the virtqueue (0 disabled, 1 enabled), +as described in struct virtio_transportq_ctrl_vq_status_set. +There's no command-in-data. + +The VIRTIO_TRANSPTQ_CTRL_VQ_ENABLE_GET is used to get the status of a +specific virtqueue. The command-out-data is an u16 of the queue +index. The command-in-data is the virtqueue status (0 disabled, 1 +enabled). + +\devicenormative{\subsubsection}{Virtqueue Status}{Virtio Transport Options / Virtio +Over Transport Virtqueue / Virtqueue Status} + +When disabled, the virtual device MUST stop processing requests from +this virtqueue. + +The management device MUST present a 0 via +VIRTIO_TRANSPTQ_CTRL_VQ_ENABLE_GET upon a reset of the virtual device. + +The management device MUST fail the virtqueue status access if +\field{device_id} is 0. + +\drivernormative{\subsubsection}{Virtqueue Status}{Virtio Transport Options / Virtio +Over Transport Virtqueue / Virtqueue Status} + +The driver MUST configure other virtqueue fields before enabling +the virtqueue with VIRTIO_TRANSPTQ_CTRL_VQ_ENABLE_SET. + +\subsection{Virtqueue Size}\label{sec:Virtio Transport Options / Virtio Over +Transport Virtqueue / Virtqueue Size} + +When VIRTIO_F_TRNSPT_VQ_VDEV is negotiated, +virtqueue size is accessed through the following command: + +\begin{lstlisting} +#define VIRTIO_TRANSPTQ_CTRL_VQ_SIZE 10 + #define VIRTIO_TRANSPTQ_CTRL_VQ_SIZE_SET 0 + #define VIRTIO_TRANSPTQ_CTRL_VQ_SIZE_GET 1 + +struct virtio_transportq_ctrl_vdev_vq_size_set { + u16 queue_index; + u16 size; +}; +\end{lstlisting} + +The VIRTIO_TRANSPTQ_CTRL_VQ_SIZE_SET command is used to set the virtqueue +size. The command-out-data is the queue index and the size of the +virtqueue (as described in struct +virtio_transportq_ctrl_vdev_vq_size_set). There's no command-in-data. + +The VIRTIO_TRANSPTQ_CTRL_VQ_SIZE_GET command is used to get the virtqueue +size. On reset, the maximum queue size supported by the device is +returned. The command-out-data is an u16 of the virtqueue index. The +command-in-data is an u16 of queue size for the virtqueue. + +\devicenormative{\subsubsection}{Virtqueue Status}{Virtio Transport Options / Virtio +Over Transport Virtqueue / Virtqueue Size} + +The management device MUST fail the virtqueue size access if +\field{device_id} is 0. + +\subsection{Virtqueue Notification}\label{sec:Virtio Transport Options / Virtio +Over Transport Virtqueue / Virtqueue Notification} + +When VIRTIO_F_TRNSPT_VQ_VDEV is negotiated, +the virtqueue notification is done through the following commands: + +\begin{lstlisting} +#define VIRTIO_TRANSPTQ_CTRL_VQ_NOTIFY 11 + #define VIRTIO_TRANSPTQ_CTRL_VQ_NOTIFY_GET 1 + +struct virtio_transportq_ctrl_vdev_vq_notification_area { + le64 addr + le64 size; +}; +\end{lstlisting} + +The VIRTIO_TRANSPTQ_CTRL_VQ_NOTIFY_GET is used to get the transport +specific address area that can be used to notify a virtqueue. The
content.tex \subsubsection{Resetting Devices}\label{sec:Virtio Transport Options / Virtio ov · line 3332
Quoted 10 lines
content.tex +10
-0+command-out-data is an u16 of the virtqueue index. The command-in-data +contains the address and the size of the notification area (as +described in struct virtio_transportq_ctrl_vdev_vq_notification_area). + +\devicenormative{\subsubsection}{Virtqueue Notification}{Virtio Transport Options / +Virtio Over Transport Virtqueue / Virtqueue Notification} + +The management device MUST fail the VIRTIO_TRANSPTQ_CTRL_VQ_NOTIFY_GET command +if there's no transport specific notification address for a virtqueue of +its virtual device.
Then how can we kick a device in this case?
Quoted 22 lines
content.tex +22
-0+ +The management device MUST fail the virtqueue notification access if +\field{device_id} is 0. + +The management device MUST forbid the notification area of a specific +virtual device to be accessed from another virtual device. + +\drivernormative{\subsubsection}{Virtqueue Notification}{Virtio Transport Options / +Virtio Over Transport Virtqueue / Virtqueue Notification} + +The driver MAY choose to notify the virtqueue by writing the queue +index at address \field{addr} which is fetched from the +VIRTIO_TRANSPTQ_CTRL_VQ_NOTIFY_GET command. + +\subsection{Virtqueue Index}\label{sec:Virtio Transport Options / Virtio +Over Transport Virtqueue / Virtqueue Index} + +When VIRTIO_F_TRNSPT_VQ_VDEV is negotiated, +the virtqueue available index is accessed through the following commands: + +\begin{lstlisting} +#define VIRTIO_TRANSPTQ_CTRL_VQ_AVAIL_INDEX 12
content.tex \subsubsection{Resetting Devices}\label{sec:Virtio Transport Options / Virtio ov · line 3364
Quoted 10 lines
content.tex +10
-0+ #define VIRTIO_TRANSPTQ_CTRL_VQ_AVAIL_INDEX_SET 0 + #define VIRTIO_TRANSPTQ_CTRL_VQ_ AVAIL_INDEX_GET 1 + +struct virtio_transportq_ctrl_vdev_vq_avail_idx_set { + u16 queue_index; + u16 avail_idx; +}; +\end{lstlisting} + +The VIRTIO_TRANSPTQ_CTRL_VQ_AVAIL_INDEX_SET commmand is used to set the available index of a specific virtqueue.
s/commmand/command/g
We need the support of packed virtqueue as well here.
content.tex \subsubsection{Resetting Devices}\label{sec:Virtio Transport Options / Virtio ov · line 3375
Quoted 11 lines
content.tex +11
-0+The command-out-data is the virtqueue index and the available index (as described in struct virtio_transportq_ctrl_vdev_vq_avail_idx). +There is no command-in-data. + +The VIRTIO_TRANSPTQ_CTRL_VQ_ AVAIL_INDEX_GET command is used to get the available index of a specific virtqueue. +The command-out-data is the queue index, the command-in-data is the available index of the queue. + +\devicenormative{\subsubsection}{Virtqueue Index}{Virtio Transport Options / +Virtio Over Transport Virtqueue / Virtqueue Index} + +The management device MUST fail the virtqueue index access if \field{device_id} is 0 +The management device MUST fail the virtqueue index access if \field{queue_id} is invalid.
We don't have similar normatives for other commands, anything make this command different?
Quoted 7 lines
content.tex +7
-0+ +\subsection{Presenting Virtual Devices}\label{sec:Virtio Transport Options / +Virtio Over Transport Virtqueue / Presenting Virtual Device} + +If VIRTIO_TRANSPTQ_F_VDEV is offered by a management device. The presenting of +the virtual devices requires co-operation between the management +driver and the transport virtqueue. This means, from the view of the
content.tex \subsubsection{Resetting Devices}\label{sec:Virtio Transport Options / Virtio ov · line 3392
Quoted 10 lines
content.tex +10
-0+virtual device driver, the transport is done via the communication +with the management device driver. It's up to the software to decide +what kind of method that is needed be used for those communications. + +The management driver typically takes the following steps for creating a +virtual device: + +\begin{enumerate} +\item Determine the virtio id and device specific configuration. +\item Create the virtual devices using the virtio_transportq_ctrl_vdev_CREATE command.
Need to use the upper case for the command name here.
content.tex \subsubsection{Resetting Devices}\label{sec:Virtio Transport Options / Virtio ov · line 3398
Quoted 6 lines
content.tex +6
-0+\item Optionally, configure the MSI. +\item Perform device specific setup. +\item Let the virtual device to be probed by the virtual device +driver. The management driver will then use the transport virtqueue to +implement the requests of basic facility from the virtual device +driver.
I think we need to be more verbose here, e.g to mention the actual command that is used.
content.tex \subsection{Virtqueues}\label{sec:Device Types / Network Device / Virtqueues} · line 3527
Quoted 9 lines
content.tex +2
-0+\end{enumerate} \chapter{Device Types}\label{sec:Device Types}@@ -2911,12 +3524,14 @@ \subsection{Virtqueues}\label{sec:Device Types / Network Device / Virtqueues} \item[2(N-1)] receiveqN \item[2(N-1)+1] transmitqN \item[2N] controlq +\item[2N+1] transportq
Let's use a separate patch for this.
Quoted 38 lines
content.tex +5
-2\end{description} N=1 if neither VIRTIO_NET_F_MQ nor VIRTIO_NET_F_RSS are negotiated, otherwise N is set by \field{max_virtqueue_pairs}. - controlq only exists if VIRTIO_NET_F_CTRL_VQ set. + controlq only exists if VIRTIO_NET_F_CTRL_VQ set.\\ + transportq only exists if VIRTIO_F_TRNSPT_VQ is offered. \subsection{Feature bits}\label{sec:Device Types / Network Device / Feature bits}@@ -4399,10 +5014,12 @@ \subsection{Virtqueues}\label{sec:Device Types / Block Device / Virtqueues} \item[0] requestq1 \item[\ldots] \item[N-1] requestqN +\item[N] transportq \end{description} N=1 if VIRTIO_BLK_F_MQ is not negotiated, otherwise N is set by - \field{num_queues}. + \field{num_queues}.\\ + transportq only exists if VIRTIO_F_TRNSPT_VQ is offered. \subsection{Feature bits}\label{sec:Device Types / Block Device / Feature bits}introduction.tex +3
-0@@ -79,6 +79,9 @@ \section{Normative References}\label{sec:Normative References} \phantomsection\label{intro:SCMI}\textbf{[SCMI]} & Arm System Control and Management Interface, DEN0056, \newline\url{https://developer.arm.com/docs/den0056/c}, version C and any future revisions\\ + \phantomsection\laçbel{intro:SIOV}\textbf{[SIOV]} & + Scalable I/O Virtualization, + \newline\url{https://www.opencompute.org/documents/ocp-scalable-io-virtualization-technical-specification-revision-1-v1-2-pdf}\\ \end{longtable} v
Quoted 8 lines · Jason Wang
Quoted 7 lines · Zhu Lingshan
This patch introduces a new device type: virtio virtual device. An virtio virtual device example is virtio based Scalable I/O Virtualization devices.
This commit also introduces a new transport other than PCI and MMIO - the transport virtqueue.
Quoted 7 lines
Quoted 6 lines
This transport is useful for implementing virtual devices with a limited transport specific resources or presenting the virtual device in a transport independent way.
This means, all the basic device facilities are provided solely via the the transport virtqueue. Additionally, the transport virtqueue is also in
Two "the".
will fix
Quoted 9 lines
Quoted 5 lines
charge of the creating and destroying of the virtual device.
With the help of the transport virtqueue, the presenting of the virtual device is done via the co-operation between the management device and its driver.
Having a dedicated virtqueue is probably fine but there is some overlap at the description of management/managed device with the admin virtqueue proposal from Max. Need to consider a way to unify the definition at least.
OK, I can re-phrase this part
Quoted 36 lines
Quoted 36 lines
Signed-off-by Jason Wang<[email protected]>Signed-off-by Zhu Lingshan<[email protected]>--- content.tex | 621 ++++++++++++++++++++++++++++++++++++++++++++++- introduction.tex | 3 + 2 files changed, 622 insertions(+), 2 deletions(-)content.tex +21
-0@@ -2784,6 +2784,619 @@ \subsubsection{Resetting Devices}\label{sec:Virtio Transport Options / Virtio ov In order to reset a device, a driver sends the CCW_CMD_VDEV_RESET command. +\section{Virtio Over Transport Virtqueue}\label{sec:Virtio Transport Options Virtio Over Transport Virtqueue} +Sometimes it's hard to implement the device in a transport specific method. +One example is that a physical device may try to present multiple virtual devices +with limited transport specific resources. +Another example is to implement virtual devices which are transport independent. +In those cases, the transport virtqueue provided by the management device could be used to replace +the transport specific method to implement the virtual device. +Then the presenting of the virtual device is done through the cooperation between +the transport virtqueue and the management device's driver. + +Feature bit VIRTIO_F_TRNSPT_VQ indicates that the device offers a transport virtqueue. + +\subsection{Basic Concepts}\label{sec:Virtio Transport Options / Virtio over Transport Virtqueue / Basic Concepts} + +All transport virtqueue commands are of the following form: + +\begin{lstlisting} +struct virtio_transportq_ctrl { + u64 device_id; + u16 class; + u16 command;
Quoted 8 lines
content.tex \subsubsection{Resetting Devices}\label{sec:Virtio Transport Options / Virtio ov · line 2812
Quoted 5 lines
content.tex +5
-0+ u32 length_out; + u8 command-out-data[]; + u32 length_in; + u32 ack; + u8 command-in-data[];Any reason we need length_in/length_out (we have length in descriptors)? FYI, we don't have them for ctrl virtqueue of networking devices.
OK, I can remove length_in and length_out, just pick the length from the descriptors.
Quoted 10 lines
Quoted 10 lines
content.tex +10
-0+}; + +/* ack values */ +#define VIRTIO_TRANSPTQ_OK 0 +#define VIRTIO_TRANSPTQ_EIO 1 +#define VIRTIO_TRANSPTQ_EAGAIN 11 +#define VIRTIO_TRANSPTQ_EBUSY 16 +#define VIRTIO_TRANSPTQ_EINVAL 22 +#define VIRTIO_TRANSPTQ_ERR 255 +\end{lstlisting}
Quoted 8 lines
content.tex \subsubsection{Resetting Devices}\label{sec:Virtio Transport Options / Virtio ov · line 2829
Quoted 7 lines
content.tex +7
-0+ +The \field{device_id}, \field{class}, \field{command}, \field{length_out} and +\field{command-out-data} are set by the management driver, +and the device sets the \field{ack}, \field{length_in} and \field{command-in-data}. + +\field{device_id} is an UUID to address a virtio virtual device, +The \field{device_id} value 0 is used for identify the management device itselfidentifying?
yes, will fix
Quoted 3 lines
content.tex \subsubsection{Resetting Devices}\label{sec:Virtio Transport Options / Virtio ov · line 2831
Quoted 2 lines
content.tex +2
-0+ +\subsubsection{The managment devices}\label{sec:Virtio Transport Options / Virtio over Transport Virtqueue / Basic Concepts / The management devices}s/managment/management/
will fix
Quoted 3 lines
content.tex \subsubsection{Resetting Devices}\label{sec:Virtio Transport Options / Virtio ov · line 2833
Quoted 2 lines
content.tex +2
-0+A device that offers a transport virtqueue (via feature VIRTIO_F_TRNSPT_VQ) +is the management device of the virtual devices which are the managed devices.Not a native speaker but I couldn't parse this sentence.
OK, I can re-phrase this one.
Quoted 6 lines
content.tex \subsubsection{Resetting Devices}\label{sec:Virtio Transport Options / Virtio ov · line 2837
Quoted 4 lines
content.tex +4
-0+ +For example, a PCIe PF with a transport virtqueue is a management device. + +\subsubsection{The virtual devices}\label{sec:Virtio Transport Options / Virtio over Transport Virtqueue / Basic Concepts / The virtual devices}I think to be consistent, per Michael's suggestion, we should use "managed" here which is better than "virtual".
do you mean replace all "virtual device" with "managed device" in this patch?
We can emphasize a virtual device is a type of managed device here, however "virtual device"
implies that the device is a logical device may not has a dedicated physical transport,
like SIOV devices (https://www.opencompute.org/documents/ocp-scalable-io-virtualization-technical-specification-revision-1-v1-2-pdf).
Quoted 8 lines
content.tex \subsubsection{Resetting Devices}\label{sec:Virtio Transport Options / Virtio ov · line 2843
Quoted 6 lines
content.tex +6
-0+A device that is created through a transport virtqueue which is offered by a management +device is a virtual device. Virtual devices are logical devices, managed by the management devices, +they usually don't have a dedicated physical transport. + +For example, a Scalable I/O Virtualization \hyperref[intro:SIOV]{[SIOV]} device which is created and managed through a transport virtqueue of +a management device is a virtual device.Maybe we can remove the "is a virtual device" here, since we use "For example" in the subsection of "The virtual device"
sure
Quoted 6 lines
content.tex \subsubsection{Resetting Devices}\label{sec:Virtio Transport Options / Virtio ov · line 2847
Quoted 4 lines
content.tex +4
-0+ +\devicenormative{\subsubsection}{Basic Concepts}{Virtio Transport Options / Virtio Over Transport Virtqueue / Basic Concepts} +The management device MUST offer feature bit VIRTIO_F_TRNSPT_VQ and a transport virtqueue as a device specific virtqueue.\\ +The virtual devices MUST not offer VIRTIO_F_TRNSPT_VQ feature.\\Is this a must? What's the value of prohibiting the nesting of the management device?
Of course it is not a must, I mean it is possible to implement VIRTIO_F_TRANSPT_VQ for a virtual device. But I am not sure whether it is meaningful to have nested virtual devices, creating virtual device on another virtual device, both layers are logical devices.
And this introduce extra complexities for hardware implementation and resource management, like usually don't create VFs from another VF.
Quoted 35 lines
Quoted 35 lines
content.tex +35
-0+The management device MUST fail all commands against the transport virtqueue if VIRTIO_F_TRNSPT_VQ is not negotiated. + +\drivernormative{\subsubsection}{Basic Concepts}{Virtio Transport Options / Virtio Over Transport Virtqueue / Basic Concepts} +The management driver MUST not send any commands against the transport virtqueue if VIRTIO_F_TRNSPT_VQ is not negotiated. + +\subsection{Virtual Devices Discovery}\label{sec:Virtio Transport Options /Virtio Over Transport Virtqueue / Virtual Devices Discovery} + +The management device is discovered through its own transport and device specific method. +Virtual devices are created and discovered via the transport virtqueue. + +\subsection{Transport Virtqueue Features}\label{sec:Virtio Transport Options /Virtio Over Transport Virtqueue / Transport Virtqueue Features} + +Feature bit VIRTIO_F_TRNSPT_VQ_VDEV indicates that the virtual devices are created, configured +and destroyed through the transport virtqueue. +This means the transport virtqueue is the transport for the virtual devices. + +Feature bit VIRTIO_F_TRNSPT_VQ_VDEV depends on VIRTIO_F_TRNSPT_VQ + +\devicenormative{\subsubsection}{Transport Virtqueue Features}{Virtio Transport Options / Virtio Over Transport Virtqueue / Transport Virtqueue Features} +The management device MUST not accept VIRTIO_F_TRNSPT_VQ_VDEV if VIRTIO_F_TRNSPT_VQ is not negotiated. + +\subsection{Device Management}\label{sec:Virtio Transport Options / Virtio Over Transport Virtqueue / Device Management} + +When VIRTIO_F_TRNSPT_VQ_VDEV is negotiated, +virtual devices must be created and destroyed through the transport virtqueue. + +\begin{lstlisting} +struct virtio_transportq_ctrl_vdev_attribute { + u32 virtio_device_id; + u32 asid; + u8 config[]; +}; + +#define virtio_transportq_ctrl_vdev 1 (class) + #define virtio_transportq_ctrl_vdev_CREATE 0 (command)
Quoted 8 lines
content.tex \subsubsection{Resetting Devices}\label{sec:Virtio Transport Options / Virtio ov · line 2888
Quoted 6 lines
content.tex +6
-0+ #define virtio_transportq_ctrl_vdev_DESTROY 1 (command) +\end{lstlisting} + +The virtio_transportq_ctrl_vdev_CREATE command is used to create a virtual device. +The command-out-data for VIRTIO_TRANSPTQ_CTRL_CREATE is the virtio device id (\field{virtio_device_id}), +the address space id (\field{asid})We need to be verbose here. Is this a transport specific identifier or a device specific identifier?
This asid field is an id for the device, it should be the PASID. I will add more explanations here.
Quoted 5 lines
I also wonder if this is the best place for this. To me, it looks more suitable to have a dedicated command to configure it before
DRIVER_OKlike virtqueue address. Then we can configure it per virtqueue per device with more flexibility.
I understand the point, this can gives the device more flexibility for sure.
It is the guest driver initialize the device. Device status and vq configs are usually set by the guest driver, and the guest driver is not aware of the asid, it is the host side setting the asid.
So shall we set the asid upon DRIVER_OK?
And how to decide whether the device should use a device-global asid or a per-vq asid,
or even hybrid? I think this is transparent to the guest
Quoted 7 lines
content.tex \subsubsection{Resetting Devices}\label{sec:Virtio Transport Options / Virtio ov · line 2892
Quoted 5 lines
content.tex +4
-0+and device specific configuration (\field{config}) for creating the virtual device. +When succeed, the device returns an u64 as a unique identifier of the created virtual device in command-in-data. + +The field \field{config} is device type specific. E.g., for virtio-net, it should follow the struct virtio_net_configI think we need a dedicated section to describe the transport virtqueue support for virtio-net.
More hints for what should be discussed in the section?
I see all the descriptions for virtio device types are in chapter 5 Device Types, other transports don't have their own sections for the device types, and chapter 5 discussed that in general.
Here we take virtio-net device as an example to explain the filed config contents.
Quoted 13 lines
content.tex \subsubsection{Resetting Devices}\label{sec:Virtio Transport Options / Virtio ov · line 2903
Quoted 12 lines
content.tex +11
-0+in section \ref{sec:Device Types / Network Device / Device configuration layout} + +The virtio_transportq_ctrl_vdev_DESTROY command is used to destroy a +virtual device which is identified by its 64bit identifier +\field{virtual_device_id}. There's no command-in-data for +VIRTIO_TRANSPTQ_CTRL_DESTROY command. + +\devicenormative{\subsubsection}{Device Management}{Virtio Transport Options / Virtio Over Transport Virtqueue / Device Management} + +The management device MUST fail the virtio_transportq_ctrl_vdev_CREATE if +\field{device_id} is not 0.Let's use
VIRTIO_TRANSPTQ_OKinstead of the magic number here.
this zero is the filed device_id = 0, means identifying the management device. And I found I have mixed up the upper and lower cases in the commands, they should all be upper cases, I will fix this.
Quoted 11 lines
content.tex \subsubsection{Resetting Devices}\label{sec:Virtio Transport Options / Virtio ov · line 2906
Quoted 4 lines
content.tex +3
-0+ +The management device MUST fail the virtio_transportq_ctrl_vdev_DESTROY if +\field{device_id} is 0.Same here.
content.tex \subsubsection{Resetting Devices}\label{sec:Virtio Transport Options / Virtio ov · line 2909
Quoted 3 lines
content.tex +3
-0+ +All virtual devices MUST be created via the transport virtqueue if the management +device offers VIRTIO_F_TRNSPT_VQ_VDEV.Does this imply no pre-created virtual devices? If this is true, we don't need a dedicated feature bit (
VIRTIO_F_TRNSPT_VQ_VDEV).
There can be pre-created virtual devices, in the following sections,
struct mgmt_dev_avail_res{}, if min_vdev_vqs == max_vdev_vqs, means the management device only supports creating virtual devices with fixed capabilities, this kind of virtual devices can be pre-created. Like the slab allocator, vendors can pre-create the device. Then when the management device receiving the VDEV_CREATE command, it can just pick up a free virtual device, and return the virtual device.
Quoted 4 lines
The bit only makes sense for the case when some managed(virtual) devices are pre-created like
initial_vfsin SR-IOV. If we allow pre-creation, we need commands to discover them.
I think we need the feature bit VIRTIO_F_TRNSPT_VQ_VDEV, this feature bit indicates that the transport virtqueue is the transport layer for the virtual devices, like PCI for a virtio PCI hardware. So, not only create, we use the transport virtqueue to config the device as well.
And if the virtual devices are pre-created, we can just send the create command, and the management device can find a available device for us, this is also discovery.
Quoted 17 lines
Quoted 5 lines
content.tex +4
-0+ +\drivernormative{\subsubsection}{Driver Management}{Virtio Transport Options / Virtio Over Transport Virtqueue / Device Management} + +The management driver MUST use 0 as \field{device_id} forcontent.tex \subsubsection{Resetting Devices}\label{sec:Virtio Transport Options / Virtio ov · line 2923
Quoted 10 lines
content.tex +10
-0+virtio_transportq_ctrl_vdev_CREATE command. + +\subsection{Available resource of the management device}\label{sec:Virtio Transport Options / Virtio Over Transport Virtqueue / Available resource of the management devic} +When VIRTIO_F_TRNSPT_VQ_VDEV is negotiated, +statistical information of available resource of the management device could be fetched +by the following command: + +\begin{lstlisting} +#define VIRTIO_TRANSPTQ_CTRL_AVAIL_RES 2 + #define VIRTIO_TRANSPTQ_CTRL_AVAIL_RES_GET 0Nit, I think this command should come first, mgmt need to query this before creating devices.
sure, I can re-order the commands.
Quoted 5 lines
content.tex \subsubsection{Resetting Devices}\label{sec:Virtio Transport Options / Virtio ov · line 2926
Quoted 4 lines
content.tex +3
-0+\end{lstlisting} + +VIRTIO_TRANSPTQ_CTRL_AVAIL_RES_GET command is used to get the statstical information of availables/statstical/statistical/
will fix
Quoted 9 lines
content.tex \subsubsection{Resetting Devices}\label{sec:Virtio Transport Options / Virtio ov · line 2932
Quoted 7 lines
content.tex +6
-0+resource of the management device. There is no command-out-data for VIRTIO_TRANSPTQ_CTRL_AVAIL_RES_GET +command. The management device fills command-in-data in the following format: + +\begin{lstlisting} +struct mgmt_dev_avail_res{ + /* The number of total remaining virtqueus of the management device */Should be virtqueues.
Quoted 3 lines
Actually the description here is confusing, something like this might be better
"The number of total remaining virtqueues for the managed(virtual) devices */
will fix this.
Quoted 4 lines
content.tex \subsubsection{Resetting Devices}\label{sec:Virtio Transport Options / Virtio ov · line 2934
Quoted 3 lines
content.tex +2
-0+ u16 num_avail_vqs; + /* The minimal number of virtqueues of the virtual devices */"of a virtual device"?
will fix
Quoted 49 lines
Quoted 49 lines
content.tex +48
-0+ u16 min_vdev_vqs; + /* The maximum number of virtqueues of the virtual devices */ + u16 max_vdev_vqs; + /* The number of virtual devices that can be created with min_vdev_vqs virtqueues. + * This number may not equal to num_avail_vqs divided by min_vdev_vqs due to + * the limitations of other resource + */ + u16 num_avail_vdev; +}; +\end{lstlisting} + +If \field{min_vdev_vqs} equals to \field{min_vdev_vqs}, that means the management device +can only create virtual devices with a fixed number of virtqueues + +\devicenormative{\subsubsection}{Available resource of the management device}{Virtio Transport Options / +Virtio Over Transport Virtqueue / Available resource of the management device} +The management device MUST fail the VIRTIO_TRANSPTQ_CTRL_AVAIL_RES_GET command if \field{device_id} is not 0 + +\drivernormative{\subsubsection}{Available resource of the management device}{Virtio Transport Options / +Virtio Over Transport Virtqueue / Available resource of the management device} +The management driver MUST use 0 as \field{device_id} for +VIRTIO_TRANSPTQ_CTRL_AVAIL_RES_GET command. + +\subsection{Features Negotiation}\label{sec:Virtio Transport Options / Virtio +Over Transport Virtqueue / Features Negotiation} + +When VIRTIO_F_TRNSPT_VQ_VDEV is negotiated, +the feature negotiation of virtual devices is done by the +following commands: + +\begin{lstlisting} +#define VIRTIO_TRANSPTQ_CTRL_FEAT 3 + #define VIRTIO_TRANSPTQ_CTRL_FEAT_DEVICE_GET 0 + #define VIRTIO_TRANSPTQ_CTRL_FEAT_DRIVER_SET 1 + #define VIRTIO_TRANSPTQ_CTRL_FEAT_DRIVER_GET 2 +\end{lstlisting} + +The VIRTIO_TRANSPTQ_CTRL_FEAT_DEVICE_GET is to get the features offered +by a virtual device. + +The VIRTIO_TRANSPTQ_CTRL_FEAT_DRIVER_SET is for driver to accept feature +bits offered by the virtual device. + +The VIRTIO_TRANSPTQ_CTRL_FEAT_DRIVER_GET is to get the features accepted +by both the virtual driver and the device. + +The features is 64 bits mask of the virtio features bit. For +VIRTIO_TRANSPTQ_CTRL_DRIVER_SET, the feature is passed to the device
Quoted 8 lines
content.tex \subsubsection{Resetting Devices}\label{sec:Virtio Transport Options / Virtio ov · line 2988
Quoted 6 lines
content.tex +6
-0+through command-out-data. For VIRTIO_TRANSPTQ_CTRL_FEAT_DEVICE_GET and +VIRTIO_TRANSPTQ_CTRL_DRIVER_GET the feature is returned for the device +through command-in-data. + +\devicenormative{\subsubsection}{Features Negotiation}{Virtio Transport Options / +Virtio Over Transport Virtqueue / Features Negotiation}Can one of the above three commands fail? If yes, we need to document the behaviour.
can the ack values cover this? Like if failed to read the device feature bits due to IO errors, it can set this EIO below. Or do I miss something?
+/* ack values */
+#define VIRTIO_TRANSPTQ_OK 0
+#define VIRTIO_TRANSPTQ_EIO 1
+#define VIRTIO_TRANSPTQ_EAGAIN 11
+#define VIRTIO_TRANSPTQ_EBUSY 16
+#define VIRTIO_TRANSPTQ_EINVAL 22
+#define VIRTIO_TRANSPTQ_ERR 255
+\end{lstlisting}
Quoted 56 lines
Quoted 44 lines
content.tex +43
-0+ +The management device MUST fail VIRTIO_TRANSPTQ_F_CTRL_FEAT class for the +command that use 0 as its \field{device_id}. + +\drivernormative{\subsubsection}{Features Negotiation}{Virtio Transport Options / +Virtio Over Transport Virtqueue / Features Negotiation} + +The management driver MAY mediate between the feature negotiation +request of the virtual devices and the transport virtqueue. E.g when +offering features to the virtual device, the management driver MAY +exclude some features in order to limit the behaviour of the virtual +device. + +\subsection{Device Status}\label{sec:Virtio Transport Options / Virtio Over +Transport Virtqueue / Device Status} + +When VIRTIO_F_TRNSPT_VQ_VDEV is negotiated, +the status of virtual device is accessed by the following +commands: + +\begin{lstlisting} +#define VIRTIO_TRANSPTQ_CTRL_STATUS 4 + #define VIRTIO_TRANSPTQ_CTRL_STATUS_SET 0 + #define VIRTIO_TRANSPTQ_CTRL_STATUS_GET 1 +\end{lstlisting} + +The VIRTIO_TRANSPTQ_CTRL_STATUS_SET is used to set the device status of +a virtual device here. The command-out-data is the one byte status +to set to the device. There's no command-in-data for this command. + +The VIRTIO_TRANSPTQ_CTRL_STATUS_GET is used to get the device status of +a virtual device. The command-in-data is the one byte status +returned from the device. There's no command-out-data for this +command. + +\devicenormative{\subsubsection}{Device Status}{Virtio Transport Options / Virtio +Over Transport Virtqueue / Device Status} + +The management device MUST reset the virtual device when 0 +is written via VIRTIO_TRANSPTQ_CTRL_STATUS_SET, the success of this +command demonstrate the success of the reset. + +The management device MUST present 0 throughcontent.tex \subsubsection{Resetting Devices}\label{sec:Virtio Transport Options / Virtio ov · line 3041
Quoted 10 lines
content.tex +10
-0+VIRTIO_TRANSPTQ_CTRL_STATUS_GET once the reset is done. + +The management device MUST fail the device status access if +\field{device_id} is 0. + +\drivernormative{\subsubsection}{Device Status}{Virtio Transport Options / Virtio +Over Transport Virtqueue / Device Status} + +After writing 0 via VIRTIO_TRANSPTQ_CTRL_STATUS_SET, the driver MUST wait +for the success of the command before re-initializing the device.So we need to clarify which is the condition for the succeed of a reset:
Quoted 2 lines
1) the succeed of
STATUS_SET(this is how ccw did)
2) 0 is read fromSTATUS_GET(this is how pci did)
sure, I think we can use the pci way, present 0 after reset.
Quoted 81 lines
Quoted 81 lines
content.tex +80
-0+ +\subsection{Device Generation}\label{sec:Virtio Transport Options / Virtio Over Transport Virtqueue / Device Generation} + +When VIRTIO_F_TRNSPT_VQ_VDEV is negotiated, +the device generation count is read from the following commands: + +\begin{lstlisting} +#define VIRTIO_TRANSPTQ_CTRL_GENERATION 5 + #define VIRTIO_TRANSPTQ_CTRL_GENERATION_GET 0 +\end{lstlisting} + +The VIRTIO_TRANSPTQ_CTRL_GENERATION_GET is used to get the device configuration generation field +of the virtual device. The command-in-data is the u32 device +generation returned from the device. There's no command-out-data for +this command. + +\devicenormative{\subsubsection}{Device Generation}{Virtio Transport Options / Virtio Over Transport Virtqueue / Device Generation} + +The management device MUST present a changed generation count after the driver +has read a device-specific configuration value which has changed since +any part of the device-specific configuration was last read. + +The management device MUST fail the device generation access if \field{device_id} is 0. + +\subsection{Device Specific Configuration}\label{sec:Virtio Transport Options / +Virtio Over Transport Virtqueue / Device Specific Configuration} + +When VIRTIO_F_TRNSPT_VQ_VDEV is negotiated, +the device config space contents of a virtual device are accessed through +VIRTIO_TRANSPTQ_CTRL_CONFIG_GET and VIRTIO_TRANSPTQ_CTRL_CONFIG_SET. + +\begin{lstlisting} +#define VIRTIO_TRANSPTQ_CTRL_CONFIG 6 + #define VIRTIO_TRANSPTQ_CTRL_CONFIG_GET 0 + #define VIRTIO_TRANSPTQ_CTRL_CONFIG_SET 1 + +struct virtio_transportq_ctrl_vdev_config_get { + u32 offset; + u32 size; +}; + +struct virtio_transportq_ctrl_vdev_config_set { + u32 offset; + u32 size; + u8 data[]; +}; +\end{lstlisting} + +The VIRTIO_TRANSPTQ_CTRL_CONFIG_GET is used to read data from the +device configuration space. As described in struct +virtio_transportq_ctrl_vdev_config_get, The command-out-data is the offset +from the start of the config space and the size of the data. The +command-in-data is an array of u8 data that read from the config +space. + +The VIRTIO_TRANSPTQ_CTRL_CONFIG_SET is used to write data to the device +configuration space. As described in struct +virtio_transportq_ctrl_vdev_config_set, the command-out-data contains the +offset from the start of the config space, the size of the data and +the data that will be written. There's no command-in-data for this +command. + +\devicenormative{\subsubsection}{Device Specific Configuration}{Virtio Transport +Options / Virtio Over Transport Virtqueue / Device Specific Configuration} + +The management device MUST fail the device configuration space access +if the driver wants to access the range which is outside the config space. + +The management device MUST fail the device configuration space access +if \field{device_id} is 0. + +\subsection{MSI Configuration}\label{sec:Virtio Transport Options / Virtio Over +dmin Virtqueue / MSI Configuration} + +When VIRTIO_F_TRNSPT_VQ_VDEV is negotiated, +the MSI entry for a specific virtqueue is set through following command: + +\begin{lstlisting} +#define VIRTIO_TRANSPTQ_CTRL_MSI 7 + #define VIRTIO_TRANSPTQ_CTRL_MSI_VQ_SET 0
Quoted 8 lines
content.tex \subsubsection{Resetting Devices}\label{sec:Virtio Transport Options / Virtio ov · line 3128
Quoted 7 lines
content.tex +7
-0+ #define VIRTIO_TRANSPTQ_CTRL_MSI_VQ_ENABLE 1 + #define VIRTIO_TRANSPTQ_CTRL_MSI_VQ_MASK 2 + #define VIRTIO_TRANSPTQ_CTRL_MSI_CONFIG_SET 3 + #define VIRTIO_TRANSPTQ_CTRL_MSI_CONFIG_ENABLE 4 + #define VIRTIO_TRANSPTQ_CTRL_MSI_CONFIG_MASK 5 + +struct virtio_transportq_ctrl_vdev_msi_vq_set {To be consistent with msi, let's use
_vdev_mst_vq_config?
OK, it will be virtio_transportq_ctrl_vdev_msi_vq_config in the next version
Quoted 23 lines
Quoted 23 lines
content.tex +22
-0+ u16 queue_index; + u64 addr; + u32 data; +}; + +struct virtio_transportq_ctrl_vdev_msi_vq_enable { + u16 queue_index; + u8 enable; +}; + +struct virtio_transportq_ctrl_vdev_msi_vq_mask { + u16 queue_index; + u8 mask; +}; + +struct virtio_transportq_ctrl_vdev_msi_config { + u64 addr; + u32 data; +}; +\end{lstlisting} + +The VIRTIO_TRANSPTQ_CTRL_MSI_VQ_SET command is used to set the MSI entry for a
Quoted 8 lines
content.tex \subsubsection{Resetting Devices}\label{sec:Virtio Transport Options / Virtio ov · line 3157
Quoted 7 lines
content.tex +7
-0+specific virtqueue. The command-out-data is the virtqueue index and +the MSI address and data (as described in struct +virtio_transportq_ctrl_vdev_msix_vq_set). + +The VIRTIO_TRANSPTQ_CTRL_MSI_VQ_ENABLE command is used to enable or disable +the MSI interrupt for a specific virtqueue. The command-out-data is the +virtqueue index and whether to enable the MSI: 0 means to enable and 1It might be better to use macros instead of magic numbers.
sure!
Quoted 32 lines
Quoted 32 lines
content.tex +31
-0+means to disable (as described in struct +virtio_transportq_ctrl_vdev_msi_vq_enable). + +The VIRTIO_TRANSPTQ_CTRL_MSI_VQ_MASK command is used to mask or unmask the +MSI interrupt for a specific virtqueue. The command-out-data is the +virtqueue index and the mask status: 0 means unmak and 1 means mask +(as described in struct virtio_transportq_ctrl_vdev_msi_vq_mask). + +The VIRTIO_TRANSPTQ_CTRL_MSI_CONFIG_SET command is used to set the MSI entry +for the config interrupt. The command-out-data is the MSI address and +data (as described in struct virtio_transportq_ctrl_vdev_msi_config). + +The VIRTIO_TRANSPTQ_CTRL_MSI_CONFIG_ENABLE command is used to enable and disable +MSI for config space. The command-out-data is an u8: 0 means to +disable and 1 means to enable. + +The VIRTIO_TRANSPTQ_CTRL_MSI_CONFIG_MASK command is used to mask and unmask MSI +interrupt for config space. The command-out-data is an u8: 0 means to +mask and 1 means to unmask. + +There's no command-in-data for all the above MSI commands. + +\devicenormative{\subsubsection}{MSI Configuration}{Virtio Transport Options / Virtio +ver Transport Virtqueue / MSI Configuration} + +The virtual device MUST record the pending MSI interrupts and +re-generate the pending MSI interrupts when unmasking. + +The virtual device MUST disable the MSI for both virtqueue and config space +upon reset. +
Quoted 8 lines
content.tex \subsubsection{Resetting Devices}\label{sec:Virtio Transport Options / Virtio ov · line 3195
Quoted 7 lines
content.tex +7
-0+\drivernormative{\subsubsection}{MSI Configuration}{Virtio Transport Options / Virtio +Over Transport Virtqueue / MSI Configuration} + +The driver MUST allocate transport or platform specific MSI entries +for both virtqueue and config space if it wants to use interrupt. + +The driver MAY choose disable the MSI if polling mode is used."choose to disable" ?
when use polling mode, I think it is not a must to disable the interrupts. So the driver may disable the interrupts for better performance or not.
Did I miss something?
Quoted 131 lines
Quoted 131 lines
content.tex +130
-0+ +\subsection{Virtqueue Address}\label{sec:Virtio Transport Options / Virtio Over +Transport Virtqueue / Virtqueue Address} + +When VIRTIO_F_TRNSPT_VQ_VDEV is negotiated, +the addresses of a specific virtqueue are accessed through the following command: + +\begin{lstlisting} +#define VIRTIO_TRANSPTQ_CTRL_VQ_ADDR 8 + #define VIRTIO_TRANSPTQ_CTRL_VQ_ADDR_SET 0 + +struct virtio_transportq_ctrl_vdev_vq_addr { + u16 queue_index; + u64 device_area; + u64 descriptor_area; + u64 driver_area; +}; +\end{lstlisting} + +The command-out-data is the queue index, the addresses of device area, +descriptor area and driver area (as described in struct +virtio_transportq_ctrl_vdev_vq_addr); There's no command-in-data. + +\devicenormative{\subsubsection}{Virtqueue Address}{Virtio Transport Options / Virtio +Over Transport Virtqueue / Virtqueeu Address} + +The management device MUST fail the commands of class +VIRTIO_TRANSPTQ_CTRL_VQ_ADDR if \field{device_id} is 0. + +\subsection{Virtqueue Status}\label{sec:Virtio Transport Options / Virtio Over +Transport Virtqueue / Virtqueue Status} + +When VIRTIO_F_TRNSPT_VQ_VDEV is negotiated, +virtqueue status is set and get through the following command: + +\begin{lstlisting} +#define VIRTIO_TRANSPTQ_CTRL_VQ_ENABLE 9 + #define VIRTIO_TRANSPTQ_CTRL_VQ_ENABLE_SET 0 + #define VIRTIO_TRANSPTQ_CTRL_VQ_ENABLE_GET 1 + +struct virtio_transportq_ctrl_vq_status_set { + u16 queue_index; + u8 status; +}; + +\end{lstlisting} + +The VIRTIO_TRANSPTQ_CTRL_VQ_ENABLE_SET command is used to set the status of a +specific virtqueue. The command-out-data is the queue index and the +status that is set to the virtqueue (0 disabled, 1 enabled), +as described in struct virtio_transportq_ctrl_vq_status_set. +There's no command-in-data. + +The VIRTIO_TRANSPTQ_CTRL_VQ_ENABLE_GET is used to get the status of a +specific virtqueue. The command-out-data is an u16 of the queue +index. The command-in-data is the virtqueue status (0 disabled, 1 +enabled). + +\devicenormative{\subsubsection}{Virtqueue Status}{Virtio Transport Options / Virtio +Over Transport Virtqueue / Virtqueue Status} + +When disabled, the virtual device MUST stop processing requests from +this virtqueue. + +The management device MUST present a 0 via +VIRTIO_TRANSPTQ_CTRL_VQ_ENABLE_GET upon a reset of the virtual device. + +The management device MUST fail the virtqueue status access if +\field{device_id} is 0. + +\drivernormative{\subsubsection}{Virtqueue Status}{Virtio Transport Options / Virtio +Over Transport Virtqueue / Virtqueue Status} + +The driver MUST configure other virtqueue fields before enabling +the virtqueue with VIRTIO_TRANSPTQ_CTRL_VQ_ENABLE_SET. + +\subsection{Virtqueue Size}\label{sec:Virtio Transport Options / Virtio Over +Transport Virtqueue / Virtqueue Size} + +When VIRTIO_F_TRNSPT_VQ_VDEV is negotiated, +virtqueue size is accessed through the following command: + +\begin{lstlisting} +#define VIRTIO_TRANSPTQ_CTRL_VQ_SIZE 10 + #define VIRTIO_TRANSPTQ_CTRL_VQ_SIZE_SET 0 + #define VIRTIO_TRANSPTQ_CTRL_VQ_SIZE_GET 1 + +struct virtio_transportq_ctrl_vdev_vq_size_set { + u16 queue_index; + u16 size; +}; +\end{lstlisting} + +The VIRTIO_TRANSPTQ_CTRL_VQ_SIZE_SET command is used to set the virtqueue +size. The command-out-data is the queue index and the size of the +virtqueue (as described in struct +virtio_transportq_ctrl_vdev_vq_size_set). There's no command-in-data. + +The VIRTIO_TRANSPTQ_CTRL_VQ_SIZE_GET command is used to get the virtqueue +size. On reset, the maximum queue size supported by the device is +returned. The command-out-data is an u16 of the virtqueue index. The +command-in-data is an u16 of queue size for the virtqueue. + +\devicenormative{\subsubsection}{Virtqueue Status}{Virtio Transport Options / Virtio +Over Transport Virtqueue / Virtqueue Size} + +The management device MUST fail the virtqueue size access if +\field{device_id} is 0. + +\subsection{Virtqueue Notification}\label{sec:Virtio Transport Options / Virtio +Over Transport Virtqueue / Virtqueue Notification} + +When VIRTIO_F_TRNSPT_VQ_VDEV is negotiated, +the virtqueue notification is done through the following commands: + +\begin{lstlisting} +#define VIRTIO_TRANSPTQ_CTRL_VQ_NOTIFY 11 + #define VIRTIO_TRANSPTQ_CTRL_VQ_NOTIFY_GET 1 + +struct virtio_transportq_ctrl_vdev_vq_notification_area { + le64 addr + le64 size; +}; +\end{lstlisting} + +The VIRTIO_TRANSPTQ_CTRL_VQ_NOTIFY_GET is used to get the transport +specific address area that can be used to notify a virtqueue. The +command-out-data is an u16 of the virtqueue index. The command-in-data +contains the address and the size of the notification area (as +described in struct virtio_transportq_ctrl_vdev_vq_notification_area).
Quoted 8 lines
content.tex \subsubsection{Resetting Devices}\label{sec:Virtio Transport Options / Virtio ov · line 3332
Quoted 7 lines
content.tex +7
-0+ +\devicenormative{\subsubsection}{Virtqueue Notification}{Virtio Transport Options / +Virtio Over Transport Virtqueue / Virtqueue Notification} + +The management device MUST fail the VIRTIO_TRANSPTQ_CTRL_VQ_NOTIFY_GET command +if there's no transport specific notification address for a virtqueue of +its virtual device.Then how can we kick a device in this case?
I think I should remove this sub-section. Because there should be a notification area, or it is a defective device, and if failed to get the notification area, the command should set a proper ack value reflecting the error
Quoted 35 lines
Quoted 23 lines
content.tex +22
-0+ +The management device MUST fail the virtqueue notification access if +\field{device_id} is 0. + +The management device MUST forbid the notification area of a specific +virtual device to be accessed from another virtual device. + +\drivernormative{\subsubsection}{Virtqueue Notification}{Virtio Transport Options / +Virtio Over Transport Virtqueue / Virtqueue Notification} + +The driver MAY choose to notify the virtqueue by writing the queue +index at address \field{addr} which is fetched from the +VIRTIO_TRANSPTQ_CTRL_VQ_NOTIFY_GET command. + +\subsection{Virtqueue Index}\label{sec:Virtio Transport Options / Virtio +Over Transport Virtqueue / Virtqueue Index} + +When VIRTIO_F_TRNSPT_VQ_VDEV is negotiated, +the virtqueue available index is accessed through the following commands: + +\begin{lstlisting} +#define VIRTIO_TRANSPTQ_CTRL_VQ_AVAIL_INDEX 12content.tex \subsubsection{Resetting Devices}\label{sec:Virtio Transport Options / Virtio ov · line 3364
Quoted 10 lines
content.tex +10
-0+ #define VIRTIO_TRANSPTQ_CTRL_VQ_AVAIL_INDEX_SET 0 + #define VIRTIO_TRANSPTQ_CTRL_VQ_ AVAIL_INDEX_GET 1 + +struct virtio_transportq_ctrl_vdev_vq_avail_idx_set { + u16 queue_index; + u16 avail_idx; +}; +\end{lstlisting} + +The VIRTIO_TRANSPTQ_CTRL_VQ_AVAIL_INDEX_SET commmand is used to set the available index of a specific virtqueue.s/commmand/command/g
Quoted 1 line
We need the support of packed virtqueue as well here.
sure, will work on this.
Quoted 14 lines
content.tex \subsubsection{Resetting Devices}\label{sec:Virtio Transport Options / Virtio ov · line 3375
Quoted 12 lines
content.tex +11
-0+The command-out-data is the virtqueue index and the available index (as described in struct virtio_transportq_ctrl_vdev_vq_avail_idx). +There is no command-in-data. + +The VIRTIO_TRANSPTQ_CTRL_VQ_ AVAIL_INDEX_GET command is used to get the available index of a specific virtqueue. +The command-out-data is the queue index, the command-in-data is the available index of the queue. + +\devicenormative{\subsubsection}{Virtqueue Index}{Virtio Transport Options / +Virtio Over Transport Virtqueue / Virtqueue Index} + +The management device MUST fail the virtqueue index access if \field{device_id} is 0 +The management device MUST fail the virtqueue index access if \field{queue_id} is invalid.We don't have similar normatives for other commands, anything make this command different?
If it is PCI, if we set queue_select to an invalid value, I think it is a undefined behavior, this could be ignored by the hardware, the device has no ways to report such errors.
Here, for the transport virtqueue, it is a must to return an ACK value, and it can be meaningful, like VIRTIO_TRANSPTQ_EINVAL
Quoted 11 lines
Quoted 11 lines
content.tex +10
-0+ +\subsection{Presenting Virtual Devices}\label{sec:Virtio Transport Options / +Virtio Over Transport Virtqueue / Presenting Virtual Device} + +If VIRTIO_TRANSPTQ_F_VDEV is offered by a management device. The presenting of +the virtual devices requires co-operation between the management +driver and the transport virtqueue. This means, from the view of the +virtual device driver, the transport is done via the communication +with the management device driver. It's up to the software to decide +what kind of method that is needed be used for those communications.
Quoted 8 lines
content.tex \subsubsection{Resetting Devices}\label{sec:Virtio Transport Options / Virtio ov · line 3392
Quoted 7 lines
content.tex +7
-0+ +The management driver typically takes the following steps for creating a +virtual device: + +\begin{enumerate} +\item Determine the virtio id and device specific configuration. +\item Create the virtual devices using the virtio_transportq_ctrl_vdev_CREATE command.Need to use the upper case for the command name here.
yes
Quoted 9 lines
content.tex \subsubsection{Resetting Devices}\label{sec:Virtio Transport Options / Virtio ov · line 3398
Quoted 7 lines
content.tex +6
-0+\item Optionally, configure the MSI. +\item Perform device specific setup. +\item Let the virtual device to be probed by the virtual device +driver. The management driver will then use the transport virtqueue to +implement the requests of basic facility from the virtual device +driver.I think we need to be more verbose here, e.g to mention the actual command that is used.
sure, will add these
Quoted 11 lines
content.tex \subsection{Virtqueues}\label{sec:Device Types / Network Device / Virtqueues} · line 3527
Quoted 10 lines
content.tex +2
-0+\end{enumerate} \chapter{Device Types}\label{sec:Device Types}@@ -2911,12 +3524,14 @@ \subsection{Virtqueues}\label{sec:Device Types / Network Device / Virtqueues} \item[2(N-1)] receiveqN \item[2(N-1)+1] transmitqN \item[2N] controlq +\item[2N+1] transportqLet's use a separate patch for this.
can do.
Thanks for your review!
Quoted 69 lines
Quoted 31 lines
content.tex +5
-2\end{description} N=1 if neither VIRTIO_NET_F_MQ nor VIRTIO_NET_F_RSS are negotiated, otherwise N is set by \field{max_virtqueue_pairs}. - controlq only exists if VIRTIO_NET_F_CTRL_VQ set. + controlq only exists if VIRTIO_NET_F_CTRL_VQ set.\\ + transportq only exists if VIRTIO_F_TRNSPT_VQ is offered. \subsection{Feature bits}\label{sec:Device Types / Network Device / Feature bits}@@ -4399,10 +5014,12 @@ \subsection{Virtqueues}\label{sec:Device Types / Block Device / Virtqueues} \item[0] requestq1 \item[\ldots] \item[N-1] requestqN +\item[N] transportq \end{description} N=1 if VIRTIO_BLK_F_MQ is not negotiated, otherwise N is set by - \field{num_queues}. + \field{num_queues}.\\ + transportq only exists if VIRTIO_F_TRNSPT_VQ is offered. \subsection{Feature bits}\label{sec:Device Types / Block Device / Feature bits}introduction.tex +0
-0introduction.tex \section{Normative References}\label{sec:Normative References} · line 86
Quoted 8 lines
introduction.tex +3
-0@@ -79,6 +79,9 @@ \section{Normative References}\label{sec:Normative References} \phantomsection\label{intro:SCMI}\textbf{[SCMI]} & Arm System Control and Management Interface, DEN0056, \newline\url{https://developer.arm.com/docs/den0056/c}, version C and any future revisions\\ + \phantomsection\laçbel{intro:SIOV}\textbf{[SIOV]} & + Scalable I/O Virtualization, + \newline\url{https://www.opencompute.org/documents/ocp-scalable-io-virtualization-technical-specification-revision-1-v1-2-pdf}\\ \end{longtable} vThis publicly archived list offers a means to provide input to the
OASIS Virtual I/O Device (VIRTIO) TC.
In order to verify user consent to the Feedback License terms and
to minimize spam in the list archive, subscription is required
before posting.
Subscribe:[email protected]
Unsubscribe:[email protected]
List help:[email protected]
List archive:https://lists.oasis-open.org/archives/virtio-comment/
Feedback License:https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines:https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee:https://www.oasis-open.org/committees/virtio/
Join OASIS:https://www.oasis-open.org/join/
Quoted 3 lines · Lingshan
Lingshan, please configure your email client properly. It's very hard to differentiate your reply from mine.
Quoted 124 lines · Jason Wang
On Mon, Jul 4, 2022 at 3:44 PM Zhu Lingshan <[email protected]> wrote:
This patch introduces a new device type: virtio virtual device. An virtio virtual device example is virtio based Scalable I/O Virtualization devices.
This commit also introduces a new transport other than PCI and MMIO - the transport virtqueue.
This transport is useful for implementing virtual devices with a limited transport specific resources or presenting the virtual device in a transport independent way.
This means, all the basic device facilities are provided solely via the the transport virtqueue. Additionally, the transport virtqueue is also in
Two "the".
will fix
charge of the creating and destroying of the virtual device.
With the help of the transport virtqueue, the presenting of the virtual device is done via the co-operation between the management device and its driver.
Having a dedicated virtqueue is probably fine but there is some overlap at the description of management/managed device with the admin virtqueue proposal from Max. Need to consider a way to unify the definition at least.
OK, I can re-phrase this part
Signed-off-by Jason Wang <[email protected]>Signed-off-by Zhu Lingshan <[email protected]>--- content.tex | 621 ++++++++++++++++++++++++++++++++++++++++++++++- introduction.tex | 3 + 2 files changed, 622 insertions(+), 2 deletions(-)content.tex +26
-0@@ -2784,6 +2784,619 @@ \subsubsection{Resetting Devices}\label{sec:Virtio Transport Options / Virtio ov In order to reset a device, a driver sends the CCW_CMD_VDEV_RESET command. +\section{Virtio Over Transport Virtqueue}\label{sec:Virtio Transport Options Virtio Over Transport Virtqueue} +Sometimes it's hard to implement the device in a transport specific method. +One example is that a physical device may try to present multiple virtual devices +with limited transport specific resources. +Another example is to implement virtual devices which are transport independent. +In those cases, the transport virtqueue provided by the management device could be used to replace +the transport specific method to implement the virtual device. +Then the presenting of the virtual device is done through the cooperation between +the transport virtqueue and the management device's driver. + +Feature bit VIRTIO_F_TRNSPT_VQ indicates that the device offers a transport virtqueue. + +\subsection{Basic Concepts}\label{sec:Virtio Transport Options / Virtio over Transport Virtqueue / Basic Concepts} + +All transport virtqueue commands are of the following form: + +\begin{lstlisting} +struct virtio_transportq_ctrl { + u64 device_id; + u16 class; + u16 command; + u32 length_out; + u8 command-out-data[]; + u32 length_in; + u32 ack; + u8 command-in-data[];Any reason we need length_in/length_out (we have length in descriptors)? FYI, we don't have them for ctrl virtqueue of networking devices.
OK, I can remove
length_inandlength_out, just pick the length from the descriptors.content.tex +17
-0+}; + +/* ack values */ +#define VIRTIO_TRANSPTQ_OK 0 +#define VIRTIO_TRANSPTQ_EIO 1 +#define VIRTIO_TRANSPTQ_EAGAIN 11 +#define VIRTIO_TRANSPTQ_EBUSY 16 +#define VIRTIO_TRANSPTQ_EINVAL 22 +#define VIRTIO_TRANSPTQ_ERR 255 +\end{lstlisting} + +The \field{device_id}, \field{class}, \field{command}, \field{length_out} and +\field{command-out-data} are set by the management driver, +and the device sets the \field{ack}, \field{length_in} and \field{command-in-data}. + +\field{device_id} is an UUID to address a virtio virtual device, +The \field{device_id} value 0 is used for identify the management device itselfidentifying?
yes, will fix
+
+\subsubsection{The managment devices}\label{sec:Virtio Transport Options / Virtio over Transport Virtqueue / Basic Concepts / The management devices}s/managment/management/
will fix
content.tex +2
-0+A device that offers a transport virtqueue (via feature VIRTIO_F_TRNSPT_VQ) +is the management device of the virtual devices which are the managed devices.Not a native speaker but I couldn't parse this sentence.
OK, I can re-phrase this one.
content.tex +4
-0+ +For example, a PCIe PF with a transport virtqueue is a management device. + +\subsubsection{The virtual devices}\label{sec:Virtio Transport Options / Virtio over Transport Virtqueue / Basic Concepts / The virtual devices}
Quoted 7 lines
I think to be consistent, per Michael's suggestion, we should use "managed" here which is better than "virtual".
do you mean replace all "virtual device" with "managed device" in this patch?
We can emphasize a virtual device is a type of managed device here, however "virtual device"
implies that the device is a logical device may not has a dedicated physical transport,
like SIOV devices (https://www.opencompute.org/documents/ocp-scalable-io-virtualization-technical-specification-revision-1-v1-2-pdf).
That should be fine.
Quoted 18 lines
content.tex +6
-0+A device that is created through a transport virtqueue which is offered by a management +device is a virtual device. Virtual devices are logical devices, managed by the management devices, +they usually don't have a dedicated physical transport. + +For example, a Scalable I/O Virtualization \hyperref[intro:SIOV]{[SIOV]} device which is created and managed through a transport virtqueue of +a management device is a virtual device.Maybe we can remove the "is a virtual device" here, since we use "For example" in the subsection of "The virtual device"
sure
content.tex +4
-0+ +\devicenormative{\subsubsection}{Basic Concepts}{Virtio Transport Options / Virtio Over Transport Virtqueue / Basic Concepts} +The management device MUST offer feature bit VIRTIO_F_TRNSPT_VQ and a transport virtqueue as a device specific virtqueue.\\ +The virtual devices MUST not offer VIRTIO_F_TRNSPT_VQ feature.\\
Quoted 6 lines
Is this a must? What's the value of prohibiting the nesting of the management device?
Of course it is not a must, I mean it is possible to implement
VIRTIO_F_TRANSPT_VQfor a virtual device. But I am not sure whether it is meaningful to have nested virtual devices, creating virtual device on another virtual device, both layers are logical devices.
Not that it's not necessarily a virtual device, it could be another type.
Quoted 2 lines
And this introduce extra complexities for hardware implementation and resource management, like usually don't create VFs from another VF.
So the implementation can choose to offer VIRTIO_F_TRNSPT_VQ or not for a managed device. That means
That managed device is managed by its management device. But that managed device can choose to manage sub-managed devices. This almost come for free since the definition in the proposal seems self-contained for such implementation.
Quoted 43 lines
content.tex +41
-0+The management device MUST fail all commands against the transport virtqueue if VIRTIO_F_TRNSPT_VQ is not negotiated. + +\drivernormative{\subsubsection}{Basic Concepts}{Virtio Transport Options / Virtio Over Transport Virtqueue / Basic Concepts} +The management driver MUST not send any commands against the transport virtqueue if VIRTIO_F_TRNSPT_VQ is not negotiated. + +\subsection{Virtual Devices Discovery}\label{sec:Virtio Transport Options /Virtio Over Transport Virtqueue / Virtual Devices Discovery} + +The management device is discovered through its own transport and device specific method. +Virtual devices are created and discovered via the transport virtqueue. + +\subsection{Transport Virtqueue Features}\label{sec:Virtio Transport Options /Virtio Over Transport Virtqueue / Transport Virtqueue Features} + +Feature bit VIRTIO_F_TRNSPT_VQ_VDEV indicates that the virtual devices are created, configured +and destroyed through the transport virtqueue. +This means the transport virtqueue is the transport for the virtual devices. + +Feature bit VIRTIO_F_TRNSPT_VQ_VDEV depends on VIRTIO_F_TRNSPT_VQ + +\devicenormative{\subsubsection}{Transport Virtqueue Features}{Virtio Transport Options / Virtio Over Transport Virtqueue / Transport Virtqueue Features} +The management device MUST not accept VIRTIO_F_TRNSPT_VQ_VDEV if VIRTIO_F_TRNSPT_VQ is not negotiated. + +\subsection{Device Management}\label{sec:Virtio Transport Options / Virtio Over Transport Virtqueue / Device Management} + +When VIRTIO_F_TRNSPT_VQ_VDEV is negotiated, +virtual devices must be created and destroyed through the transport virtqueue. + +\begin{lstlisting} +struct virtio_transportq_ctrl_vdev_attribute { + u32 virtio_device_id; + u32 asid; + u8 config[]; +}; + +#define virtio_transportq_ctrl_vdev 1 (class) + #define virtio_transportq_ctrl_vdev_CREATE 0 (command) + #define virtio_transportq_ctrl_vdev_DESTROY 1 (command) +\end{lstlisting} + +The virtio_transportq_ctrl_vdev_CREATE command is used to create a virtual device. +The command-out-data for VIRTIO_TRANSPTQ_CTRL_CREATE is the virtio device id (\field{virtio_device_id}), +the address space id (\field{asid})
Quoted 4 lines
We need to be verbose here. Is this a transport specific identifier or a device specific identifier?
This asid field is an id for the device, it should be the PASID. I will add more explanations here.
I think we'd better try hard to avoid mandating transport specific ASID like PASID here. Instead, we can use PASID as an example.
Quoted 8 lines
I also wonder if this is the best place for this. To me, it looks more suitable to have a dedicated command to configure it before
DRIVER_OKlike virtqueue address. Then we can configure it per virtqueue per device with more flexibility.I understand the point, this can gives the device more flexibility for sure. It is the guest driver initialize the device.
Note that we can't allow guests to set ASID directly, the hypervisor must trap any access to ASID assignment from guests and convert it to host ASID.
Quoted 5 lines
Device status and vq configs are usually set by the guest driver,
and the guest driver is not aware of the asid, it is the host side setting the asid. So shall we set the asid uponDRIVER_OK?
And how to decide whether the device should use a device-global asid or a per-vq asid, or even hybrid? I think this is transparent to the guest
See above.
Quoted 10 lines
content.tex +4
-0+and device specific configuration (\field{config}) for creating the virtual device. +When succeed, the device returns an u64 as a unique identifier of the created virtual device in command-in-data. + +The field \field{config} is device type specific. E.g., for virtio-net, it should follow the struct virtio_net_configI think we need a dedicated section to describe the transport virtqueue support for virtio-net.
More hints for what should be discussed in the section?
Need a dedicated patch to support virtio-net. And describe the format accurately.
The virtio_net_config is not sufficient, we need at least features.
Quoted 3 lines
I see all the descriptions for virtio device types are in chapter 5 Device Types, other transports don't have their own sections for the device types, and chapter 5 discussed that in general.
I think we need
1) add transport vq in virqtueues subsection (as this patch did)
2) another subsection to describe the config for creating managed(virtual) devices
Quoted 34 lines
Here we take virtio-net device as an example to explain the filed config contents.
content.tex +11
-0+in section \ref{sec:Device Types / Network Device / Device configuration layout} + +The virtio_transportq_ctrl_vdev_DESTROY command is used to destroy a +virtual device which is identified by its 64bit identifier +\field{virtual_device_id}. There's no command-in-data for +VIRTIO_TRANSPTQ_CTRL_DESTROY command. + +\devicenormative{\subsubsection}{Device Management}{Virtio Transport Options / Virtio Over Transport Virtqueue / Device Management} + +The management device MUST fail the virtio_transportq_ctrl_vdev_CREATE if +\field{device_id} is not 0.Let's use
VIRTIO_TRANSPTQ_OKinstead of the magic number here.this zero is the filed
device_id= 0, means identifying the management device. And I found I have mixed up the upper and lower cases in the commands, they should all be upper cases, I will fix this.content.tex +3
-0+ +The management device MUST fail the virtio_transportq_ctrl_vdev_DESTROY if +\field{device_id} is 0.Same here.
content.tex +3
-0+ +All virtual devices MUST be created via the transport virtqueue if the management +device offers VIRTIO_F_TRNSPT_VQ_VDEV.Does this imply no pre-created virtual devices? If this is true, we don't need a dedicated feature bit (
VIRTIO_F_TRNSPT_VQ_VDEV).
Quoted 7 lines
There can be pre-created virtual devices, in the following sections,
struct mgmt_dev_avail_res{}, if
min_vdev_vqs==max_vdev_vqs, means the management device only supports creating virtual devices with fixed capabilities, this kind of virtual devices can be pre-created. Like the slab allocator, vendors can pre-create the device. Then when the management device receiving theVDEV_CREATEcommand, it can just pick up a free virtual device, and return the virtual device.
So the question still remains, how could we discover those pre-created devices? (having something like initial_vfs?)
Quoted 6 lines
The bit only makes sense for the case when some managed(virtual) devices are pre-created like
initial_vfsin SR-IOV. If we allow pre-creation, we need commands to discover them.
Quoted 6 lines
I think we need the feature bit
VIRTIO_F_TRNSPT_VQ_VDEV, this feature bit indicates that the transport virtqueue is the transport layer for the virtual devices, like PCI for a virtio PCI hardware. So, not only create, we use the transport virtqueue to config the device as well.And if the virtual devices are pre-created, we can just send the create command, and the management device can find a available device for us, this is also discovery.
This seems to conflict with the semantics of pre-created devices.
Quoted 106 lines
content.tex +14
-0+ +\drivernormative{\subsubsection}{Driver Management}{Virtio Transport Options / Virtio Over Transport Virtqueue / Device Management} + +The management driver MUST use 0 as \field{device_id} for +virtio_transportq_ctrl_vdev_CREATE command. + +\subsection{Available resource of the management device}\label{sec:Virtio Transport Options / Virtio Over Transport Virtqueue / Available resource of the management devic} +When VIRTIO_F_TRNSPT_VQ_VDEV is negotiated, +statistical information of available resource of the management device could be fetched +by the following command: + +\begin{lstlisting} +#define VIRTIO_TRANSPTQ_CTRL_AVAIL_RES 2 + #define VIRTIO_TRANSPTQ_CTRL_AVAIL_RES_GET 0Nit, I think this command should come first, mgmt need to query this before creating devices.
sure, I can re-order the commands.
+\end{lstlisting}
content.tex +2
-0+ +VIRTIO_TRANSPTQ_CTRL_AVAIL_RES_GET command is used to get the statstical information of availables/statstical/statistical/
will fix
content.tex +6
-0+resource of the management device. There is no command-out-data for VIRTIO_TRANSPTQ_CTRL_AVAIL_RES_GET +command. The management device fills command-in-data in the following format: + +\begin{lstlisting} +struct mgmt_dev_avail_res{ + /* The number of total remaining virtqueus of the management device */Should be virtqueues.
Actually the description here is confusing, something like this might be better
"The number of total remaining virtqueues for the managed(virtual) devices */
will fix this.
content.tex +2
-0+ u16 num_avail_vqs; + /* The minimal number of virtqueues of the virtual devices */"of a virtual device"?
will fix
content.tex +54
-0+ u16 min_vdev_vqs; + /* The maximum number of virtqueues of the virtual devices */ + u16 max_vdev_vqs; + /* The number of virtual devices that can be created with min_vdev_vqs virtqueues. + * This number may not equal to num_avail_vqs divided by min_vdev_vqs due to + * the limitations of other resource + */ + u16 num_avail_vdev; +}; +\end{lstlisting} + +If \field{min_vdev_vqs} equals to \field{min_vdev_vqs}, that means the management device +can only create virtual devices with a fixed number of virtqueues + +\devicenormative{\subsubsection}{Available resource of the management device}{Virtio Transport Options / +Virtio Over Transport Virtqueue / Available resource of the management device} +The management device MUST fail the VIRTIO_TRANSPTQ_CTRL_AVAIL_RES_GET command if \field{device_id} is not 0 + +\drivernormative{\subsubsection}{Available resource of the management device}{Virtio Transport Options / +Virtio Over Transport Virtqueue / Available resource of the management device} +The management driver MUST use 0 as \field{device_id} for +VIRTIO_TRANSPTQ_CTRL_AVAIL_RES_GET command. + +\subsection{Features Negotiation}\label{sec:Virtio Transport Options / Virtio +Over Transport Virtqueue / Features Negotiation} + +When VIRTIO_F_TRNSPT_VQ_VDEV is negotiated, +the feature negotiation of virtual devices is done by the +following commands: + +\begin{lstlisting} +#define VIRTIO_TRANSPTQ_CTRL_FEAT 3 + #define VIRTIO_TRANSPTQ_CTRL_FEAT_DEVICE_GET 0 + #define VIRTIO_TRANSPTQ_CTRL_FEAT_DRIVER_SET 1 + #define VIRTIO_TRANSPTQ_CTRL_FEAT_DRIVER_GET 2 +\end{lstlisting} + +The VIRTIO_TRANSPTQ_CTRL_FEAT_DEVICE_GET is to get the features offered +by a virtual device. + +The VIRTIO_TRANSPTQ_CTRL_FEAT_DRIVER_SET is for driver to accept feature +bits offered by the virtual device. + +The VIRTIO_TRANSPTQ_CTRL_FEAT_DRIVER_GET is to get the features accepted +by both the virtual driver and the device. + +The features is 64 bits mask of the virtio features bit. For +VIRTIO_TRANSPTQ_CTRL_DRIVER_SET, the feature is passed to the device +through command-out-data. For VIRTIO_TRANSPTQ_CTRL_FEAT_DEVICE_GET and +VIRTIO_TRANSPTQ_CTRL_DRIVER_GET the feature is returned for the device +through command-in-data. + +\devicenormative{\subsubsection}{Features Negotiation}{Virtio Transport Options / +Virtio Over Transport Virtqueue / Features Negotiation}
Quoted 5 lines
Can one of the above three commands fail? If yes, we need to document the behaviour.
can the ack values cover this? Like if failed to read the device feature bits due to IO errors, it can set this EIO below. Or do I miss something?
So you define several different error codes, that means you need to define which error codes is used in which kind of condition.
Or maybe we can simply use a single type of error code.
Quoted 236 lines
content.tex +61
-0+/* ack values */ +#define VIRTIO_TRANSPTQ_OK 0 +#define VIRTIO_TRANSPTQ_EIO 1 +#define VIRTIO_TRANSPTQ_EAGAIN 11 +#define VIRTIO_TRANSPTQ_EBUSY 16 +#define VIRTIO_TRANSPTQ_EINVAL 22 +#define VIRTIO_TRANSPTQ_ERR 255 +\end{lstlisting} + +The management device MUST fail VIRTIO_TRANSPTQ_F_CTRL_FEAT class for the +command that use 0 as its \field{device_id}. + +\drivernormative{\subsubsection}{Features Negotiation}{Virtio Transport Options / +Virtio Over Transport Virtqueue / Features Negotiation} + +The management driver MAY mediate between the feature negotiation +request of the virtual devices and the transport virtqueue. E.g when +offering features to the virtual device, the management driver MAY +exclude some features in order to limit the behaviour of the virtual +device. + +\subsection{Device Status}\label{sec:Virtio Transport Options / Virtio Over +Transport Virtqueue / Device Status} + +When VIRTIO_F_TRNSPT_VQ_VDEV is negotiated, +the status of virtual device is accessed by the following +commands: + +\begin{lstlisting} +#define VIRTIO_TRANSPTQ_CTRL_STATUS 4 + #define VIRTIO_TRANSPTQ_CTRL_STATUS_SET 0 + #define VIRTIO_TRANSPTQ_CTRL_STATUS_GET 1 +\end{lstlisting} + +The VIRTIO_TRANSPTQ_CTRL_STATUS_SET is used to set the device status of +a virtual device here. The command-out-data is the one byte status +to set to the device. There's no command-in-data for this command. + +The VIRTIO_TRANSPTQ_CTRL_STATUS_GET is used to get the device status of +a virtual device. The command-in-data is the one byte status +returned from the device. There's no command-out-data for this +command. + +\devicenormative{\subsubsection}{Device Status}{Virtio Transport Options / Virtio +Over Transport Virtqueue / Device Status} + +The management device MUST reset the virtual device when 0 +is written via VIRTIO_TRANSPTQ_CTRL_STATUS_SET, the success of this +command demonstrate the success of the reset. + +The management device MUST present 0 through +VIRTIO_TRANSPTQ_CTRL_STATUS_GET once the reset is done. + +The management device MUST fail the device status access if +\field{device_id} is 0. + +\drivernormative{\subsubsection}{Device Status}{Virtio Transport Options / Virtio +Over Transport Virtqueue / Device Status} + +After writing 0 via VIRTIO_TRANSPTQ_CTRL_STATUS_SET, the driver MUST wait +for the success of the command before re-initializing the device.So we need to clarify which is the condition for the succeed of a reset:
1) the succeed of
STATUS_SET(this is how ccw did)
2) 0 is read fromSTATUS_GET(this is how pci did)sure, I think we can use the pci way, present 0 after reset.
content.tex +87
-0+ +\subsection{Device Generation}\label{sec:Virtio Transport Options / Virtio Over Transport Virtqueue / Device Generation} + +When VIRTIO_F_TRNSPT_VQ_VDEV is negotiated, +the device generation count is read from the following commands: + +\begin{lstlisting} +#define VIRTIO_TRANSPTQ_CTRL_GENERATION 5 + #define VIRTIO_TRANSPTQ_CTRL_GENERATION_GET 0 +\end{lstlisting} + +The VIRTIO_TRANSPTQ_CTRL_GENERATION_GET is used to get the device configuration generation field +of the virtual device. The command-in-data is the u32 device +generation returned from the device. There's no command-out-data for +this command. + +\devicenormative{\subsubsection}{Device Generation}{Virtio Transport Options / Virtio Over Transport Virtqueue / Device Generation} + +The management device MUST present a changed generation count after the driver +has read a device-specific configuration value which has changed since +any part of the device-specific configuration was last read. + +The management device MUST fail the device generation access if \field{device_id} is 0. + +\subsection{Device Specific Configuration}\label{sec:Virtio Transport Options / +Virtio Over Transport Virtqueue / Device Specific Configuration} + +When VIRTIO_F_TRNSPT_VQ_VDEV is negotiated, +the device config space contents of a virtual device are accessed through +VIRTIO_TRANSPTQ_CTRL_CONFIG_GET and VIRTIO_TRANSPTQ_CTRL_CONFIG_SET. + +\begin{lstlisting} +#define VIRTIO_TRANSPTQ_CTRL_CONFIG 6 + #define VIRTIO_TRANSPTQ_CTRL_CONFIG_GET 0 + #define VIRTIO_TRANSPTQ_CTRL_CONFIG_SET 1 + +struct virtio_transportq_ctrl_vdev_config_get { + u32 offset; + u32 size; +}; + +struct virtio_transportq_ctrl_vdev_config_set { + u32 offset; + u32 size; + u8 data[]; +}; +\end{lstlisting} + +The VIRTIO_TRANSPTQ_CTRL_CONFIG_GET is used to read data from the +device configuration space. As described in struct +virtio_transportq_ctrl_vdev_config_get, The command-out-data is the offset +from the start of the config space and the size of the data. The +command-in-data is an array of u8 data that read from the config +space. + +The VIRTIO_TRANSPTQ_CTRL_CONFIG_SET is used to write data to the device +configuration space. As described in struct +virtio_transportq_ctrl_vdev_config_set, the command-out-data contains the +offset from the start of the config space, the size of the data and +the data that will be written. There's no command-in-data for this +command. + +\devicenormative{\subsubsection}{Device Specific Configuration}{Virtio Transport +Options / Virtio Over Transport Virtqueue / Device Specific Configuration} + +The management device MUST fail the device configuration space access +if the driver wants to access the range which is outside the config space. + +The management device MUST fail the device configuration space access +if \field{device_id} is 0. + +\subsection{MSI Configuration}\label{sec:Virtio Transport Options / Virtio Over +dmin Virtqueue / MSI Configuration} + +When VIRTIO_F_TRNSPT_VQ_VDEV is negotiated, +the MSI entry for a specific virtqueue is set through following command: + +\begin{lstlisting} +#define VIRTIO_TRANSPTQ_CTRL_MSI 7 + #define VIRTIO_TRANSPTQ_CTRL_MSI_VQ_SET 0 + #define VIRTIO_TRANSPTQ_CTRL_MSI_VQ_ENABLE 1 + #define VIRTIO_TRANSPTQ_CTRL_MSI_VQ_MASK 2 + #define VIRTIO_TRANSPTQ_CTRL_MSI_CONFIG_SET 3 + #define VIRTIO_TRANSPTQ_CTRL_MSI_CONFIG_ENABLE 4 + #define VIRTIO_TRANSPTQ_CTRL_MSI_CONFIG_MASK 5 + +struct virtio_transportq_ctrl_vdev_msi_vq_set {To be consistent with msi, let's use
_vdev_mst_vq_config?OK, it will be
virtio_transportq_ctrl_vdev_msi_vq_configin the next versioncontent.tex +29
-0+ u16 queue_index; + u64 addr; + u32 data; +}; + +struct virtio_transportq_ctrl_vdev_msi_vq_enable { + u16 queue_index; + u8 enable; +}; + +struct virtio_transportq_ctrl_vdev_msi_vq_mask { + u16 queue_index; + u8 mask; +}; + +struct virtio_transportq_ctrl_vdev_msi_config { + u64 addr; + u32 data; +}; +\end{lstlisting} + +The VIRTIO_TRANSPTQ_CTRL_MSI_VQ_SET command is used to set the MSI entry for a +specific virtqueue. The command-out-data is the virtqueue index and +the MSI address and data (as described in struct +virtio_transportq_ctrl_vdev_msix_vq_set). + +The VIRTIO_TRANSPTQ_CTRL_MSI_VQ_ENABLE command is used to enable or disable +the MSI interrupt for a specific virtqueue. The command-out-data is the +virtqueue index and whether to enable the MSI: 0 means to enable and 1It might be better to use macros instead of magic numbers.
sure!
content.tex +38
-0+means to disable (as described in struct +virtio_transportq_ctrl_vdev_msi_vq_enable). + +The VIRTIO_TRANSPTQ_CTRL_MSI_VQ_MASK command is used to mask or unmask the +MSI interrupt for a specific virtqueue. The command-out-data is the +virtqueue index and the mask status: 0 means unmak and 1 means mask +(as described in struct virtio_transportq_ctrl_vdev_msi_vq_mask). + +The VIRTIO_TRANSPTQ_CTRL_MSI_CONFIG_SET command is used to set the MSI entry +for the config interrupt. The command-out-data is the MSI address and +data (as described in struct virtio_transportq_ctrl_vdev_msi_config). + +The VIRTIO_TRANSPTQ_CTRL_MSI_CONFIG_ENABLE command is used to enable and disable +MSI for config space. The command-out-data is an u8: 0 means to +disable and 1 means to enable. + +The VIRTIO_TRANSPTQ_CTRL_MSI_CONFIG_MASK command is used to mask and unmask MSI +interrupt for config space. The command-out-data is an u8: 0 means to +mask and 1 means to unmask. + +There's no command-in-data for all the above MSI commands. + +\devicenormative{\subsubsection}{MSI Configuration}{Virtio Transport Options / Virtio +ver Transport Virtqueue / MSI Configuration} + +The virtual device MUST record the pending MSI interrupts and +re-generate the pending MSI interrupts when unmasking. + +The virtual device MUST disable the MSI for both virtqueue and config space +upon reset. + +\drivernormative{\subsubsection}{MSI Configuration}{Virtio Transport Options / Virtio +Over Transport Virtqueue / MSI Configuration} + +The driver MUST allocate transport or platform specific MSI entries +for both virtqueue and config space if it wants to use interrupt. + +The driver MAY choose disable the MSI if polling mode is used.
Quoted 6 lines
"choose to disable" ?
when use polling mode, I think it is not a must to disable the interrupts. So the driver may disable the interrupts for better performance or not.
Did I miss something?
I meant gmail reminds me you need a "to" between "choose" and "disable".
Quoted 140 lines
content.tex +137
-0+ +\subsection{Virtqueue Address}\label{sec:Virtio Transport Options / Virtio Over +Transport Virtqueue / Virtqueue Address} + +When VIRTIO_F_TRNSPT_VQ_VDEV is negotiated, +the addresses of a specific virtqueue are accessed through the following command: + +\begin{lstlisting} +#define VIRTIO_TRANSPTQ_CTRL_VQ_ADDR 8 + #define VIRTIO_TRANSPTQ_CTRL_VQ_ADDR_SET 0 + +struct virtio_transportq_ctrl_vdev_vq_addr { + u16 queue_index; + u64 device_area; + u64 descriptor_area; + u64 driver_area; +}; +\end{lstlisting} + +The command-out-data is the queue index, the addresses of device area, +descriptor area and driver area (as described in struct +virtio_transportq_ctrl_vdev_vq_addr); There's no command-in-data. + +\devicenormative{\subsubsection}{Virtqueue Address}{Virtio Transport Options / Virtio +Over Transport Virtqueue / Virtqueeu Address} + +The management device MUST fail the commands of class +VIRTIO_TRANSPTQ_CTRL_VQ_ADDR if \field{device_id} is 0. + +\subsection{Virtqueue Status}\label{sec:Virtio Transport Options / Virtio Over +Transport Virtqueue / Virtqueue Status} + +When VIRTIO_F_TRNSPT_VQ_VDEV is negotiated, +virtqueue status is set and get through the following command: + +\begin{lstlisting} +#define VIRTIO_TRANSPTQ_CTRL_VQ_ENABLE 9 + #define VIRTIO_TRANSPTQ_CTRL_VQ_ENABLE_SET 0 + #define VIRTIO_TRANSPTQ_CTRL_VQ_ENABLE_GET 1 + +struct virtio_transportq_ctrl_vq_status_set { + u16 queue_index; + u8 status; +}; + +\end{lstlisting} + +The VIRTIO_TRANSPTQ_CTRL_VQ_ENABLE_SET command is used to set the status of a +specific virtqueue. The command-out-data is the queue index and the +status that is set to the virtqueue (0 disabled, 1 enabled), +as described in struct virtio_transportq_ctrl_vq_status_set. +There's no command-in-data. + +The VIRTIO_TRANSPTQ_CTRL_VQ_ENABLE_GET is used to get the status of a +specific virtqueue. The command-out-data is an u16 of the queue +index. The command-in-data is the virtqueue status (0 disabled, 1 +enabled). + +\devicenormative{\subsubsection}{Virtqueue Status}{Virtio Transport Options / Virtio +Over Transport Virtqueue / Virtqueue Status} + +When disabled, the virtual device MUST stop processing requests from +this virtqueue. + +The management device MUST present a 0 via +VIRTIO_TRANSPTQ_CTRL_VQ_ENABLE_GET upon a reset of the virtual device. + +The management device MUST fail the virtqueue status access if +\field{device_id} is 0. + +\drivernormative{\subsubsection}{Virtqueue Status}{Virtio Transport Options / Virtio +Over Transport Virtqueue / Virtqueue Status} + +The driver MUST configure other virtqueue fields before enabling +the virtqueue with VIRTIO_TRANSPTQ_CTRL_VQ_ENABLE_SET. + +\subsection{Virtqueue Size}\label{sec:Virtio Transport Options / Virtio Over +Transport Virtqueue / Virtqueue Size} + +When VIRTIO_F_TRNSPT_VQ_VDEV is negotiated, +virtqueue size is accessed through the following command: + +\begin{lstlisting} +#define VIRTIO_TRANSPTQ_CTRL_VQ_SIZE 10 + #define VIRTIO_TRANSPTQ_CTRL_VQ_SIZE_SET 0 + #define VIRTIO_TRANSPTQ_CTRL_VQ_SIZE_GET 1 + +struct virtio_transportq_ctrl_vdev_vq_size_set { + u16 queue_index; + u16 size; +}; +\end{lstlisting} + +The VIRTIO_TRANSPTQ_CTRL_VQ_SIZE_SET command is used to set the virtqueue +size. The command-out-data is the queue index and the size of the +virtqueue (as described in struct +virtio_transportq_ctrl_vdev_vq_size_set). There's no command-in-data. + +The VIRTIO_TRANSPTQ_CTRL_VQ_SIZE_GET command is used to get the virtqueue +size. On reset, the maximum queue size supported by the device is +returned. The command-out-data is an u16 of the virtqueue index. The +command-in-data is an u16 of queue size for the virtqueue. + +\devicenormative{\subsubsection}{Virtqueue Status}{Virtio Transport Options / Virtio +Over Transport Virtqueue / Virtqueue Size} + +The management device MUST fail the virtqueue size access if +\field{device_id} is 0. + +\subsection{Virtqueue Notification}\label{sec:Virtio Transport Options / Virtio +Over Transport Virtqueue / Virtqueue Notification} + +When VIRTIO_F_TRNSPT_VQ_VDEV is negotiated, +the virtqueue notification is done through the following commands: + +\begin{lstlisting} +#define VIRTIO_TRANSPTQ_CTRL_VQ_NOTIFY 11 + #define VIRTIO_TRANSPTQ_CTRL_VQ_NOTIFY_GET 1 + +struct virtio_transportq_ctrl_vdev_vq_notification_area { + le64 addr + le64 size; +}; +\end{lstlisting} + +The VIRTIO_TRANSPTQ_CTRL_VQ_NOTIFY_GET is used to get the transport +specific address area that can be used to notify a virtqueue. The +command-out-data is an u16 of the virtqueue index. The command-in-data +contains the address and the size of the notification area (as +described in struct virtio_transportq_ctrl_vdev_vq_notification_area). + +\devicenormative{\subsubsection}{Virtqueue Notification}{Virtio Transport Options / +Virtio Over Transport Virtqueue / Virtqueue Notification} + +The management device MUST fail the VIRTIO_TRANSPTQ_CTRL_VQ_NOTIFY_GET command +if there's no transport specific notification address for a virtqueue of +its virtual device.
Quoted 5 lines
Then how can we kick a device in this case?
I think I should remove this sub-section. Because there should be a notification area, or it is a defective device, and if failed to get the notification area, the command should set a proper ack value reflecting the error
That's fine.
Quoted 52 lines
content.tex +32
-0+ +The management device MUST fail the virtqueue notification access if +\field{device_id} is 0. + +The management device MUST forbid the notification area of a specific +virtual device to be accessed from another virtual device. + +\drivernormative{\subsubsection}{Virtqueue Notification}{Virtio Transport Options / +Virtio Over Transport Virtqueue / Virtqueue Notification} + +The driver MAY choose to notify the virtqueue by writing the queue +index at address \field{addr} which is fetched from the +VIRTIO_TRANSPTQ_CTRL_VQ_NOTIFY_GET command. + +\subsection{Virtqueue Index}\label{sec:Virtio Transport Options / Virtio +Over Transport Virtqueue / Virtqueue Index} + +When VIRTIO_F_TRNSPT_VQ_VDEV is negotiated, +the virtqueue available index is accessed through the following commands: + +\begin{lstlisting} +#define VIRTIO_TRANSPTQ_CTRL_VQ_AVAIL_INDEX 12 + #define VIRTIO_TRANSPTQ_CTRL_VQ_AVAIL_INDEX_SET 0 + #define VIRTIO_TRANSPTQ_CTRL_VQ_ AVAIL_INDEX_GET 1 + +struct virtio_transportq_ctrl_vdev_vq_avail_idx_set { + u16 queue_index; + u16 avail_idx; +}; +\end{lstlisting} + +The VIRTIO_TRANSPTQ_CTRL_VQ_AVAIL_INDEX_SET commmand is used to set the available index of a specific virtqueue.s/commmand/command/g
We need the support of packed virtqueue as well here.
sure, will work on this.
content.tex +11
-0+The command-out-data is the virtqueue index and the available index (as described in struct virtio_transportq_ctrl_vdev_vq_avail_idx). +There is no command-in-data. + +The VIRTIO_TRANSPTQ_CTRL_VQ_ AVAIL_INDEX_GET command is used to get the available index of a specific virtqueue. +The command-out-data is the queue index, the command-in-data is the available index of the queue. + +\devicenormative{\subsubsection}{Virtqueue Index}{Virtio Transport Options / +Virtio Over Transport Virtqueue / Virtqueue Index} + +The management device MUST fail the virtqueue index access if \field{device_id} is 0 +The management device MUST fail the virtqueue index access if \field{queue_id} is invalid.
Quoted 7 lines
We don't have similar normatives for other commands, anything make this command different?
If it is PCI, if we set
queue_selectto an invalid value, I think it is a undefined behavior, this could be ignored by the hardware, the device has no ways to report such errors.Here, for the transport virtqueue, it is a must to return an ACK value, and it can be meaningful, like
VIRTIO_TRANSPTQ_EINVAL
Yes, but what I meant is that it applies to other types of commands.
Thanks
Quoted 121 lines
content.tex +17
-0+ +\subsection{Presenting Virtual Devices}\label{sec:Virtio Transport Options / +Virtio Over Transport Virtqueue / Presenting Virtual Device} + +If VIRTIO_TRANSPTQ_F_VDEV is offered by a management device. The presenting of +the virtual devices requires co-operation between the management +driver and the transport virtqueue. This means, from the view of the +virtual device driver, the transport is done via the communication +with the management device driver. It's up to the software to decide +what kind of method that is needed be used for those communications. + +The management driver typically takes the following steps for creating a +virtual device: + +\begin{enumerate} +\item Determine the virtio id and device specific configuration. +\item Create the virtual devices using the virtio_transportq_ctrl_vdev_CREATE command.Need to use the upper case for the command name here.
yes
+\item Optionally, configure the MSI.
+\item Perform device specific setup.
+\item Let the virtual device to be probed by the virtual devicecontent.tex +3
-0+driver. The management driver will then use the transport virtqueue to +implement the requests of basic facility from the virtual device +driver.I think we need to be more verbose here, e.g to mention the actual command that is used.
sure, will add these
+\end{enumerate}
\chapter{Device Types}\label{sec:Device Types}
content.tex +1
-0@@ -2911,12 +3524,14 @@ \subsection{Virtqueues}\label{sec:Device Types / Network Device / Virtqueues} \item[2(N-1)] receiveqN \item[2(N-1)+1] transmitqN \item[2N] controlq +\item[2N+1] transportqLet's use a separate patch for this.
can do.
Thanks for your review!
\end{description}
N=1 if neither
VIRTIO_NET_F_MQnorVIRTIO_NET_F_RSSare negotiated, otherwise N is set by \field{max_virtqueue_pairs}.- controlq only exists if
VIRTIO_NET_F_CTRL_VQset.
+ controlq only exists ifVIRTIO_NET_F_CTRL_VQset.\\
+ transportq only exists ifVIRTIO_F_TRNSPT_VQis offered.\subsection{Feature bits}\label{sec:Device Types / Network Device / Feature bits}
content.tex +3
-1@@ -4399,10 +5014,12 @@ \subsection{Virtqueues}\label{sec:Device Types / Block Device / Virtqueues} \item[0] requestq1 \item[\ldots] \item[N-1] requestqN +\item[N] transportq \end{description} N=1 if VIRTIO_BLK_F_MQ is not negotiated, otherwise N is set by - \field{num_queues}. + \field{num_queues}.\\ + transportq only exists if VIRTIO_F_TRNSPT_VQ is offered. \subsection{Feature bits}\label{sec:Device Types / Block Device / Feature bits}introduction.tex +3
-0@@ -79,6 +79,9 @@ \section{Normative References}\label{sec:Normative References} \phantomsection\label{intro:SCMI}\textbf{[SCMI]} & Arm System Control and Management Interface, DEN0056, \newline\url{https://developer.arm.com/docs/den0056/c}, version C and any future revisions\\ + \phantomsection\laçbel{intro:SIOV}\textbf{[SIOV]} & + Scalable I/O Virtualization, + \newline\url{https://www.opencompute.org/documents/ocp-scalable-io-virtualization-technical-specification-revision-1-v1-2-pdf}\\ \end{longtable} vThis publicly archived list offers a means to provide input to the
OASIS Virtual I/O Device (VIRTIO) TC.
In order to verify user consent to the Feedback License terms and
to minimize spam in the list archive, subscription is required
before posting.
Subscribe: [email protected]
Unsubscribe: [email protected]
List help: [email protected]
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/
Quoted 5 lines · Jason Wang
Quoted 2 lines · Lingshan
Lingshan, please configure your email client properly. It's very hard to differentiate your reply from mine.
Oh, sorry for the inconvenience, maybe the reason is that I have saved a draft in thunderbird and outlook has re-formatted the draft.
Quoted 125 lines
Quoted 125 lines · Jason Wang
On Mon, Jul 4, 2022 at 3:44 PM Zhu Lingshan <[email protected]> wrote:
This patch introduces a new device type: virtio virtual device. An virtio virtual device example is virtio based Scalable I/O Virtualization devices.
This commit also introduces a new transport other than PCI and MMIO - the transport virtqueue.
This transport is useful for implementing virtual devices with a limited transport specific resources or presenting the virtual device in a transport independent way.
This means, all the basic device facilities are provided solely via the the transport virtqueue. Additionally, the transport virtqueue is also in
Two "the".
will fix
charge of the creating and destroying of the virtual device.
With the help of the transport virtqueue, the presenting of the virtual device is done via the co-operation between the management device and its driver.
Having a dedicated virtqueue is probably fine but there is some overlap at the description of management/managed device with the admin virtqueue proposal from Max. Need to consider a way to unify the definition at least.
OK, I can re-phrase this part
Signed-off-by Jason Wang <[email protected]>Signed-off-by Zhu Lingshan <[email protected]>--- content.tex | 621 ++++++++++++++++++++++++++++++++++++++++++++++- introduction.tex | 3 + 2 files changed, 622 insertions(+), 2 deletions(-)content.tex +26
-0@@ -2784,6 +2784,619 @@ \subsubsection{Resetting Devices}\label{sec:Virtio Transport Options / Virtio ov In order to reset a device, a driver sends the CCW_CMD_VDEV_RESET command. +\section{Virtio Over Transport Virtqueue}\label{sec:Virtio Transport Options Virtio Over Transport Virtqueue} +Sometimes it's hard to implement the device in a transport specific method. +One example is that a physical device may try to present multiple virtual devices +with limited transport specific resources. +Another example is to implement virtual devices which are transport independent. +In those cases, the transport virtqueue provided by the management device could be used to replace +the transport specific method to implement the virtual device. +Then the presenting of the virtual device is done through the cooperation between +the transport virtqueue and the management device's driver. + +Feature bit VIRTIO_F_TRNSPT_VQ indicates that the device offers a transport virtqueue. + +\subsection{Basic Concepts}\label{sec:Virtio Transport Options / Virtio over Transport Virtqueue / Basic Concepts} + +All transport virtqueue commands are of the following form: + +\begin{lstlisting} +struct virtio_transportq_ctrl { + u64 device_id; + u16 class; + u16 command; + u32 length_out; + u8 command-out-data[]; + u32 length_in; + u32 ack; + u8 command-in-data[];Any reason we need length_in/length_out (we have length in descriptors)? FYI, we don't have them for ctrl virtqueue of networking devices.
OK, I can remove
length_inandlength_out, just pick the length from the descriptors.content.tex +17
-0+}; + +/* ack values */ +#define VIRTIO_TRANSPTQ_OK 0 +#define VIRTIO_TRANSPTQ_EIO 1 +#define VIRTIO_TRANSPTQ_EAGAIN 11 +#define VIRTIO_TRANSPTQ_EBUSY 16 +#define VIRTIO_TRANSPTQ_EINVAL 22 +#define VIRTIO_TRANSPTQ_ERR 255 +\end{lstlisting} + +The \field{device_id}, \field{class}, \field{command}, \field{length_out} and +\field{command-out-data} are set by the management driver, +and the device sets the \field{ack}, \field{length_in} and \field{command-in-data}. + +\field{device_id} is an UUID to address a virtio virtual device, +The \field{device_id} value 0 is used for identify the management device itselfidentifying?
yes, will fix
+
+\subsubsection{The managment devices}\label{sec:Virtio Transport Options / Virtio over Transport Virtqueue / Basic Concepts / The management devices}s/managment/management/
will fix
content.tex +2
-0+A device that offers a transport virtqueue (via feature VIRTIO_F_TRNSPT_VQ) +is the management device of the virtual devices which are the managed devices.Not a native speaker but I couldn't parse this sentence.
OK, I can re-phrase this one.
content.tex +4
-0+ +For example, a PCIe PF with a transport virtqueue is a management device. + +\subsubsection{The virtual devices}\label{sec:Virtio Transport Options / Virtio over Transport Virtqueue / Basic Concepts / The virtual devices}
Quoted 8 lines
Quoted 7 lines
I think to be consistent, per Michael's suggestion, we should use "managed" here which is better than "virtual".
do you mean replace all "virtual device" with "managed device" in this patch?
We can emphasize a virtual device is a type of managed device here, however "virtual device"
implies that the device is a logical device may not has a dedicated physical transport,
like SIOV devices (https://www.opencompute.org/documents/ocp-scalable-io-virtualization-technical-specification-revision-1-v1-2-pdf).That should be fine.
OK, I will use "managed devices" instead of "virtual devices" in the next version
Quoted 28 lines
Quoted 18 lines
content.tex +6
-0+A device that is created through a transport virtqueue which is offered by a management +device is a virtual device. Virtual devices are logical devices, managed by the management devices, +they usually don't have a dedicated physical transport. + +For example, a Scalable I/O Virtualization \hyperref[intro:SIOV]{[SIOV]} device which is created and managed through a transport virtqueue of +a management device is a virtual device.Maybe we can remove the "is a virtual device" here, since we use "For example" in the subsection of "The virtual device"
sure
content.tex +4
-0+ +\devicenormative{\subsubsection}{Basic Concepts}{Virtio Transport Options / Virtio Over Transport Virtqueue / Basic Concepts} +The management device MUST offer feature bit VIRTIO_F_TRNSPT_VQ and a transport virtqueue as a device specific virtqueue.\\ +The virtual devices MUST not offer VIRTIO_F_TRNSPT_VQ feature.\\Quoted 6 lines
Is this a must? What's the value of prohibiting the nesting of the management device?
Of course it is not a must, I mean it is possible to implement
VIRTIO_F_TRANSPT_VQfor a virtual device. But I am not sure whether it is meaningful to have nested virtual devices, creating virtual device on another virtual device, both layers are logical devices.Not that it's not necessarily a virtual device, it could be another type.
Quoted 2 lines
And this introduce extra complexities for hardware implementation and resource management, like usually don't create VFs from another VF.
Quoted 7 lines
So the implementation can choose to offer
VIRTIO_F_TRNSPT_VQor not for a managed device. That meansThat managed device is managed by its management device. But that managed device can choose to manage sub-managed devices. This almost come for free since the definition in the proposal seems self-contained for such implementation.
sure, I will remove this. So the vendor can choose to enable nested managed devices or not.
Quoted 43 lines
Quoted 43 lines
content.tex +41
-0+The management device MUST fail all commands against the transport virtqueue if VIRTIO_F_TRNSPT_VQ is not negotiated. + +\drivernormative{\subsubsection}{Basic Concepts}{Virtio Transport Options / Virtio Over Transport Virtqueue / Basic Concepts} +The management driver MUST not send any commands against the transport virtqueue if VIRTIO_F_TRNSPT_VQ is not negotiated. + +\subsection{Virtual Devices Discovery}\label{sec:Virtio Transport Options /Virtio Over Transport Virtqueue / Virtual Devices Discovery} + +The management device is discovered through its own transport and device specific method. +Virtual devices are created and discovered via the transport virtqueue. + +\subsection{Transport Virtqueue Features}\label{sec:Virtio Transport Options /Virtio Over Transport Virtqueue / Transport Virtqueue Features} + +Feature bit VIRTIO_F_TRNSPT_VQ_VDEV indicates that the virtual devices are created, configured +and destroyed through the transport virtqueue. +This means the transport virtqueue is the transport for the virtual devices. + +Feature bit VIRTIO_F_TRNSPT_VQ_VDEV depends on VIRTIO_F_TRNSPT_VQ + +\devicenormative{\subsubsection}{Transport Virtqueue Features}{Virtio Transport Options / Virtio Over Transport Virtqueue / Transport Virtqueue Features} +The management device MUST not accept VIRTIO_F_TRNSPT_VQ_VDEV if VIRTIO_F_TRNSPT_VQ is not negotiated. + +\subsection{Device Management}\label{sec:Virtio Transport Options / Virtio Over Transport Virtqueue / Device Management} + +When VIRTIO_F_TRNSPT_VQ_VDEV is negotiated, +virtual devices must be created and destroyed through the transport virtqueue. + +\begin{lstlisting} +struct virtio_transportq_ctrl_vdev_attribute { + u32 virtio_device_id; + u32 asid; + u8 config[]; +}; + +#define virtio_transportq_ctrl_vdev 1 (class) + #define virtio_transportq_ctrl_vdev_CREATE 0 (command) + #define virtio_transportq_ctrl_vdev_DESTROY 1 (command) +\end{lstlisting} + +The virtio_transportq_ctrl_vdev_CREATE command is used to create a virtual device. +The command-out-data for VIRTIO_TRANSPTQ_CTRL_CREATE is the virtio device id (\field{virtio_device_id}), +the address space id (\field{asid})
Quoted 6 lines
Quoted 4 lines
We need to be verbose here. Is this a transport specific identifier or a device specific identifier?
This asid field is an id for the device, it should be the PASID. I will add more explanations here.
I think we'd better try hard to avoid mandating transport specific ASID like PASID here. Instead, we can use PASID as an example.
OK
Quoted 11 lines
Quoted 8 lines
I also wonder if this is the best place for this. To me, it looks more suitable to have a dedicated command to configure it before
DRIVER_OKlike virtqueue address. Then we can configure it per virtqueue per device with more flexibility.I understand the point, this can gives the device more flexibility for sure. It is the guest driver initialize the device.
Note that we can't allow guests to set ASID directly, the hypervisor must trap any access to ASID assignment from guests and convert it to host ASID.
I will add a new command to set vq asid in the next version.
Quoted 13 lines
Quoted 6 lines
Device status and vq configs are usually set by the guest driver,
and the guest driver is not aware of the asid, it is the host side setting the asid. So shall we set the asid uponDRIVER_OK?
And how to decide whether the device should use a device-global asid or a per-vq asid, or even hybrid? I think this is transparent to the guestSee above.
Quoted 5 lines
content.tex +4
-0+and device specific configuration (\field{config}) for creating the virtual device. +When succeed, the device returns an u64 as a unique identifier of the created virtual device in command-in-data. + +The field \field{config} is device type specific. E.g., for virtio-net, it should follow the struct virtio_net_config
Quoted 8 lines
Quoted 4 lines
I think we need a dedicated section to describe the transport virtqueue support for virtio-net.
More hints for what should be discussed in the section?
Need a dedicated patch to support virtio-net. And describe the format accurately.
The
virtio_net_configis not sufficient, we need at least features.
Yes, I will add features for the creation, add rename config[] to dev_config[]. I should also add the number of vqs for the creation, and a new command to get the number of vqs of a managed device. I missed these in this version.
Quoted 7 lines
Quoted 4 lines
I see all the descriptions for virtio device types are in chapter 5 Device Types, other transports don't have their own sections for the device types, and chapter 5 discussed that in general.
I think we need
1) add transport vq in virqtueues subsection (as this patch did)
will move these contents to a separate patch
Quoted 2 lines
2) another subsection to describe the config for creating managed(virtual) devices
I will add more detailed description in the subsection "Presenting Virtual Devices", and take virtio-net as an example.
Quoted 36 lines
Quoted 36 lines
Here we take virtio-net device as an example to explain the filed config contents.
content.tex +11
-0+in section \ref{sec:Device Types / Network Device / Device configuration layout} + +The virtio_transportq_ctrl_vdev_DESTROY command is used to destroy a +virtual device which is identified by its 64bit identifier +\field{virtual_device_id}. There's no command-in-data for +VIRTIO_TRANSPTQ_CTRL_DESTROY command. + +\devicenormative{\subsubsection}{Device Management}{Virtio Transport Options / Virtio Over Transport Virtqueue / Device Management} + +The management device MUST fail the virtio_transportq_ctrl_vdev_CREATE if +\field{device_id} is not 0.Let's use
VIRTIO_TRANSPTQ_OKinstead of the magic number here.this zero is the filed
device_id= 0, means identifying the management device. And I found I have mixed up the upper and lower cases in the commands, they should all be upper cases, I will fix this.content.tex +3
-0+ +The management device MUST fail the virtio_transportq_ctrl_vdev_DESTROY if +\field{device_id} is 0.Same here.
content.tex +3
-0+ +All virtual devices MUST be created via the transport virtqueue if the management +device offers VIRTIO_F_TRNSPT_VQ_VDEV.Does this imply no pre-created virtual devices? If this is true, we don't need a dedicated feature bit (
VIRTIO_F_TRNSPT_VQ_VDEV).There can be pre-created virtual devices, in the following sections,
Quoted 7 lines
Quoted 5 lines
struct mgmt_dev_avail_res{}, if
min_vdev_vqs==max_vdev_vqs, means the management device only supports creating virtual devices with fixed capabilities, this kind of virtual devices can be pre-created. Like the slab allocator, vendors can pre-create the device. Then when the management device receiving theVDEV_CREATEcommand, it can just pick up a free virtual device, and return the virtual device.So the question still remains, how could we discover those pre-created devices? (having something like
initial_vfs?)
I think for the management devices, pre-creation could be pre-allocated
hw resource. The management device could pre-allocate a certain number of "standard" managed devices, when receive a create command, the management device can pick a free managed device, assign an UUID to it, then return
to the management driver. So the driver may not be aware of this pre-creation, it just sees the device is created rapidly. Just like the slab system, or a threads pool.
Quoted 6 lines
Quoted 6 lines
The bit only makes sense for the case when some managed(virtual) devices are pre-created like
initial_vfsin SR-IOV. If we allow pre-creation, we need commands to discover them.
Quoted 7 lines
Quoted 6 lines
I think we need the feature bit
VIRTIO_F_TRNSPT_VQ_VDEV, this feature bit indicates that the transport virtqueue is the transport layer for the virtual devices, like PCI for a virtio PCI hardware. So, not only create, we use the transport virtqueue to config the device as well.And if the virtual devices are pre-created, we can just send the create command, and the management device can find a available device for us, this is also discovery.
This seems to conflict with the semantics of pre-created devices.
same to the above
Quoted 109 lines
Quoted 109 lines
content.tex +14
-0+ +\drivernormative{\subsubsection}{Driver Management}{Virtio Transport Options / Virtio Over Transport Virtqueue / Device Management} + +The management driver MUST use 0 as \field{device_id} for +virtio_transportq_ctrl_vdev_CREATE command. + +\subsection{Available resource of the management device}\label{sec:Virtio Transport Options / Virtio Over Transport Virtqueue / Available resource of the management devic} +When VIRTIO_F_TRNSPT_VQ_VDEV is negotiated, +statistical information of available resource of the management device could be fetched +by the following command: + +\begin{lstlisting} +#define VIRTIO_TRANSPTQ_CTRL_AVAIL_RES 2 + #define VIRTIO_TRANSPTQ_CTRL_AVAIL_RES_GET 0Nit, I think this command should come first, mgmt need to query this before creating devices.
sure, I can re-order the commands.
+\end{lstlisting}
content.tex +2
-0+ +VIRTIO_TRANSPTQ_CTRL_AVAIL_RES_GET command is used to get the statstical information of availables/statstical/statistical/
will fix
content.tex +6
-0+resource of the management device. There is no command-out-data for VIRTIO_TRANSPTQ_CTRL_AVAIL_RES_GET +command. The management device fills command-in-data in the following format: + +\begin{lstlisting} +struct mgmt_dev_avail_res{ + /* The number of total remaining virtqueus of the management device */Should be virtqueues.
Actually the description here is confusing, something like this might be better
"The number of total remaining virtqueues for the managed(virtual) devices */
will fix this.
content.tex +2
-0+ u16 num_avail_vqs; + /* The minimal number of virtqueues of the virtual devices */"of a virtual device"?
will fix
content.tex +54
-0+ u16 min_vdev_vqs; + /* The maximum number of virtqueues of the virtual devices */ + u16 max_vdev_vqs; + /* The number of virtual devices that can be created with min_vdev_vqs virtqueues. + * This number may not equal to num_avail_vqs divided by min_vdev_vqs due to + * the limitations of other resource + */ + u16 num_avail_vdev; +}; +\end{lstlisting} + +If \field{min_vdev_vqs} equals to \field{min_vdev_vqs}, that means the management device +can only create virtual devices with a fixed number of virtqueues + +\devicenormative{\subsubsection}{Available resource of the management device}{Virtio Transport Options / +Virtio Over Transport Virtqueue / Available resource of the management device} +The management device MUST fail the VIRTIO_TRANSPTQ_CTRL_AVAIL_RES_GET command if \field{device_id} is not 0 + +\drivernormative{\subsubsection}{Available resource of the management device}{Virtio Transport Options / +Virtio Over Transport Virtqueue / Available resource of the management device} +The management driver MUST use 0 as \field{device_id} for +VIRTIO_TRANSPTQ_CTRL_AVAIL_RES_GET command. + +\subsection{Features Negotiation}\label{sec:Virtio Transport Options / Virtio +Over Transport Virtqueue / Features Negotiation} + +When VIRTIO_F_TRNSPT_VQ_VDEV is negotiated, +the feature negotiation of virtual devices is done by the +following commands: + +\begin{lstlisting} +#define VIRTIO_TRANSPTQ_CTRL_FEAT 3 + #define VIRTIO_TRANSPTQ_CTRL_FEAT_DEVICE_GET 0 + #define VIRTIO_TRANSPTQ_CTRL_FEAT_DRIVER_SET 1 + #define VIRTIO_TRANSPTQ_CTRL_FEAT_DRIVER_GET 2 +\end{lstlisting} + +The VIRTIO_TRANSPTQ_CTRL_FEAT_DEVICE_GET is to get the features offered +by a virtual device. + +The VIRTIO_TRANSPTQ_CTRL_FEAT_DRIVER_SET is for driver to accept feature +bits offered by the virtual device. + +The VIRTIO_TRANSPTQ_CTRL_FEAT_DRIVER_GET is to get the features accepted +by both the virtual driver and the device. + +The features is 64 bits mask of the virtio features bit. For +VIRTIO_TRANSPTQ_CTRL_DRIVER_SET, the feature is passed to the device +through command-out-data. For VIRTIO_TRANSPTQ_CTRL_FEAT_DEVICE_GET and +VIRTIO_TRANSPTQ_CTRL_DRIVER_GET the feature is returned for the device +through command-in-data. + +\devicenormative{\subsubsection}{Features Negotiation}{Virtio Transport Options / +Virtio Over Transport Virtqueue / Features Negotiation}Can one of the above three commands fail? If yes, we need to document the behaviour.
Quoted 6 lines
Quoted 2 lines
can the ack values cover this? Like if failed to read the device feature bits due to IO errors, it can set this EIO below. Or do I miss something?
So you define several different error codes, that means you need to define which error codes is used in which kind of condition.
Or maybe we can simply use a single type of error code.
Sure, only OK and ERR
Quoted 236 lines
Quoted 236 lines
content.tex +61
-0+/* ack values */ +#define VIRTIO_TRANSPTQ_OK 0 +#define VIRTIO_TRANSPTQ_EIO 1 +#define VIRTIO_TRANSPTQ_EAGAIN 11 +#define VIRTIO_TRANSPTQ_EBUSY 16 +#define VIRTIO_TRANSPTQ_EINVAL 22 +#define VIRTIO_TRANSPTQ_ERR 255 +\end{lstlisting} + +The management device MUST fail VIRTIO_TRANSPTQ_F_CTRL_FEAT class for the +command that use 0 as its \field{device_id}. + +\drivernormative{\subsubsection}{Features Negotiation}{Virtio Transport Options / +Virtio Over Transport Virtqueue / Features Negotiation} + +The management driver MAY mediate between the feature negotiation +request of the virtual devices and the transport virtqueue. E.g when +offering features to the virtual device, the management driver MAY +exclude some features in order to limit the behaviour of the virtual +device. + +\subsection{Device Status}\label{sec:Virtio Transport Options / Virtio Over +Transport Virtqueue / Device Status} + +When VIRTIO_F_TRNSPT_VQ_VDEV is negotiated, +the status of virtual device is accessed by the following +commands: + +\begin{lstlisting} +#define VIRTIO_TRANSPTQ_CTRL_STATUS 4 + #define VIRTIO_TRANSPTQ_CTRL_STATUS_SET 0 + #define VIRTIO_TRANSPTQ_CTRL_STATUS_GET 1 +\end{lstlisting} + +The VIRTIO_TRANSPTQ_CTRL_STATUS_SET is used to set the device status of +a virtual device here. The command-out-data is the one byte status +to set to the device. There's no command-in-data for this command. + +The VIRTIO_TRANSPTQ_CTRL_STATUS_GET is used to get the device status of +a virtual device. The command-in-data is the one byte status +returned from the device. There's no command-out-data for this +command. + +\devicenormative{\subsubsection}{Device Status}{Virtio Transport Options / Virtio +Over Transport Virtqueue / Device Status} + +The management device MUST reset the virtual device when 0 +is written via VIRTIO_TRANSPTQ_CTRL_STATUS_SET, the success of this +command demonstrate the success of the reset. + +The management device MUST present 0 through +VIRTIO_TRANSPTQ_CTRL_STATUS_GET once the reset is done. + +The management device MUST fail the device status access if +\field{device_id} is 0. + +\drivernormative{\subsubsection}{Device Status}{Virtio Transport Options / Virtio +Over Transport Virtqueue / Device Status} + +After writing 0 via VIRTIO_TRANSPTQ_CTRL_STATUS_SET, the driver MUST wait +for the success of the command before re-initializing the device.So we need to clarify which is the condition for the succeed of a reset:
1) the succeed of
STATUS_SET(this is how ccw did)
2) 0 is read fromSTATUS_GET(this is how pci did)sure, I think we can use the pci way, present 0 after reset.
content.tex +87
-0+ +\subsection{Device Generation}\label{sec:Virtio Transport Options / Virtio Over Transport Virtqueue / Device Generation} + +When VIRTIO_F_TRNSPT_VQ_VDEV is negotiated, +the device generation count is read from the following commands: + +\begin{lstlisting} +#define VIRTIO_TRANSPTQ_CTRL_GENERATION 5 + #define VIRTIO_TRANSPTQ_CTRL_GENERATION_GET 0 +\end{lstlisting} + +The VIRTIO_TRANSPTQ_CTRL_GENERATION_GET is used to get the device configuration generation field +of the virtual device. The command-in-data is the u32 device +generation returned from the device. There's no command-out-data for +this command. + +\devicenormative{\subsubsection}{Device Generation}{Virtio Transport Options / Virtio Over Transport Virtqueue / Device Generation} + +The management device MUST present a changed generation count after the driver +has read a device-specific configuration value which has changed since +any part of the device-specific configuration was last read. + +The management device MUST fail the device generation access if \field{device_id} is 0. + +\subsection{Device Specific Configuration}\label{sec:Virtio Transport Options / +Virtio Over Transport Virtqueue / Device Specific Configuration} + +When VIRTIO_F_TRNSPT_VQ_VDEV is negotiated, +the device config space contents of a virtual device are accessed through +VIRTIO_TRANSPTQ_CTRL_CONFIG_GET and VIRTIO_TRANSPTQ_CTRL_CONFIG_SET. + +\begin{lstlisting} +#define VIRTIO_TRANSPTQ_CTRL_CONFIG 6 + #define VIRTIO_TRANSPTQ_CTRL_CONFIG_GET 0 + #define VIRTIO_TRANSPTQ_CTRL_CONFIG_SET 1 + +struct virtio_transportq_ctrl_vdev_config_get { + u32 offset; + u32 size; +}; + +struct virtio_transportq_ctrl_vdev_config_set { + u32 offset; + u32 size; + u8 data[]; +}; +\end{lstlisting} + +The VIRTIO_TRANSPTQ_CTRL_CONFIG_GET is used to read data from the +device configuration space. As described in struct +virtio_transportq_ctrl_vdev_config_get, The command-out-data is the offset +from the start of the config space and the size of the data. The +command-in-data is an array of u8 data that read from the config +space. + +The VIRTIO_TRANSPTQ_CTRL_CONFIG_SET is used to write data to the device +configuration space. As described in struct +virtio_transportq_ctrl_vdev_config_set, the command-out-data contains the +offset from the start of the config space, the size of the data and +the data that will be written. There's no command-in-data for this +command. + +\devicenormative{\subsubsection}{Device Specific Configuration}{Virtio Transport +Options / Virtio Over Transport Virtqueue / Device Specific Configuration} + +The management device MUST fail the device configuration space access +if the driver wants to access the range which is outside the config space. + +The management device MUST fail the device configuration space access +if \field{device_id} is 0. + +\subsection{MSI Configuration}\label{sec:Virtio Transport Options / Virtio Over +dmin Virtqueue / MSI Configuration} + +When VIRTIO_F_TRNSPT_VQ_VDEV is negotiated, +the MSI entry for a specific virtqueue is set through following command: + +\begin{lstlisting} +#define VIRTIO_TRANSPTQ_CTRL_MSI 7 + #define VIRTIO_TRANSPTQ_CTRL_MSI_VQ_SET 0 + #define VIRTIO_TRANSPTQ_CTRL_MSI_VQ_ENABLE 1 + #define VIRTIO_TRANSPTQ_CTRL_MSI_VQ_MASK 2 + #define VIRTIO_TRANSPTQ_CTRL_MSI_CONFIG_SET 3 + #define VIRTIO_TRANSPTQ_CTRL_MSI_CONFIG_ENABLE 4 + #define VIRTIO_TRANSPTQ_CTRL_MSI_CONFIG_MASK 5 + +struct virtio_transportq_ctrl_vdev_msi_vq_set {To be consistent with msi, let's use
_vdev_mst_vq_config?OK, it will be
virtio_transportq_ctrl_vdev_msi_vq_configin the next versioncontent.tex +29
-0+ u16 queue_index; + u64 addr; + u32 data; +}; + +struct virtio_transportq_ctrl_vdev_msi_vq_enable { + u16 queue_index; + u8 enable; +}; + +struct virtio_transportq_ctrl_vdev_msi_vq_mask { + u16 queue_index; + u8 mask; +}; + +struct virtio_transportq_ctrl_vdev_msi_config { + u64 addr; + u32 data; +}; +\end{lstlisting} + +The VIRTIO_TRANSPTQ_CTRL_MSI_VQ_SET command is used to set the MSI entry for a +specific virtqueue. The command-out-data is the virtqueue index and +the MSI address and data (as described in struct +virtio_transportq_ctrl_vdev_msix_vq_set). + +The VIRTIO_TRANSPTQ_CTRL_MSI_VQ_ENABLE command is used to enable or disable +the MSI interrupt for a specific virtqueue. The command-out-data is the +virtqueue index and whether to enable the MSI: 0 means to enable and 1It might be better to use macros instead of magic numbers.
sure!
content.tex +38
-0+means to disable (as described in struct +virtio_transportq_ctrl_vdev_msi_vq_enable). + +The VIRTIO_TRANSPTQ_CTRL_MSI_VQ_MASK command is used to mask or unmask the +MSI interrupt for a specific virtqueue. The command-out-data is the +virtqueue index and the mask status: 0 means unmak and 1 means mask +(as described in struct virtio_transportq_ctrl_vdev_msi_vq_mask). + +The VIRTIO_TRANSPTQ_CTRL_MSI_CONFIG_SET command is used to set the MSI entry +for the config interrupt. The command-out-data is the MSI address and +data (as described in struct virtio_transportq_ctrl_vdev_msi_config). + +The VIRTIO_TRANSPTQ_CTRL_MSI_CONFIG_ENABLE command is used to enable and disable +MSI for config space. The command-out-data is an u8: 0 means to +disable and 1 means to enable. + +The VIRTIO_TRANSPTQ_CTRL_MSI_CONFIG_MASK command is used to mask and unmask MSI +interrupt for config space. The command-out-data is an u8: 0 means to +mask and 1 means to unmask. + +There's no command-in-data for all the above MSI commands. + +\devicenormative{\subsubsection}{MSI Configuration}{Virtio Transport Options / Virtio +ver Transport Virtqueue / MSI Configuration} + +The virtual device MUST record the pending MSI interrupts and +re-generate the pending MSI interrupts when unmasking. + +The virtual device MUST disable the MSI for both virtqueue and config space +upon reset. + +\drivernormative{\subsubsection}{MSI Configuration}{Virtio Transport Options / Virtio +Over Transport Virtqueue / MSI Configuration} + +The driver MUST allocate transport or platform specific MSI entries +for both virtqueue and config space if it wants to use interrupt. + +The driver MAY choose disable the MSI if polling mode is used.
Quoted 7 lines
Quoted 6 lines
"choose to disable" ?
when use polling mode, I think it is not a must to disable the interrupts. So the driver may disable the interrupts for better performance or not.
Did I miss something?
I meant gmail reminds me you need a "to" between "choose" and "disable".
OK
Quoted 140 lines
Quoted 140 lines
content.tex +137
-0+ +\subsection{Virtqueue Address}\label{sec:Virtio Transport Options / Virtio Over +Transport Virtqueue / Virtqueue Address} + +When VIRTIO_F_TRNSPT_VQ_VDEV is negotiated, +the addresses of a specific virtqueue are accessed through the following command: + +\begin{lstlisting} +#define VIRTIO_TRANSPTQ_CTRL_VQ_ADDR 8 + #define VIRTIO_TRANSPTQ_CTRL_VQ_ADDR_SET 0 + +struct virtio_transportq_ctrl_vdev_vq_addr { + u16 queue_index; + u64 device_area; + u64 descriptor_area; + u64 driver_area; +}; +\end{lstlisting} + +The command-out-data is the queue index, the addresses of device area, +descriptor area and driver area (as described in struct +virtio_transportq_ctrl_vdev_vq_addr); There's no command-in-data. + +\devicenormative{\subsubsection}{Virtqueue Address}{Virtio Transport Options / Virtio +Over Transport Virtqueue / Virtqueeu Address} + +The management device MUST fail the commands of class +VIRTIO_TRANSPTQ_CTRL_VQ_ADDR if \field{device_id} is 0. + +\subsection{Virtqueue Status}\label{sec:Virtio Transport Options / Virtio Over +Transport Virtqueue / Virtqueue Status} + +When VIRTIO_F_TRNSPT_VQ_VDEV is negotiated, +virtqueue status is set and get through the following command: + +\begin{lstlisting} +#define VIRTIO_TRANSPTQ_CTRL_VQ_ENABLE 9 + #define VIRTIO_TRANSPTQ_CTRL_VQ_ENABLE_SET 0 + #define VIRTIO_TRANSPTQ_CTRL_VQ_ENABLE_GET 1 + +struct virtio_transportq_ctrl_vq_status_set { + u16 queue_index; + u8 status; +}; + +\end{lstlisting} + +The VIRTIO_TRANSPTQ_CTRL_VQ_ENABLE_SET command is used to set the status of a +specific virtqueue. The command-out-data is the queue index and the +status that is set to the virtqueue (0 disabled, 1 enabled), +as described in struct virtio_transportq_ctrl_vq_status_set. +There's no command-in-data. + +The VIRTIO_TRANSPTQ_CTRL_VQ_ENABLE_GET is used to get the status of a +specific virtqueue. The command-out-data is an u16 of the queue +index. The command-in-data is the virtqueue status (0 disabled, 1 +enabled). + +\devicenormative{\subsubsection}{Virtqueue Status}{Virtio Transport Options / Virtio +Over Transport Virtqueue / Virtqueue Status} + +When disabled, the virtual device MUST stop processing requests from +this virtqueue. + +The management device MUST present a 0 via +VIRTIO_TRANSPTQ_CTRL_VQ_ENABLE_GET upon a reset of the virtual device. + +The management device MUST fail the virtqueue status access if +\field{device_id} is 0. + +\drivernormative{\subsubsection}{Virtqueue Status}{Virtio Transport Options / Virtio +Over Transport Virtqueue / Virtqueue Status} + +The driver MUST configure other virtqueue fields before enabling +the virtqueue with VIRTIO_TRANSPTQ_CTRL_VQ_ENABLE_SET. + +\subsection{Virtqueue Size}\label{sec:Virtio Transport Options / Virtio Over +Transport Virtqueue / Virtqueue Size} + +When VIRTIO_F_TRNSPT_VQ_VDEV is negotiated, +virtqueue size is accessed through the following command: + +\begin{lstlisting} +#define VIRTIO_TRANSPTQ_CTRL_VQ_SIZE 10 + #define VIRTIO_TRANSPTQ_CTRL_VQ_SIZE_SET 0 + #define VIRTIO_TRANSPTQ_CTRL_VQ_SIZE_GET 1 + +struct virtio_transportq_ctrl_vdev_vq_size_set { + u16 queue_index; + u16 size; +}; +\end{lstlisting} + +The VIRTIO_TRANSPTQ_CTRL_VQ_SIZE_SET command is used to set the virtqueue +size. The command-out-data is the queue index and the size of the +virtqueue (as described in struct +virtio_transportq_ctrl_vdev_vq_size_set). There's no command-in-data. + +The VIRTIO_TRANSPTQ_CTRL_VQ_SIZE_GET command is used to get the virtqueue +size. On reset, the maximum queue size supported by the device is +returned. The command-out-data is an u16 of the virtqueue index. The +command-in-data is an u16 of queue size for the virtqueue. + +\devicenormative{\subsubsection}{Virtqueue Status}{Virtio Transport Options / Virtio +Over Transport Virtqueue / Virtqueue Size} + +The management device MUST fail the virtqueue size access if +\field{device_id} is 0. + +\subsection{Virtqueue Notification}\label{sec:Virtio Transport Options / Virtio +Over Transport Virtqueue / Virtqueue Notification} + +When VIRTIO_F_TRNSPT_VQ_VDEV is negotiated, +the virtqueue notification is done through the following commands: + +\begin{lstlisting} +#define VIRTIO_TRANSPTQ_CTRL_VQ_NOTIFY 11 + #define VIRTIO_TRANSPTQ_CTRL_VQ_NOTIFY_GET 1 + +struct virtio_transportq_ctrl_vdev_vq_notification_area { + le64 addr + le64 size; +}; +\end{lstlisting} + +The VIRTIO_TRANSPTQ_CTRL_VQ_NOTIFY_GET is used to get the transport +specific address area that can be used to notify a virtqueue. The +command-out-data is an u16 of the virtqueue index. The command-in-data +contains the address and the size of the notification area (as +described in struct virtio_transportq_ctrl_vdev_vq_notification_area). + +\devicenormative{\subsubsection}{Virtqueue Notification}{Virtio Transport Options / +Virtio Over Transport Virtqueue / Virtqueue Notification} + +The management device MUST fail the VIRTIO_TRANSPTQ_CTRL_VQ_NOTIFY_GET command +if there's no transport specific notification address for a virtqueue of +its virtual device.
Quoted 6 lines
Quoted 5 lines
Then how can we kick a device in this case?
I think I should remove this sub-section. Because there should be a notification area, or it is a defective device, and if failed to get the notification area, the command should set a proper ack value reflecting the error
That's fine.
OK, thx
Quoted 52 lines
Quoted 52 lines
content.tex +32
-0+ +The management device MUST fail the virtqueue notification access if +\field{device_id} is 0. + +The management device MUST forbid the notification area of a specific +virtual device to be accessed from another virtual device. + +\drivernormative{\subsubsection}{Virtqueue Notification}{Virtio Transport Options / +Virtio Over Transport Virtqueue / Virtqueue Notification} + +The driver MAY choose to notify the virtqueue by writing the queue +index at address \field{addr} which is fetched from the +VIRTIO_TRANSPTQ_CTRL_VQ_NOTIFY_GET command. + +\subsection{Virtqueue Index}\label{sec:Virtio Transport Options / Virtio +Over Transport Virtqueue / Virtqueue Index} + +When VIRTIO_F_TRNSPT_VQ_VDEV is negotiated, +the virtqueue available index is accessed through the following commands: + +\begin{lstlisting} +#define VIRTIO_TRANSPTQ_CTRL_VQ_AVAIL_INDEX 12 + #define VIRTIO_TRANSPTQ_CTRL_VQ_AVAIL_INDEX_SET 0 + #define VIRTIO_TRANSPTQ_CTRL_VQ_ AVAIL_INDEX_GET 1 + +struct virtio_transportq_ctrl_vdev_vq_avail_idx_set { + u16 queue_index; + u16 avail_idx; +}; +\end{lstlisting} + +The VIRTIO_TRANSPTQ_CTRL_VQ_AVAIL_INDEX_SET commmand is used to set the available index of a specific virtqueue.s/commmand/command/g
We need the support of packed virtqueue as well here.
sure, will work on this.
content.tex +11
-0+The command-out-data is the virtqueue index and the available index (as described in struct virtio_transportq_ctrl_vdev_vq_avail_idx). +There is no command-in-data. + +The VIRTIO_TRANSPTQ_CTRL_VQ_ AVAIL_INDEX_GET command is used to get the available index of a specific virtqueue. +The command-out-data is the queue index, the command-in-data is the available index of the queue. + +\devicenormative{\subsubsection}{Virtqueue Index}{Virtio Transport Options / +Virtio Over Transport Virtqueue / Virtqueue Index} + +The management device MUST fail the virtqueue index access if \field{device_id} is 0 +The management device MUST fail the virtqueue index access if \field{queue_id} is invalid.
Quoted 8 lines
Quoted 7 lines
We don't have similar normatives for other commands, anything make this command different?
If it is PCI, if we set
queue_selectto an invalid value, I think it is a undefined behavior, this could be ignored by the hardware, the device has no ways to report such errors.Here, for the transport virtqueue, it is a must to return an ACK value, and it can be meaningful, like
VIRTIO_TRANSPTQ_EINVALYes, but what I meant is that it applies to other types of commands.
I will add similar description for other commands.
Quoted 147 lines
Thanks
Quoted 111 lines
content.tex +17
-0+ +\subsection{Presenting Virtual Devices}\label{sec:Virtio Transport Options / +Virtio Over Transport Virtqueue / Presenting Virtual Device} + +If VIRTIO_TRANSPTQ_F_VDEV is offered by a management device. The presenting of +the virtual devices requires co-operation between the management +driver and the transport virtqueue. This means, from the view of the +virtual device driver, the transport is done via the communication +with the management device driver. It's up to the software to decide +what kind of method that is needed be used for those communications. + +The management driver typically takes the following steps for creating a +virtual device: + +\begin{enumerate} +\item Determine the virtio id and device specific configuration. +\item Create the virtual devices using the virtio_transportq_ctrl_vdev_CREATE command.Need to use the upper case for the command name here.
yes
+\item Optionally, configure the MSI.
+\item Perform device specific setup.
+\item Let the virtual device to be probed by the virtual devicecontent.tex +3
-0+driver. The management driver will then use the transport virtqueue to +implement the requests of basic facility from the virtual device +driver.I think we need to be more verbose here, e.g to mention the actual command that is used.
sure, will add these
+\end{enumerate}
\chapter{Device Types}\label{sec:Device Types}
content.tex +1
-0@@ -2911,12 +3524,14 @@ \subsection{Virtqueues}\label{sec:Device Types / Network Device / Virtqueues} \item[2(N-1)] receiveqN \item[2(N-1)+1] transmitqN \item[2N] controlq +\item[2N+1] transportqLet's use a separate patch for this.
can do.
Thanks for your review!
\end{description}
N=1 if neither
VIRTIO_NET_F_MQnorVIRTIO_NET_F_RSSare negotiated, otherwise N is set by \field{max_virtqueue_pairs}.- controlq only exists if
VIRTIO_NET_F_CTRL_VQset.
+ controlq only exists ifVIRTIO_NET_F_CTRL_VQset.\\
+ transportq only exists ifVIRTIO_F_TRNSPT_VQis offered.\subsection{Feature bits}\label{sec:Device Types / Network Device / Feature bits}
content.tex +3
-1@@ -4399,10 +5014,12 @@ \subsection{Virtqueues}\label{sec:Device Types / Block Device / Virtqueues} \item[0] requestq1 \item[\ldots] \item[N-1] requestqN +\item[N] transportq \end{description} N=1 if VIRTIO_BLK_F_MQ is not negotiated, otherwise N is set by - \field{num_queues}. + \field{num_queues}.\\ + transportq only exists if VIRTIO_F_TRNSPT_VQ is offered. \subsection{Feature bits}\label{sec:Device Types / Block Device / Feature bits}introduction.tex +3
-0@@ -79,6 +79,9 @@ \section{Normative References}\label{sec:Normative References} \phantomsection\label{intro:SCMI}\textbf{[SCMI]} & Arm System Control and Management Interface, DEN0056, \newline\url{https://developer.arm.com/docs/den0056/c}, version C and any future revisions\\ + \phantomsection\laçbel{intro:SIOV}\textbf{[SIOV]} & + Scalable I/O Virtualization, + \newline\url{https://www.opencompute.org/documents/ocp-scalable-io-virtualization-technical-specification-revision-1-v1-2-pdf}\\ \end{longtable} vThis publicly archived list offers a means to provide input to the
OASIS Virtual I/O Device (VIRTIO) TC.
In order to verify user consent to the Feedback License terms and
to minimize spam in the list archive, subscription is required
before posting.
Subscribe: [email protected]
Unsubscribe: [email protected]
List help: [email protected]
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Quoted 9 lines
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/
This publicly archived list offers a means to provide input to the
OASIS Virtual I/O Device (VIRTIO) TC.
In order to verify user consent to the Feedback License terms and
to minimize spam in the list archive, subscription is required
before posting.
Subscribe: [email protected]
Unsubscribe: [email protected]
List help: [email protected]
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Quoted 8 lines
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/
This publicly archived list offers a means to provide input to the OASIS Virtual I/O Device (VIRTIO) TC.
In order to verify user consent to the Feedback License terms and to minimize spam in the list archive, subscription is required before posting.
Subscribe: [email protected]
Unsubscribe: [email protected]
List help: [email protected]
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/
Quoted 300 lines · Lingshan
Quoted 5 lines · Jason Wang
Quoted 2 lines · Lingshan
Lingshan, please configure your email client properly. It's very hard to differentiate your reply from mine.
Oh, sorry for the inconvenience, maybe the reason is that I have saved a draft in thunderbird and outlook has re-formatted the draft.
Quoted 144 lines
Quoted 144 lines · Jason Wang
On Mon, Jul 4, 2022 at 3:44 PM Zhu Lingshan <[email protected]> wrote:
This patch introduces a new device type: virtio virtual device. An virtio virtual device example is virtio based Scalable I/O Virtualization devices.
This commit also introduces a new transport other than PCI and MMIO
- the
transport virtqueue.This transport is useful for implementing virtual devices with a limited transport specific resources or presenting the virtual device in a transport independent way.
This means, all the basic device facilities are provided solely via the the transport virtqueue. Additionally, the transport virtqueue is also in
Two "the".
will fix
charge of the creating and destroying of the virtual device.
With the help of the transport virtqueue, the presenting of the virtual device is done via the co-operation between the management device and its driver.
Having a dedicated virtqueue is probably fine but there is some overlap at the description of management/managed device with the admin virtqueue proposal from Max. Need to consider a way to unify the definition at least.
OK, I can re-phrase this part
Signed-off-by Jason Wang <[email protected]>Signed-off-by Zhu Lingshan <[email protected]>---
content.tex | 621
++++++++++++++++++++++++++++++++++++++++++++++-
introduction.tex | 3 +2 files changed, 622 insertions(+), 2 deletions(-)content.tex +0
-0@@ -2784,6 +2784,619 @@ \subsubsection{ResettingDevices}\label{sec:Virtio Transport Options / Virtio ov In order to reset a device, a driver sends the
CCW_CMD_VDEV_RESETcommand.+\section{Virtio Over Transport Virtqueue}\label{sec:Virtio Transport Options Virtio Over Transport Virtqueue} +Sometimes it's hard to implement the device in a transport specific method.
+One example is that a physical device may try to present multiple virtual devicescontent.tex +2
-0+with limited transport specific resources. +Another example is to implement virtual devices which are transportindependent.
+In those cases, the transport virtqueue provided by the management device could be used to replacecontent.tex +2
-0+the transport specific method to implement the virtual device. +Then the presenting of the virtual device is done through thecooperation between
content.tex +3
-0+the transport virtqueue and the management device's driver. + +Feature bit VIRTIO_F_TRNSPT_VQ indicates that the device offers atransport virtqueue.
+
+\subsection{Basic Concepts}\label{sec:Virtio Transport Options / Virtio over Transport Virtqueue / Basic Concepts}content.tex +13
-0+ +All transport virtqueue commands are of the following form: + +\begin{lstlisting} +struct virtio_transportq_ctrl { + u64 device_id; + u16 class; + u16 command; + u32 length_out; + u8 command-out-data[]; + u32 length_in; + u32 ack; + u8 command-in-data[];Any reason we need length_in/length_out (we have length in descriptors)? FYI, we don't have them for ctrl virtqueue of networking devices.
OK, I can remove
length_inandlength_out, just pick the length from the descriptors.content.tex +12
-0+}; + +/* ack values */ +#define VIRTIO_TRANSPTQ_OK 0 +#define VIRTIO_TRANSPTQ_EIO 1 +#define VIRTIO_TRANSPTQ_EAGAIN 11 +#define VIRTIO_TRANSPTQ_EBUSY 16 +#define VIRTIO_TRANSPTQ_EINVAL 22 +#define VIRTIO_TRANSPTQ_ERR 255 +\end{lstlisting} + +The \field{device_id}, \field{class}, \field{command},\field{length_out} and
+\field{command-out-data} are set by the management driver, +and the device sets the \field{ack}, \field{length_in} and \field{command-in-data}.content.tex +3
-0+ +\field{device_id} is an UUID to address a virtio virtual device, +The \field{device_id} value 0 is used for identify the managementdevice itself
identifying?
yes, will fix
+
+\subsubsection{The managment devices}\label{sec:Virtio Transport Options / Virtio over Transport Virtqueue / Basic Concepts / The management devices}s/managment/management/
will fix
+A device that offers a transport virtqueue (via feature
VIRTIO_F_TRNSPT_VQ)
+is the management device of the virtual devices which are the managed devices.Not a native speaker but I couldn't parse this sentence.
OK, I can re-phrase this one.
content.tex +2
-0+ +For example, a PCIe PF with a transport virtqueue is a managementdevice.
+
+\subsubsection{The virtual devices}\label{sec:Virtio Transport Options / Virtio over Transport Virtqueue / Basic Concepts / The virtual devices}Quoted 8 lines
Quoted 7 lines
I think to be consistent, per Michael's suggestion, we should use "managed" here which is better than "virtual".
do you mean replace all "virtual device" with "managed device" in this patch?
We can emphasize a virtual device is a type of managed device here, however "virtual device"
implies that the device is a logical device may not has a dedicated physical transport,
like SIOV devices (https://www.opencompute.org/documents/ocp-scalable-io-virtualization-technical-specification-revision-1-v1-2-pdf).That should be fine.
OK, I will use "managed devices" instead of "virtual devices" in the next version
Quoted 33 lines
Quoted 23 lines
+A device that is created through a transport virtqueue which is offered by a management
+device is a virtual device. Virtual devices are logical devices, managed by the management devices,content.tex +3
-0+they usually don't have a dedicated physical transport. + +For example, a Scalable I/O Virtualization\hyperref[intro:SIOV]{[SIOV]} device which is created and managed through a transport virtqueue of +a management device is a virtual device.
Maybe we can remove the "is a virtual device" here, since we use "For example" in the subsection of "The virtual device"
sure
+
+\devicenormative{\subsubsection}{Basic Concepts}{Virtio Transport Options / Virtio Over Transport Virtqueue / Basic Concepts} +The management device MUST offer feature bitVIRTIO_F_TRNSPT_VQand a transport virtqueue as a device specific virtqueue.\\
+The virtual devices MUST not offerVIRTIO_F_TRNSPT_VQfeature.\\Quoted 6 lines
Is this a must? What's the value of prohibiting the nesting of the management device?
Of course it is not a must, I mean it is possible to implement
VIRTIO_F_TRANSPT_VQfor a virtual device. But I am not sure whether it is meaningful to have nested virtual devices, creating virtual device on another virtual device, both layers are logical devices.Not that it's not necessarily a virtual device, it could be another type.
Quoted 2 lines
And this introduce extra complexities for hardware implementation and resource management, like usually don't create VFs from another VF.
Quoted 7 lines
So the implementation can choose to offer
VIRTIO_F_TRNSPT_VQor not for a managed device. That meansThat managed device is managed by its management device. But that managed device can choose to manage sub-managed devices. This almost come for free since the definition in the proposal seems self-contained for such implementation.
sure, I will remove this. So the vendor can choose to enable nested managed devices or not.
Quoted 58 lines
Quoted 58 lines
+The management device MUST fail all commands against the transport virtqueue if
VIRTIO_F_TRNSPT_VQis not negotiated.
+
+\drivernormative{\subsubsection}{Basic Concepts}{Virtio Transport Options / Virtio Over Transport Virtqueue / Basic Concepts} +The management driver MUST not send any commands against the transport virtqueue ifVIRTIO_F_TRNSPT_VQis not negotiated. +
+\subsection{Virtual Devices Discovery}\label{sec:Virtio Transport Options /Virtio Over Transport Virtqueue / Virtual Devices Discovery}content.tex +2
-0+ +The management device is discovered through its own transport anddevice specific method.
+Virtual devices are created and discovered via the transport virtqueue.
+
+\subsection{Transport Virtqueue Features}\label{sec:Virtio
Transport Options /Virtio Over Transport Virtqueue / Transport Virtqueue Features}content.tex +2
-0+ +Feature bit VIRTIO_F_TRNSPT_VQ_VDEV indicates that the virtualdevices are created, configured
content.tex +2
-0+and destroyed through the transport virtqueue. +This means the transport virtqueue is the transport for the virtualdevices.
content.tex +4
-0+ +Feature bit VIRTIO_F_TRNSPT_VQ_VDEV depends on VIRTIO_F_TRNSPT_VQ + +\devicenormative{\subsubsection}{Transport VirtqueueFeatures}{Virtio Transport Options / Virtio Over Transport Virtqueue / Transport Virtqueue Features} +The management device MUST not accept
VIRTIO_F_TRNSPT_VQ_VDEVifVIRTIO_F_TRNSPT_VQis not negotiated.
+
+\subsection{Device Management}\label{sec:Virtio Transport Options /
Virtio Over Transport Virtqueue / Device Management}content.tex +3
-0+ +When VIRTIO_F_TRNSPT_VQ_VDEV is negotiated, +virtual devices must be created and destroyed through the transportvirtqueue.
content.tex +14
-0+ +\begin{lstlisting} +struct virtio_transportq_ctrl_vdev_attribute { + u32 virtio_device_id; + u32 asid; + u8 config[]; +}; + +#define virtio_transportq_ctrl_vdev 1 (class) + #define virtio_transportq_ctrl_vdev_CREATE 0 (command) + #define virtio_transportq_ctrl_vdev_DESTROY 1 (command) +\end{lstlisting} + +The virtio_transportq_ctrl_vdev_CREATE command is used to create avirtual device.
+The command-out-data forVIRTIO_TRANSPTQ_CTRL_CREATEis the virtio device id (\field{virtio_device_id}),
+the address space id (\field{asid})Quoted 6 lines
Quoted 4 lines
We need to be verbose here. Is this a transport specific identifier or a device specific identifier?
This asid field is an id for the device, it should be the PASID. I will add more explanations here.
I think we'd better try hard to avoid mandating transport specific ASID like PASID here. Instead, we can use PASID as an example.
OK
Quoted 11 lines
Quoted 8 lines
I also wonder if this is the best place for this. To me, it looks more suitable to have a dedicated command to configure it before
DRIVER_OKlike virtqueue address. Then we can configure it per virtqueue per device with more flexibility.I understand the point, this can gives the device more flexibility for sure. It is the guest driver initialize the device.
Note that we can't allow guests to set ASID directly, the hypervisor must trap any access to ASID assignment from guests and convert it to host ASID.
I will add a new command to set vq asid in the next version.
Quoted 19 lines
Quoted 6 lines
Device status and vq configs are usually set by the guest driver,
and the guest driver is not aware of the asid, it is the host side setting the asid. So shall we set the asid uponDRIVER_OK?
And how to decide whether the device should use a device-global asid or a per-vq asid, or even hybrid? I think this is transparent to the guestSee above.
Quoted 11 lines
+and device specific configuration (\field{config}) for creating the virtual device.
+When succeed, the device returns an u64 as a unique identifier of the created virtual device in command-in-data.content.tex +2
-0+ +The field \field{config} is device type specific. E.g., forvirtio-net, it should follow the struct
virtio_net_configI think we need a dedicated section to describe the transport virtqueue support for virtio-net.
Quoted 8 lines
Quoted 5 lines
Quoted 1 line
More hints for what should be discussed in the section?
Need a dedicated patch to support virtio-net. And describe the format accurately.
The
virtio_net_configis not sufficient, we need at least features.Yes, I will add features for the creation, add rename config[] to dev_config[]. I should also add the number of vqs for the creation, and a new command to get the number of vqs of a managed device. I missed these in this version.
I am replying to my own reply, I realized the number of vqs could be set / get through device config space and the feature bits.
Thanks
Quoted 908 lines
Quoted 7 lines
Quoted 4 lines
I see all the descriptions for virtio device types are in chapter 5 Device Types, other transports don't have their own sections for the device types, and chapter 5 discussed that in general.
I think we need
1) add transport vq in virqtueues subsection (as this patch did)
will move these contents to a separate patch
Quoted 2 lines
2) another subsection to describe the config for creating managed(virtual) devices
I will add more detailed description in the subsection "Presenting Virtual Devices", and take virtio-net as an example.
Quoted 41 lines
Quoted 41 lines
Here we take virtio-net device as an example to explain the filed config contents.
+in section \ref{sec:Device Types / Network Device / Device configuration layout}
content.tex +7
-0+ +The virtio_transportq_ctrl_vdev_DESTROY command is used to destroy a +virtual device which is identified by its 64bit identifier +\field{virtual_device_id}. There's no command-in-data for +VIRTIO_TRANSPTQ_CTRL_DESTROY command. + +\devicenormative{\subsubsection}{Device Management}{VirtioTransport Options / Virtio Over Transport Virtqueue / Device Management}
content.tex +2
-0+ +The management device MUST fail the
virtio_transportq_ctrl_vdev_CREATEif
+\field{device_id} is not 0.Let's use
VIRTIO_TRANSPTQ_OKinstead of the magic number here.this zero is the filed
device_id= 0, means identifying the management device. And I found I have mixed up the upper and lower cases in the commands, they should all be upper cases, I will fix this.content.tex +2
-0+ +The management device MUST fail the
virtio_transportq_ctrl_vdev_DESTROYif
+\field{device_id} is 0.Same here.
content.tex +2
-0+ +All virtual devices MUST be created via the transport virtqueue ifthe management
+device offersVIRTIO_F_TRNSPT_VQ_VDEV.Does this imply no pre-created virtual devices? If this is true, we don't need a dedicated feature bit (
VIRTIO_F_TRNSPT_VQ_VDEV).There can be pre-created virtual devices, in the following sections,
Quoted 7 lines
Quoted 5 lines
struct mgmt_dev_avail_res{}, if
min_vdev_vqs==max_vdev_vqs, means the management device only supports creating virtual devices with fixed capabilities, this kind of virtual devices can be pre-created. Like the slab allocator, vendors can pre-create the device. Then when the management device receiving theVDEV_CREATEcommand, it can just pick up a free virtual device, and return the virtual device.So the question still remains, how could we discover those pre-created devices? (having something like
initial_vfs?)I think for the management devices, pre-creation could be pre-allocated
hw resource. The management device could pre-allocate a certain number of "standard" managed devices, when receive a create command, the management device can pick a free managed device, assign an UUID to it, then return
to the management driver. So the driver may not be aware of this pre-creation, it just sees the device is created rapidly. Just like the slab system, or a threads pool.Quoted 6 lines
Quoted 6 lines
The bit only makes sense for the case when some managed(virtual) devices are pre-created like
initial_vfsin SR-IOV. If we allow pre-creation, we need commands to discover them.Quoted 7 lines
Quoted 6 lines
I think we need the feature bit
VIRTIO_F_TRNSPT_VQ_VDEV, this feature bit indicates that the transport virtqueue is the transport layer for the virtual devices, like PCI for a virtio PCI hardware. So, not only create, we use the transport virtqueue to config the device as well.And if the virtual devices are pre-created, we can just send the create command, and the management device can find a available device for us, this is also discovery.
This seems to conflict with the semantics of pre-created devices.
same to the above
Quoted 129 lines
Quoted 129 lines
+
+\drivernormative{\subsubsection}{Driver Management}{Virtio Transport Options / Virtio Over Transport Virtqueue / Device Management}content.tex +5
-0+ +The management driver MUST use 0 as \field{device_id} for +virtio_transportq_ctrl_vdev_CREATE command. + +\subsection{Available resource of the managementdevice}\label{sec:Virtio Transport Options / Virtio Over Transport Virtqueue / Available resource of the management devic}
content.tex +2
-0+When VIRTIO_F_TRNSPT_VQ_VDEV is negotiated, +statistical information of available resource of the managementdevice could be fetched
content.tex +5
-0+by the following command: + +\begin{lstlisting} +#define VIRTIO_TRANSPTQ_CTRL_AVAIL_RES 2 + #define VIRTIO_TRANSPTQ_CTRL_AVAIL_RES_GET 0Nit, I think this command should come first, mgmt need to query this before creating devices.
sure, I can re-order the commands.
+\end{lstlisting}
content.tex +2
-0+ +VIRTIO_TRANSPTQ_CTRL_AVAIL_RES_GET command is used to get thestatstical information of available
s/statstical/statistical/
will fix
+resource of the management device. There is no command-out-data for
VIRTIO_TRANSPTQ_CTRL_AVAIL_RES_GET
+command. The management device fills command-in-data in the following format:content.tex +4
-0+ +\begin{lstlisting} +struct mgmt_dev_avail_res{ + /* The number of total remaining virtqueus of themanagement device */
Should be virtqueues.
Actually the description here is confusing, something like this might be better
"The number of total remaining virtqueues for the managed(virtual) devices */
will fix this.
+ u16
num_avail_vqs;
+ /* The minimal number of virtqueues of the virtual devices */"of a virtual device"?
will fix
+ u16
min_vdev_vqs;
+ /* The maximum number of virtqueues of the virtual devices */ + u16max_vdev_vqs;
+ /* The number of virtual devices that can be created withmin_vdev_vqsvirtqueues.
+ * This number may not equal tonum_avail_vqsdivided bymin_vdev_vqsdue to
+ * the limitations of other resource
+ */
+ u16num_avail_vdev;content.tex +4
-0+}; +\end{lstlisting} + +If \field{min_vdev_vqs} equals to \field{min_vdev_vqs}, that meansthe management device
content.tex +3
-0+can only create virtual devices with a fixed number of virtqueues + +\devicenormative{\subsubsection}{Available resource of themanagement device}{Virtio Transport Options /
+Virtio Over Transport Virtqueue / Available resource of the management device}
+The management device MUST fail theVIRTIO_TRANSPTQ_CTRL_AVAIL_RES_GETcommand if \field{device_id} is not 0 +
+\drivernormative{\subsubsection}{Available resource of the management device}{Virtio Transport Options /
+Virtio Over Transport Virtqueue / Available resource of the management device}content.tex +4
-0+The management driver MUST use 0 as \field{device_id} for +VIRTIO_TRANSPTQ_CTRL_AVAIL_RES_GET command. + +\subsection{Features Negotiation}\label{sec:Virtio TransportOptions / Virtio
content.tex +14
-0+Over Transport Virtqueue / Features Negotiation} + +When VIRTIO_F_TRNSPT_VQ_VDEV is negotiated, +the feature negotiation of virtual devices is done by the +following commands: + +\begin{lstlisting} +#define VIRTIO_TRANSPTQ_CTRL_FEAT 3 + #define VIRTIO_TRANSPTQ_CTRL_FEAT_DEVICE_GET 0 + #define VIRTIO_TRANSPTQ_CTRL_FEAT_DRIVER_SET 1 + #define VIRTIO_TRANSPTQ_CTRL_FEAT_DRIVER_GET 2 +\end{lstlisting} + +The VIRTIO_TRANSPTQ_CTRL_FEAT_DEVICE_GET is to get the featuresoffered
content.tex +3
-0+by a virtual device. + +The VIRTIO_TRANSPTQ_CTRL_FEAT_DRIVER_SET is for driver to acceptfeature
content.tex +3
-0+bits offered by the virtual device. + +The VIRTIO_TRANSPTQ_CTRL_FEAT_DRIVER_GET is to get the featuresaccepted
content.tex +9
-0+by both the virtual driver and the device. + +The features is 64 bits mask of the virtio features bit. For +VIRTIO_TRANSPTQ_CTRL_DRIVER_SET, the feature is passed to the device +through command-out-data. For VIRTIO_TRANSPTQ_CTRL_FEAT_DEVICE_GET and +VIRTIO_TRANSPTQ_CTRL_DRIVER_GET the feature is returned for the device +through command-in-data. + +\devicenormative{\subsubsection}{Features Negotiation}{VirtioTransport Options /
+Virtio Over Transport Virtqueue / Features Negotiation}Can one of the above three commands fail? If yes, we need to document the behaviour.
Quoted 6 lines
Quoted 2 lines
can the ack values cover this? Like if failed to read the device feature bits due to IO errors, it can set this EIO below. Or do I miss something?
So you define several different error codes, that means you need to define which error codes is used in which kind of condition.
Or maybe we can simply use a single type of error code.
Sure, only OK and ERR
Quoted 269 lines
Quoted 269 lines
content.tex +10
-0+/* ack values */ +#define VIRTIO_TRANSPTQ_OK 0 +#define VIRTIO_TRANSPTQ_EIO 1 +#define VIRTIO_TRANSPTQ_EAGAIN 11 +#define VIRTIO_TRANSPTQ_EBUSY 16 +#define VIRTIO_TRANSPTQ_EINVAL 22 +#define VIRTIO_TRANSPTQ_ERR 255 +\end{lstlisting} + +The management device MUST fail VIRTIO_TRANSPTQ_F_CTRL_FEAT classfor the
content.tex +3
-0+command that use 0 as its \field{device_id}. + +\drivernormative{\subsubsection}{Features Negotiation}{VirtioTransport Options /
content.tex +9
-0+Virtio Over Transport Virtqueue / Features Negotiation} + +The management driver MAY mediate between the feature negotiation +request of the virtual devices and the transport virtqueue. E.g when +offering features to the virtual device, the management driver MAY +exclude some features in order to limit the behaviour of the virtual +device. + +\subsection{Device Status}\label{sec:Virtio Transport Options /Virtio Over
content.tex +13
-0+Transport Virtqueue / Device Status} + +When VIRTIO_F_TRNSPT_VQ_VDEV is negotiated, +the status of virtual device is accessed by the following +commands: + +\begin{lstlisting} +#define VIRTIO_TRANSPTQ_CTRL_STATUS 4 + #define VIRTIO_TRANSPTQ_CTRL_STATUS_SET 0 + #define VIRTIO_TRANSPTQ_CTRL_STATUS_GET 1 +\end{lstlisting} + +The VIRTIO_TRANSPTQ_CTRL_STATUS_SET is used to set the devicestatus of
content.tex +4
-0+a virtual device here. The command-out-data is the one byte status +to set to the device. There's no command-in-data for this command. + +The VIRTIO_TRANSPTQ_CTRL_STATUS_GET is used to get the devicestatus of
content.tex +5
-0+a virtual device. The command-in-data is the one byte status +returned from the device. There's no command-out-data for this +command. + +\devicenormative{\subsubsection}{Device Status}{Virtio TransportOptions / Virtio
content.tex +13
-0+Over Transport Virtqueue / Device Status} + +The management device MUST reset the virtual device when 0 +is written via VIRTIO_TRANSPTQ_CTRL_STATUS_SET, the success of this +command demonstrate the success of the reset. + +The management device MUST present 0 through +VIRTIO_TRANSPTQ_CTRL_STATUS_GET once the reset is done. + +The management device MUST fail the device status access if +\field{device_id} is 0. + +\drivernormative{\subsubsection}{Device Status}{Virtio TransportOptions / Virtio
content.tex +3
-0+Over Transport Virtqueue / Device Status} + +After writing 0 via VIRTIO_TRANSPTQ_CTRL_STATUS_SET, the driverMUST wait
+for the success of the command before re-initializing the device.So we need to clarify which is the condition for the succeed of a reset:
1) the succeed of
STATUS_SET(this is how ccw did)
2) 0 is read fromSTATUS_GET(this is how pci did)sure, I think we can use the pci way, present 0 after reset.
+
+\subsection{Device Generation}\label{sec:Virtio Transport Options / Virtio Over Transport Virtqueue / Device Generation}content.tex +10
-0+ +When VIRTIO_F_TRNSPT_VQ_VDEV is negotiated, +the device generation count is read from the following commands: + +\begin{lstlisting} +#define VIRTIO_TRANSPTQ_CTRL_GENERATION 5 + #define VIRTIO_TRANSPTQ_CTRL_GENERATION_GET 0 +\end{lstlisting} + +The VIRTIO_TRANSPTQ_CTRL_GENERATION_GET is used to get the deviceconfiguration generation field
content.tex +5
-0+of the virtual device. The command-in-data is the u32 device +generation returned from the device. There's no command-out-data for +this command. + +\devicenormative{\subsubsection}{Device Generation}{VirtioTransport Options / Virtio Over Transport Virtqueue / Device Generation}
content.tex +2
-0+ +The management device MUST present a changed generation count afterthe driver
content.tex +4
-0+has read a device-specific configuration value which has changed since +any part of the device-specific configuration was last read. + +The management device MUST fail the device generation access if\field{device_id} is 0.
+
+\subsection{Device Specific Configuration}\label{sec:Virtio Transport Options /content.tex +4
-0+Virtio Over Transport Virtqueue / Device Specific Configuration} + +When VIRTIO_F_TRNSPT_VQ_VDEV is negotiated, +the device config space contents of a virtual device are accessedthrough
content.tex +22
-0+VIRTIO_TRANSPTQ_CTRL_CONFIG_GET and VIRTIO_TRANSPTQ_CTRL_CONFIG_SET. + +\begin{lstlisting} +#define VIRTIO_TRANSPTQ_CTRL_CONFIG 6 + #define VIRTIO_TRANSPTQ_CTRL_CONFIG_GET 0 + #define VIRTIO_TRANSPTQ_CTRL_CONFIG_SET 1 + +struct virtio_transportq_ctrl_vdev_config_get { + u32 offset; + u32 size; +}; + +struct virtio_transportq_ctrl_vdev_config_set { + u32 offset; + u32 size; + u8 data[]; +}; +\end{lstlisting} + +The VIRTIO_TRANSPTQ_CTRL_CONFIG_GET is used to read data from the +device configuration space. As described in struct +virtio_transportq_ctrl_vdev_config_get, The command-out-data is theoffset
content.tex +5
-0+from the start of the config space and the size of the data. The +command-in-data is an array of u8 data that read from the config +space. + +The VIRTIO_TRANSPTQ_CTRL_CONFIG_SET is used to write data to thedevice
content.tex +2
-0+configuration space. As described in struct +virtio_transportq_ctrl_vdev_config_set, the command-out-datacontains the
content.tex +5
-0+offset from the start of the config space, the size of the data and +the data that will be written. There's no command-in-data for this +command. + +\devicenormative{\subsubsection}{Device SpecificConfiguration}{Virtio Transport
+Options / Virtio Over Transport Virtqueue / Device Specific Configuration}content.tex +3
-0+ +The management device MUST fail the device configuration space access +if the driver wants to access the range which is outside the configspace.
content.tex +5
-0+ +The management device MUST fail the device configuration space access +if \field{device_id} is 0. + +\subsection{MSI Configuration}\label{sec:Virtio Transport Options /Virtio Over
content.tex +4
-0+dmin Virtqueue / MSI Configuration} + +When VIRTIO_F_TRNSPT_VQ_VDEV is negotiated, +the MSI entry for a specific virtqueue is set through followingcommand:
content.tex +11
-0+ +\begin{lstlisting} +#define VIRTIO_TRANSPTQ_CTRL_MSI 7 + #define VIRTIO_TRANSPTQ_CTRL_MSI_VQ_SET 0 + #define VIRTIO_TRANSPTQ_CTRL_MSI_VQ_ENABLE 1 + #define VIRTIO_TRANSPTQ_CTRL_MSI_VQ_MASK 2 + #define VIRTIO_TRANSPTQ_CTRL_MSI_CONFIG_SET 3 + #define VIRTIO_TRANSPTQ_CTRL_MSI_CONFIG_ENABLE 4 + #define VIRTIO_TRANSPTQ_CTRL_MSI_CONFIG_MASK 5 + +struct virtio_transportq_ctrl_vdev_msi_vq_set {To be consistent with msi, let's use
_vdev_mst_vq_config?OK, it will be
virtio_transportq_ctrl_vdev_msi_vq_configin the next version+ u16
queue_index;
+ u64 addr;
+ u32 data;content.tex +19
-0+}; + +struct virtio_transportq_ctrl_vdev_msi_vq_enable { + u16 queue_index; + u8 enable; +}; + +struct virtio_transportq_ctrl_vdev_msi_vq_mask { + u16 queue_index; + u8 mask; +}; + +struct virtio_transportq_ctrl_vdev_msi_config { + u64 addr; + u32 data; +}; +\end{lstlisting} + +The VIRTIO_TRANSPTQ_CTRL_MSI_VQ_SET command is used to set the MSIentry for a
content.tex +5
-0+specific virtqueue. The command-out-data is the virtqueue index and +the MSI address and data (as described in struct +virtio_transportq_ctrl_vdev_msix_vq_set). + +The VIRTIO_TRANSPTQ_CTRL_MSI_VQ_ENABLE command is used to enable ordisable
+the MSI interrupt for a specific virtqueue. The command-out-data is the
+virtqueue index and whether to enable the MSI: 0 means to enable and 1It might be better to use macros instead of magic numbers.
sure!
content.tex +4
-0+means to disable (as described in struct +virtio_transportq_ctrl_vdev_msi_vq_enable). + +The VIRTIO_TRANSPTQ_CTRL_MSI_VQ_MASK command is used to mask orunmask the
content.tex +5
-0+MSI interrupt for a specific virtqueue. The command-out-data is the +virtqueue index and the mask status: 0 means unmak and 1 means mask +(as described in struct virtio_transportq_ctrl_vdev_msi_vq_mask). + +The VIRTIO_TRANSPTQ_CTRL_MSI_CONFIG_SET command is used to set theMSI entry
content.tex +4
-0+for the config interrupt. The command-out-data is the MSI address and +data (as described in struct virtio_transportq_ctrl_vdev_msi_config). + +The VIRTIO_TRANSPTQ_CTRL_MSI_CONFIG_ENABLE command is used toenable and disable
content.tex +4
-0+MSI for config space. The command-out-data is an u8: 0 means to +disable and 1 means to enable. + +The VIRTIO_TRANSPTQ_CTRL_MSI_CONFIG_MASK command is used to maskand unmask MSI
content.tex +6
-0+interrupt for config space. The command-out-data is an u8: 0 means to +mask and 1 means to unmask. + +There's no command-in-data for all the above MSI commands. + +\devicenormative{\subsubsection}{MSI Configuration}{VirtioTransport Options / Virtio
content.tex +6
-0+ver Transport Virtqueue / MSI Configuration} + +The virtual device MUST record the pending MSI interrupts and +re-generate the pending MSI interrupts when unmasking. + +The virtual device MUST disable the MSI for both virtqueue andconfig space
content.tex +3
-0+upon reset. + +\drivernormative{\subsubsection}{MSI Configuration}{VirtioTransport Options / Virtio
content.tex +6
-0+Over Transport Virtqueue / MSI Configuration} + +The driver MUST allocate transport or platform specific MSI entries +for both virtqueue and config space if it wants to use interrupt. + +The driver MAY choose disable the MSI if polling mode is used.Quoted 7 lines
Quoted 6 lines
"choose to disable" ?
when use polling mode, I think it is not a must to disable the interrupts. So the driver may disable the interrupts for better performance or not.
Did I miss something?
I meant gmail reminds me you need a "to" between "choose" and "disable".
OK
Quoted 155 lines
Quoted 155 lines
+
+\subsection{Virtqueue Address}\label{sec:Virtio Transport Options / Virtio Overcontent.tex +4
-0+Transport Virtqueue / Virtqueue Address} + +When VIRTIO_F_TRNSPT_VQ_VDEV is negotiated, +the addresses of a specific virtqueue are accessed through thefollowing command:
content.tex +18
-0+ +\begin{lstlisting} +#define VIRTIO_TRANSPTQ_CTRL_VQ_ADDR 8 + #define VIRTIO_TRANSPTQ_CTRL_VQ_ADDR_SET 0 + +struct virtio_transportq_ctrl_vdev_vq_addr { + u16 queue_index; + u64 device_area; + u64 descriptor_area; + u64 driver_area; +}; +\end{lstlisting} + +The command-out-data is the queue index, the addresses of device area, +descriptor area and driver area (as described in struct +virtio_transportq_ctrl_vdev_vq_addr); There's no command-in-data. + +\devicenormative{\subsubsection}{Virtqueue Address}{VirtioTransport Options / Virtio
content.tex +6
-0+Over Transport Virtqueue / Virtqueeu Address} + +The management device MUST fail the commands of class +VIRTIO_TRANSPTQ_CTRL_VQ_ADDR if \field{device_id} is 0. + +\subsection{Virtqueue Status}\label{sec:Virtio Transport Options /Virtio Over
content.tex +18
-0+Transport Virtqueue / Virtqueue Status} + +When VIRTIO_F_TRNSPT_VQ_VDEV is negotiated, +virtqueue status is set and get through the following command: + +\begin{lstlisting} +#define VIRTIO_TRANSPTQ_CTRL_VQ_ENABLE 9 + #define VIRTIO_TRANSPTQ_CTRL_VQ_ENABLE_SET 0 + #define VIRTIO_TRANSPTQ_CTRL_VQ_ENABLE_GET 1 + +struct virtio_transportq_ctrl_vq_status_set { + u16 queue_index; + u8 status; +}; + +\end{lstlisting} + +The VIRTIO_TRANSPTQ_CTRL_VQ_ENABLE_SET command is used to set thestatus of a
content.tex +11
-0+specific virtqueue. The command-out-data is the queue index and the +status that is set to the virtqueue (0 disabled, 1 enabled), +as described in struct virtio_transportq_ctrl_vq_status_set. +There's no command-in-data. + +The VIRTIO_TRANSPTQ_CTRL_VQ_ENABLE_GET is used to get the status of a +specific virtqueue. The command-out-data is an u16 of the queue +index. The command-in-data is the virtqueue status (0 disabled, 1 +enabled). + +\devicenormative{\subsubsection}{Virtqueue Status}{Virtio TransportOptions / Virtio
content.tex +12
-0+Over Transport Virtqueue / Virtqueue Status} + +When disabled, the virtual device MUST stop processing requests from +this virtqueue. + +The management device MUST present a 0 via +VIRTIO_TRANSPTQ_CTRL_VQ_ENABLE_GET upon a reset of the virtual device. + +The management device MUST fail the virtqueue status access if +\field{device_id} is 0. + +\drivernormative{\subsubsection}{Virtqueue Status}{Virtio TransportOptions / Virtio
content.tex +6
-0+Over Transport Virtqueue / Virtqueue Status} + +The driver MUST configure other virtqueue fields before enabling +the virtqueue with VIRTIO_TRANSPTQ_CTRL_VQ_ENABLE_SET. + +\subsection{Virtqueue Size}\label{sec:Virtio Transport Options /Virtio Over
content.tex +17
-0+Transport Virtqueue / Virtqueue Size} + +When VIRTIO_F_TRNSPT_VQ_VDEV is negotiated, +virtqueue size is accessed through the following command: + +\begin{lstlisting} +#define VIRTIO_TRANSPTQ_CTRL_VQ_SIZE 10 + #define VIRTIO_TRANSPTQ_CTRL_VQ_SIZE_SET 0 + #define VIRTIO_TRANSPTQ_CTRL_VQ_SIZE_GET 1 + +struct virtio_transportq_ctrl_vdev_vq_size_set { + u16 queue_index; + u16 size; +}; +\end{lstlisting} + +The VIRTIO_TRANSPTQ_CTRL_VQ_SIZE_SET command is used to set thevirtqueue
content.tex +5
-0+size. The command-out-data is the queue index and the size of the +virtqueue (as described in struct +virtio_transportq_ctrl_vdev_vq_size_set). There's no command-in-data. + +The VIRTIO_TRANSPTQ_CTRL_VQ_SIZE_GET command is used to get thevirtqueue
content.tex +5
-0+size. On reset, the maximum queue size supported by the device is +returned. The command-out-data is an u16 of the virtqueue index. The +command-in-data is an u16 of queue size for the virtqueue. + +\devicenormative{\subsubsection}{Virtqueue Status}{Virtio TransportOptions / Virtio
content.tex +6
-0+Over Transport Virtqueue / Virtqueue Size} + +The management device MUST fail the virtqueue size access if +\field{device_id} is 0. + +\subsection{Virtqueue Notification}\label{sec:Virtio TransportOptions / Virtio
content.tex +22
-0+Over Transport Virtqueue / Virtqueue Notification} + +When VIRTIO_F_TRNSPT_VQ_VDEV is negotiated, +the virtqueue notification is done through the following commands: + +\begin{lstlisting} +#define VIRTIO_TRANSPTQ_CTRL_VQ_NOTIFY 11 + #define VIRTIO_TRANSPTQ_CTRL_VQ_NOTIFY_GET 1 + +struct virtio_transportq_ctrl_vdev_vq_notification_area { + le64 addr + le64 size; +}; +\end{lstlisting} + +The VIRTIO_TRANSPTQ_CTRL_VQ_NOTIFY_GET is used to get the transport +specific address area that can be used to notify a virtqueue. The +command-out-data is an u16 of the virtqueue index. The command-in-data +contains the address and the size of the notification area (as +described in struct virtio_transportq_ctrl_vdev_vq_notification_area). + +\devicenormative{\subsubsection}{Virtqueue Notification}{VirtioTransport Options /
content.tex +3
-0+Virtio Over Transport Virtqueue / Virtqueue Notification} + +The management device MUST fail the
VIRTIO_TRANSPTQ_CTRL_VQ_NOTIFY_GETcommand
+if there's no transport specific notification address for a virtqueue of
+its virtual device.Quoted 6 lines
Quoted 5 lines
Then how can we kick a device in this case?
I think I should remove this sub-section. Because there should be a notification area, or it is a defective device, and if failed to get the notification area, the command should set a proper ack value reflecting the error
That's fine.
OK, thx
Quoted 62 lines
Quoted 62 lines
content.tex +8
-0+ +The management device MUST fail the virtqueue notification access if +\field{device_id} is 0. + +The management device MUST forbid the notification area of a specific +virtual device to be accessed from another virtual device. + +\drivernormative{\subsubsection}{Virtqueue Notification}{VirtioTransport Options /
content.tex +7
-0+Virtio Over Transport Virtqueue / Virtqueue Notification} + +The driver MAY choose to notify the virtqueue by writing the queue +index at address \field{addr} which is fetched from the +VIRTIO_TRANSPTQ_CTRL_VQ_NOTIFY_GET command. + +\subsection{Virtqueue Index}\label{sec:Virtio Transport Options /Virtio
content.tex +4
-0+Over Transport Virtqueue / Virtqueue Index} + +When VIRTIO_F_TRNSPT_VQ_VDEV is negotiated, +the virtqueue available index is accessed through the followingcommands:
content.tex +13
-0+ +\begin{lstlisting} +#define VIRTIO_TRANSPTQ_CTRL_VQ_AVAIL_INDEX 12 + #define VIRTIO_TRANSPTQ_CTRL_VQ_AVAIL_INDEX_SET 0 + #define VIRTIO_TRANSPTQ_CTRL_VQ_ AVAIL_INDEX_GET 1 + +struct virtio_transportq_ctrl_vdev_vq_avail_idx_set { + u16 queue_index; + u16 avail_idx; +}; +\end{lstlisting} + +The VIRTIO_TRANSPTQ_CTRL_VQ_AVAIL_INDEX_SET commmand is used to setthe available index of a specific virtqueue.
s/commmand/command/g
We need the support of packed virtqueue as well here.
sure, will work on this.
+The command-out-data is the virtqueue index and the available index (as described in struct
virtio_transportq_ctrl_vdev_vq_avail_idx).content.tex +3
-0+There is no command-in-data. + +The VIRTIO_TRANSPTQ_CTRL_VQ_ AVAIL_INDEX_GET command is used to getthe available index of a specific virtqueue.
+The command-out-data is the queue index, the command-in-data is the available index of the queue.
+
+\devicenormative{\subsubsection}{Virtqueue Index}{Virtio Transport Options /content.tex +3
-0+Virtio Over Transport Virtqueue / Virtqueue Index} + +The management device MUST fail the virtqueue index access if\field{device_id} is 0
+The management device MUST fail the virtqueue index access if \field{queue_id} is invalid.Quoted 8 lines
Quoted 7 lines
We don't have similar normatives for other commands, anything make this command different?
If it is PCI, if we set
queue_selectto an invalid value, I think it is a undefined behavior, this could be ignored by the hardware, the device has no ways to report such errors.Here, for the transport virtqueue, it is a must to return an ACK value, and it can be meaningful, like
VIRTIO_TRANSPTQ_EINVALYes, but what I meant is that it applies to other types of commands.
I will add similar description for other commands.
Quoted 155 lines
Thanks
Quoted 119 lines
+
+\subsection{Presenting Virtual Devices}\label{sec:Virtio Transport Options /content.tex +3
-0+Virtio Over Transport Virtqueue / Presenting Virtual Device} + +If VIRTIO_TRANSPTQ_F_VDEV is offered by a management device. Thepresenting of
content.tex +7
-0+the virtual devices requires co-operation between the management +driver and the transport virtqueue. This means, from the view of the +virtual device driver, the transport is done via the communication +with the management device driver. It's up to the software to decide +what kind of method that is needed be used for those communications. + +The management driver typically takes the following steps forcreating a
content.tex +5
-0+virtual device: + +\begin{enumerate} +\item Determine the virtio id and device specific configuration. +\item Create the virtual devices using the
virtio_transportq_ctrl_vdev_CREATEcommand.Need to use the upper case for the command name here.
yes
+\item Optionally, configure the MSI.
+\item Perform device specific setup.
+\item Let the virtual device to be probed by the virtual devicecontent.tex +3
-0+driver. The management driver will then use the transport virtqueue to +implement the requests of basic facility from the virtual device +driver.I think we need to be more verbose here, e.g to mention the actual command that is used.
sure, will add these
+\end{enumerate}
\chapter{Device Types}\label{sec:Device Types}
content.tex +0
-0@@ -2911,12 +3524,14 @@ \subsection{Virtqueues}\label{sec:Device Types / Network Device / Virtqueues}\item[2(N-1)] receiveqN
\item[2(N-1)+1] transmitqN
\item[2N] controlq
+\item[2N+1] transportqLet's use a separate patch for this.
can do.
Thanks for your review!
\end{description}
N=1 if neither
VIRTIO_NET_F_MQnorVIRTIO_NET_F_RSSare negotiated, otherwise N is set by \field{max_virtqueue_pairs}.- controlq only exists if
VIRTIO_NET_F_CTRL_VQset.
+ controlq only exists ifVIRTIO_NET_F_CTRL_VQset.\\
+ transportq only exists ifVIRTIO_F_TRNSPT_VQis offered.\subsection{Feature bits}\label{sec:Device Types / Network Device / Feature bits}
content.tex +0
-0@@ -4399,10 +5014,12 @@ \subsection{Virtqueues}\label{sec:Device Types / Block Device / Virtqueues}\item[0] requestq1
\item[\ldots]
\item[N-1] requestqN
+\item[N] transportq
\end{description}N=1 if
VIRTIO_BLK_F_MQis not negotiated, otherwise N is set by
- \field{num_queues}.
+ \field{num_queues}.\\
+ transportq only exists ifVIRTIO_F_TRNSPT_VQis offered.\subsection{Feature bits}\label{sec:Device Types / Block Device / Feature bits}
introduction.tex +0
-0@@ -79,6 +79,9 @@ \section{Normative References}\label{sec:Normative References}\phantomsection\label{intro:SCMI}\textbf{[SCMI]} &
Arm System Control and Management Interface, DEN0056, \newline\url{https://developer.arm.com/docs/den0056/c}, version C and any future revisions\\
+ \phantomsection\laçbel{intro:SIOV}\textbf{[SIOV]} &
+ Scalable I/O Virtualization,
+ \newline\url{https://www.opencompute.org/documents/ocp-scalable-io-virtualization-technical-specification-revision-1-v1-2-pdf}\\\end{longtable} v
This publicly archived list offers a means to provide input to the
OASIS Virtual I/O Device (VIRTIO) TC.
In order to verify user consent to the Feedback License terms and
to minimize spam in the list archive, subscription is required
before posting.
Subscribe: [email protected]
Unsubscribe: [email protected]
List help: [email protected]
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Quoted 9 lines
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/
This publicly archived list offers a means to provide input to the
OASIS Virtual I/O Device (VIRTIO) TC.
In order to verify user consent to the Feedback License terms and
to minimize spam in the list archive, subscription is required
before posting.
Subscribe: [email protected]
Unsubscribe: [email protected]
List help: [email protected]
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Quoted 8 lines
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/
This publicly archived list offers a means to provide input to the OASIS Virtual I/O Device (VIRTIO) TC.
In order to verify user consent to the Feedback License terms and to minimize spam in the list archive, subscription is required before posting.
Subscribe: [email protected]
Unsubscribe: [email protected]
List help: [email protected]
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/
Quoted 6 lines · Lingshan
Quoted 6 lines
Quoted 6 lines
struct mgmt_dev_avail_res{}, if
min_vdev_vqs==max_vdev_vqs, means the management device only supports creating virtual devices with fixed capabilities, this kind of virtual devices can be pre-created. Like the slab allocator, vendors can pre-create the device. Then when the management device receiving theVDEV_CREATEcommand, it can just pick up a free virtual device, and return the virtual device.
Quoted 8 lines
Quoted 2 lines
So the question still remains, how could we discover those pre-created devices? (having something like
initial_vfs?)I think for the management devices, pre-creation could be pre-allocated
hw resource. The management device could pre-allocate a certain number of "standard" managed devices, when receive a create command, the management device can pick a free managed device, assign an UUID to it, then return
to the management driver. So the driver may not be aware of this pre-creation, it just sees the device is created rapidly. Just like the slab system, or a threads pool.
So for "pre-creating" I meant from the driver perspective. What you describe here is invisible to driver. So we don't need a dedicate feature in this case.
Btw, I think we need a command to support virtqueue reset in the next version.
Thanks
Quoted 14 lines
Quoted 6 lines
Quoted 6 lines
The bit only makes sense for the case when some managed(virtual) devices are pre-created like
initial_vfsin SR-IOV. If we allow pre-creation, we need commands to discover them.Quoted 7 lines
Quoted 6 lines
I think we need the feature bit
VIRTIO_F_TRNSPT_VQ_VDEV, this feature bit indicates that the transport virtqueue is the transport layer for the virtual devices, like PCI for a virtio PCI hardware. So, not only create, we use the transport virtqueue to config the device as well.And if the virtual devices are pre-created, we can just send the create command, and the management device can find a available device for us, this is also discovery.
This seems to conflict with the semantics of pre-created devices.
same to the above
Quoted 18 lines · Jason Wang
Quoted 6 lines · Lingshan
Quoted 6 lines
Quoted 6 lines
struct mgmt_dev_avail_res{}, if
min_vdev_vqs==max_vdev_vqs, means the management device only supports creating virtual devices with fixed capabilities, this kind of virtual devices can be pre-created. Like the slab allocator, vendors can pre-create the device. Then when the management device receiving theVDEV_CREATEcommand, it can just pick up a free virtual device, and return the virtual device.Quoted 8 lines
Quoted 2 lines
So the question still remains, how could we discover those pre-created devices? (having something like
initial_vfs?)I think for the management devices, pre-creation could be pre-allocated
hw resource. The management device could pre-allocate a certain number of "standard" managed devices, when receive a create command, the management device can pick a free managed device, assign an UUID to it, then return
to the management driver. So the driver may not be aware of this pre-creation, it just sees the device is created rapidly. Just like the slab system, or a threads pool.
Quoted 1 line
So for "pre-creating" I meant from the driver perspective. What you describe here is invisible to driver. So we don't need a dedicate feature in this case.
Yes, we should not describe "pre-create" here, I only talk about the managed devices with fixed num_queues. This is some kind of "pre-create" in the hw, pooling the source, not visible to the driver.
Quoted 2 lines
Btw, I think we need a command to support virtqueue reset in the next version.
Do you mean reset vq and re-enable vq of virtio 1.2?
I think we don't need to reset the MSI entry for the queue upon a reset, right? Because MSI entries are not queue configs and not meaningful to reset it. But we need to drop all pending MSI, right?
Thanks,
Zhu Lingshan
Quoted 19 lines
Thanks
Quoted 14 lines
Quoted 6 lines
Quoted 6 lines
The bit only makes sense for the case when some managed(virtual) devices are pre-created like
initial_vfsin SR-IOV. If we allow pre-creation, we need commands to discover them.Quoted 7 lines
Quoted 6 lines
I think we need the feature bit
VIRTIO_F_TRNSPT_VQ_VDEV, this feature bit indicates that the transport virtqueue is the transport layer for the virtual devices, like PCI for a virtio PCI hardware. So, not only create, we use the transport virtqueue to config the device as well.And if the virtual devices are pre-created, we can just send the create command, and the management device can find a available device for us, this is also discovery.
This seems to conflict with the semantics of pre-created devices.
same to the above
This publicly archived list offers a means to provide input to the OASIS Virtual I/O Device (VIRTIO) TC.
In order to verify user consent to the Feedback License terms and to minimize spam in the list archive, subscription is required before posting.
Subscribe: [email protected]
Unsubscribe: [email protected]
List help: [email protected]
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/
Quoted 33 lines · Lingshan
Quoted 26 lines · Jason Wang
Quoted 12 lines · Lingshan
Quoted 10 lines
Quoted 10 lines
struct mgmt_dev_avail_res{}, if
min_vdev_vqs==max_vdev_vqs, means the management device
only supports creating virtual devices with fixed capabilities, this kind of virtual devices
can be pre-created. Like the slab allocator, vendors can pre-create the device. Then when the management
device receiving theVDEV_CREATEcommand, it can just pick up a free virtual device, and return the
virtual device.Quoted 2 lines
So the question still remains, how could we discover those pre-created devices? (having something like
initial_vfs?)Quoted 10 lines
I think for the management devices, pre-creation could be pre-allocated hw resource. The management device could pre-allocate a certain number of
"standard" managed devices, when receive a create command, the management
device can pick a free managed device, assign an UUID to it, then return to the management driver. So the driver may not be aware of this pre-creation,
it just sees the device is created rapidly. Just like the slab system, or a threads pool.Quoted 3 lines
So for "pre-creating" I meant from the driver perspective. What you describe here is invisible to driver. So we don't need a dedicate feature in this case.
Quoted 8 lines
Yes, we should not describe "pre-create" here, I only talk about the managed devices with fixed
num_queues. This is some kind of "pre-create" in the hw,
pooling the source, not visible to the driver.Quoted 3 lines
Btw, I think we need a command to support virtqueue reset in the next version.
Do you mean reset vq and re-enable vq of virtio 1.2?
Yes.
Quoted 4 lines
I think we don't need to reset the MSI entry for the queue upon a reset, right?
Because MSI entries are not queue configs and not meaningful to reset it. But we need to drop all pending MSI, right?
I don't see any connection between vq reset and MSI. Vq reset is transport independent.
Thanks
Quoted 44 lines
Thanks,
Zhu LingshanQuoted 23 lines
Thanks
Quoted 18 lines
Quoted 13 lines
Quoted 13 lines
The bit only makes sense for the case when some managed(virtual) devices are pre-created like
initial_vfsin SR-IOV. If we allow pre-creation, we need commands to discover them.I think we need the feature bit
VIRTIO_F_TRNSPT_VQ_VDEV, this feature bit indicates that the transport
virtqueue is the transport layer for the virtual devices, like PCI for a virtio PCI hardware.
So, not only create, we use the transport virtqueue to config the device as well.Quoted 4 lines
Quoted 3 lines
And if the virtual devices are pre-created, we can just send the create command, and the management device
can find a available device for us, this is also discovery.This seems to conflict with the semantics of pre-created devices.
same to the above
This publicly archived list offers a means to provide input to the OASIS Virtual I/O Device (VIRTIO) TC.
In order to verify user consent to the Feedback License terms and to minimize spam in the list archive, subscription is required before posting.
Subscribe: [email protected]
Unsubscribe: [email protected]
List help: [email protected]
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/