add generic vDPA device support
v7 cover add generic vDPA device support
From: Longpeng <[email protected]>
Hi guys,
With the generic vDPA device, QEMU won't need to touch the device types any more, such like vfio.
We can use the generic vDPA device as follow:
-device vhost-vdpa-device-pci,vhostdev=/dev/vhost-vdpa-X Or
-M microvm -m 512m -smp 2 -kernel ... -initrd ... -device \
vhost-vdpa-device,vhostdev=/dev/vhost-vdpa-x
Changes v6 -> v7:
(v6: https://mail.gnu.org/archive/html/qemu-devel/2022-05/msg02821.html)
- rebase. [Jason]
- add documentation . [Stefan]
Changes v5 -> v6:
Patch 2:
- Turn to the original approach in the RFC to initialize the virtio_pci_id_info array. [Michael]
https://lore.kernel.org/all/[email protected]/
Patch 3:
- Fix logical error of exception handler around the post_init. [Stefano]
- Fix some coding style warnings. [Stefano]
Patch 4:
- Fix some coding style warnings. [Stefano]
Changes v4 -> v5:
Patch 3:
- remove vhostfd [Jason]
- support virtio-mmio [Jason]
Changes v3 -> v4:
v3: https://www.mail-archive.com/[email protected]/msg877015.html
- reorganize the series [Stefano]
- fix some typos [Stefano]
- fix logical error in vhost_vdpa_device_realize [Stefano]
Changes v2 -> v3
Patch 4 & 5:
- only call vdpa ioctls in vdpa-dev.c [Stefano, Longpeng]
- s/VQS_NUM/VQS_COUNT [Stefano]
- check both vdpa_dev_fd and vdpa_dev [Stefano]
Patch 6:
- move all steps into vhost_vdpa_device_unrealize. [Stefano]
Changes RFC -> v2
Patch 1:
- rename 'pdev_id' to 'trans_devid' [Michael]
- only use transitional device id for the devices listed in the spec [Michael]
- use macros to make the id_info table clearer [Longpeng]
- add some modern devices in the id_info table [Longpeng]
Patch 2:
- remove the GET_VECTORS_NUM command [Jason]
Patch 4:
- expose vdpa_dev_fd as a QOM preperty [Stefan]
- introduce vhost_vdpa_device_get_u32 as a common function to make the code clearer [Stefan]
- fix the misleading description of 'dc->desc' [Stefano]
Patch 5:
- check returned number of virtqueues [Stefan]
Patch 6:
- init s->num_queues [Stefano]
- free s->dev.vqs [Stefano]
Longpeng (Mike) (4):
virtio: get class_id and pci device id by the virtio id
vdpa: add vdpa-dev support
vdpa: add vdpa-dev-pci support
docs: Add generic vhost-vdpa device documentation
docs/system/devices/vhost-vdpa-device.rst | 43 +++ hw/virtio/Kconfig | 5 + hw/virtio/meson.build | 2 + hw/virtio/vdpa-dev-pci.c | 102 ++++++ hw/virtio/vdpa-dev.c | 377 ++++++++++++++++++++++ hw/virtio/virtio-pci.c | 88 +++++ include/hw/virtio/vdpa-dev.h | 43 +++ include/hw/virtio/virtio-pci.h | 5 + 8 files changed, 665 insertions(+) create mode 100644 docs/system/devices/vhost-vdpa-device.rst create mode 100644 hw/virtio/vdpa-dev-pci.c create mode 100644 hw/virtio/vdpa-dev.c create mode 100644 include/hw/virtio/vdpa-dev.h
Signature · 1 line
2.23.0
PATCH v7 1/4 virtio: get class_id and pci device id by the virtio id
From: Longpeng <[email protected]>
Add helpers to get the "Transitional PCI Device ID" and "class_id" of the device specified by the "Virtio Device ID".
These helpers will be used to build the generic vDPA device later.
--- hw/virtio/virtio-pci.c | 88 ++++++++++++++++++++++++++++++++++ include/hw/virtio/virtio-pci.h | 5 ++ 2 files changed, 93 insertions(+)
hw/virtio/virtio-pci.c +88 -0
@@ -19,6 +19,7 @@#include "exec/memop.h" #include "standard-headers/linux/virtio_pci.h" +#include "standard-headers/linux/virtio_ids.h" #include "hw/boards.h" #include "hw/virtio/virtio.h" #include "migration/qemu-file-types.h"
@@ -213,6 +214,90 @@ static int virtio_pci_load_queue(DeviceState *d, int n, QEMUFile *f)return 0 ; } +typedef struct VirtIOPCIIDInfo { +/* virtio id */ +uint16_t vdev_id; +/* pci device id for the transitional device */ +uint16_t trans_devid; +uint16_t class_id; +}VirtIOPCIIDInfo ; + +static const VirtIOPCIIDInfo virtio_pci_id_info[] = { + { + .vdev_id =VIRTIO_ID_CRYPTO , + .class_id =PCI_CLASS_OTHERS , + }, { + .vdev_id =VIRTIO_ID_FS , + .class_id =PCI_CLASS_STORAGE_OTHER , + }, { + .vdev_id =VIRTIO_ID_NET , + .trans_devid =PCI_DEVICE_ID_VIRTIO_NET , + .class_id =PCI_CLASS_NETWORK_ETHERNET , + }, { + .vdev_id =VIRTIO_ID_BLOCK , + .trans_devid =PCI_DEVICE_ID_VIRTIO_BLOCK , + .class_id =PCI_CLASS_STORAGE_SCSI , + }, { + .vdev_id =VIRTIO_ID_CONSOLE , + .trans_devid =PCI_DEVICE_ID_VIRTIO_CONSOLE , + .class_id =PCI_CLASS_COMMUNICATION_OTHER , + }, { + .vdev_id =VIRTIO_ID_SCSI , + .trans_devid =PCI_DEVICE_ID_VIRTIO_SCSI , + .class_id =PCI_CLASS_STORAGE_SCSI + }, { + .vdev_id =VIRTIO_ID_9P , + .trans_devid =PCI_DEVICE_ID_VIRTIO_9P , + .class_id =PCI_BASE_CLASS_NETWORK , + }, { + .vdev_id =VIRTIO_ID_BALLOON , + .trans_devid =PCI_DEVICE_ID_VIRTIO_BALLOON , + .class_id =PCI_CLASS_OTHERS , + }, { + .vdev_id =VIRTIO_ID_RNG , + .trans_devid =PCI_DEVICE_ID_VIRTIO_RNG , + .class_id =PCI_CLASS_OTHERS , + }, +}; + +static const VirtIOPCIIDInfo *virtio_pci_get_id_info (uint16_t vdev_id) +{ +const VirtIOPCIIDInfo *info =NULL ; +int i; + +for (i =0 ; i <ARRAY_SIZE (virtio_pci_id_info); i++) { +if (virtio_pci_id_info[i].vdev_id == vdev_id) { + info = &virtio_pci_id_info[i]; +break ; + } + } + +if (!info) { +/* The device id is invalid or not added to the id_info yet. */ +error_report ("Invalid virtio device(id %u)" , vdev_id); +abort (); + } + +return info; +} + +/* +* Get the Transitional Device ID for the specific device, return +* zero if the device is non-transitional. +*/ +uint16_t virtio_pci_get_trans_devid (uint16_t device_id) +{ +return virtio_pci_get_id_info (device_id)->trans_devid; +} + +/* +* Get the Class ID for the specific device. +*/ +uint16_t virtio_pci_get_class_id (uint16_t device_id) +{ +return virtio_pci_get_id_info (device_id)->class_id; +} +static bool virtio_pci_ioeventfd_enabled (DeviceState *d) {VirtIOPCIProxy *proxy =to_virtio_pci_proxy (d);
@@ -1683,6 +1768,9 @@ static void virtio_pci_device_plugged(DeviceState *d, Error **errp)* is set to PCI_SUBVENDOR_ID_REDHAT_QUMRANET by default. */ pci_set_word (config +PCI_SUBSYSTEM_ID ,virtio_bus_get_vdev_id (bus)); +if (proxy->trans_devid) { +pci_config_set_device_id (config, proxy->trans_devid); + } }else {/* pure virtio-1.0 */ pci_set_word(config + PCI_VENDOR_ID,
include/hw/virtio/virtio-pci.h +5 -0
@@ -146,6 +146,8 @@ struct VirtIOPCIProxy {bool disable_modern;bool ignore_backend_features;OnOffAuto disable_legacy; +/* Transitional device id */ +uint16_t trans_devid;uint32_t class_code;uint32_t nvectors;uint32_t dfselect;
@@ -179,6 +181,9 @@ static inline void virtio_pci_disable_modern(VirtIOPCIProxy *proxy) proxy->disable_modern = true; } +uint16_t virtio_pci_get_trans_devid (uint16_t device_id); +uint16_t virtio_pci_get_class_id (uint16_t device_id); +/* * virtio-input-pci: This extends VirtioPCIProxy. */
Signature · 1 line
2.23.0
PATCH v7 2/4 vdpa: add vdpa-dev support
In reply to Longpeng(Mike) viaFrom: Longpeng <[email protected]>
Supports vdpa-dev, we can use the deivce directly:
-M microvm -m 512m -smp 2 -kernel ... -initrd ... -device \
vhost-vdpa-device,vhostdev=/dev/vhost-vdpa-x
--- hw/virtio/Kconfig | 5 + hw/virtio/meson.build | 1 + hw/virtio/vdpa-dev.c | 377 +++++++++++++++++++++++++++++++++++ include/hw/virtio/vdpa-dev.h | 43 ++++ 4 files changed, 426 insertions(+) create mode 100644 hw/virtio/vdpa-dev.c create mode 100644 include/hw/virtio/vdpa-dev.h
hw/virtio/Kconfig +5 -0
@@ -85,3 +85,8 @@ config VHOST_USER_GPIO bool default y depends on VIRTIO && VHOST_USER + +config VHOST_VDPA_DEV + bool + default y + depends on VIRTIO && VHOST_VDPA && LINUX
hw/virtio/meson.build +1 -0
@@ -31,6 +31,7 @@ virtio_ss.add(when: 'CONFIG_VHOST_USER_I2C', if_true: files('vhost-user-i2c.c')) virtio_ss.add(when: 'CONFIG_VHOST_USER_RNG', if_true: files('vhost-user-rng.c')) virtio_ss.add(when: 'CONFIG_VHOST_USER_GPIO', if_true: files('vhost-user-gpio.c')) virtio_ss.add(when: ['CONFIG_VIRTIO_PCI', 'CONFIG_VHOST_USER_GPIO'], if_true: files('vhost-user-gpio-pci.c')) +virtio_ss.add(when: 'CONFIG_VHOST_VDPA_DEV', if_true: files('vdpa-dev.c')) virtio_pci_ss = ss.source_set() virtio_pci_ss.add(when: 'CONFIG_VHOST_VSOCK', if_true: files('vhost-vsock-pci.c'))
hw/virtio/vdpa-dev.c new file +377 -0
377 lines not on this page · raw message
include/hw/virtio/vdpa-dev.h new file +43 -0
43 lines not on this page · raw message
Signature · 1 line
2.23.0
PATCH v7 3/4 vdpa: add vdpa-dev-pci support
In reply to Longpeng(Mike) viaFrom: Longpeng <[email protected]>
Supports vdpa-dev-pci, we can use the device as follow:
-device vhost-vdpa-device-pci,vhostdev=/dev/vhost-vdpa-X
--- hw/virtio/meson.build | 1 + hw/virtio/vdpa-dev-pci.c | 102 +++++++++++++++++++++++++++++++++++++++ hw/virtio/vdpa-dev.c | 2 +- 3 files changed, 104 insertions(+), 1 deletion(-) create mode 100644 hw/virtio/vdpa-dev-pci.c
hw/virtio/meson.build +1 -0
@@ -57,6 +57,7 @@ virtio_pci_ss.add(when: 'CONFIG_VIRTIO_SERIAL', if_true: files('virtio-serial-pc virtio_pci_ss.add(when: 'CONFIG_VIRTIO_PMEM', if_true: files('virtio-pmem-pci.c')) virtio_pci_ss.add(when: 'CONFIG_VIRTIO_IOMMU', if_true: files('virtio-iommu-pci.c')) virtio_pci_ss.add(when: 'CONFIG_VIRTIO_MEM', if_true: files('virtio-mem-pci.c')) +virtio_pci_ss.add(when: 'CONFIG_VHOST_VDPA_DEV', if_true: files('vdpa-dev-pci.c')) virtio_ss.add_all(when: 'CONFIG_VIRTIO_PCI', if_true: virtio_pci_ss)
hw/virtio/vdpa-dev-pci.c new file +102 -0
102 lines not on this page · raw message
hw/virtio/vdpa-dev.c +1 -1
@@ -138,7 +138,7 @@ static void vhost_vdpa_device_realize(DeviceState *dev, Error **errp) goto free_config; } -virtio_init (vdev,"vhost-vdpa" , v->vdev_id, v->config_size); +virtio_init (vdev, v->vdev_id, v->config_size); v->virtqs = g_new0(VirtQueue *, v->dev.nvqs);for (i =0 ; i < v->dev.nvqs; i++) {
Signature · 1 line
2.23.0
PATCH v7 4/4 docs: Add generic vhost-vdpa device documentation
In reply to Longpeng(Mike) viaFrom: Longpeng <[email protected]>
--- docs/system/devices/vhost-vdpa-device.rst | 43 +++++++++++++++++++++++ 1 file changed, 43 insertions(+) create mode 100644 docs/system/devices/vhost-vdpa-device.rst
docs/system/devices/vhost-vdpa-device.rst new file +43 -0
@@ -0,0 +1,43 @@ + +========================= +generic vhost-vdpa device +========================= + +This document explains the usage of the generic vhost vdpa device. + +Description +----------- + +vDPA(virtio data path acceleration) device is a device that uses a datapath +which complies with the virtio specifications with vendor specific control +path. + +QEMU provides two types of vhost-vdpa devices to enable the vDPA device, one +is type sensitive which means QEMU needs to know the actual device type +(e.g. net, blk, scsi) and another is called "generic vdpa device" which is +type insensitive (likes vfio-pci). + +Examples +-------- + +Prepare the vhost-vdpa backends first: + +:: + host# ls -l /dev/vhost-vdpa-* + crw------- 1 root root 236, 0 Nov 2 00:49 /dev/vhost-vdpa-0 + +Start QEMU with virtio-mmio bus: + +:: + host# qemu-system \ + -M microvm -m 512 -smp 2 -kernel ... -initrd ... \ + -device vhost-vdpa-device,vhostdev=/dev/vhost-vdpa-0 \ + ... + +Start QEMU with virtio-pci bus: + +:: + host# qemu-system \ + -M pc -m 512 -smp 2 \ + -device vhost-vdpa-device-pci,vhostdev=/dev/vhost-vdpa-0 \ + ...
Signature · 1 line
2.23.0