add generic vDPA device support
v5 cover add generic vDPA device support
From: Longpeng <[email protected]>
Hi guys,
With the generic vDPA device, QEMU won't need to touch the device types any more, such like vfio.
We can use the generic vDPA device as follow:
-device vhost-vdpa-device-pci,vhostdev=/dev/vhost-vdpa-X Or
-M microvm -m 512m -smp 2 -kernel ... -initrd ... -device \
vhost-vdpa-device,vhostdev=/dev/vhost-vdpa-x
I've done some simple tests on Huawei's offloading card (net, 0.95).
Changes v4 -> v5:
Patch 3:
- remove vhostfd [Jason]
- support virtio-mmio [Jason]
Changes v3 -> v4:
v3: https://www.mail-archive.com/[email protected]/msg877015.html
- reorganize the series [Stefano]
- fix some typos [Stefano]
- fix logical error in vhost_vdpa_device_realize [Stefano]
Changes v2 -> v3
Patch 4 & 5:
- only call vdpa ioctls in vdpa-dev.c [Stefano, Longpeng]
- s/VQS_NUM/VQS_COUNT [Stefano]
- check both vdpa_dev_fd and vdpa_dev [Stefano]
Patch 6:
- move all steps into vhost_vdpa_device_unrealize. [Stefano]
Changes RFC -> v2
Patch 1:
- rename 'pdev_id' to 'trans_devid' [Michael]
- only use transitional device id for the devices listed in the spec [Michael]
- use macros to make the id_info table clearer [Longpeng]
- add some modern devices in the id_info table [Longpeng]
Patch 2:
- remove the GET_VECTORS_NUM command [Jason]
Patch 4:
- expose vdpa_dev_fd as a QOM preperty [Stefan]
- introduce vhost_vdpa_device_get_u32 as a common function to make the code clearer [Stefan]
- fix the misleading description of 'dc->desc' [Stefano]
Patch 5:
- check returned number of virtqueues [Stefan]
Patch 6:
- init s->num_queues [Stefano]
- free s->dev.vqs [Stefano]
Longpeng (Mike) (4):
linux-headers: Update headers to Linux 5.18-rc6
virtio: get class_id and pci device id by the virtio id
vdpa: add vdpa-dev support
vdpa: add vdpa-dev-pci support
hw/virtio/Kconfig | 5 + hw/virtio/meson.build | 2 + hw/virtio/vdpa-dev-pci.c | 101 ++++++++++ hw/virtio/vdpa-dev.c | 377 +++++++++++++++++++++++++++++++++++ hw/virtio/virtio-pci.c | 77 +++++++ hw/virtio/virtio-pci.h | 5 + include/hw/virtio/vdpa-dev.h | 43 ++++ linux-headers/linux/vhost.h | 7 + 8 files changed, 617 insertions(+) create mode 100644 hw/virtio/vdpa-dev-pci.c create mode 100644 hw/virtio/vdpa-dev.c create mode 100644 include/hw/virtio/vdpa-dev.h
Signature · 1 line
2.23.0
PATCH v5 1/4 linux-headers: Update headers to Linux 5.18-rc6
From: Longpeng <[email protected]>
Update headers to 5.18-rc6. I need latest vhost changes.
--- linux-headers/linux/vhost.h | 7 +++++++ 1 file changed, 7 insertions(+)
linux-headers/linux/vhost.h +7 -0
@@ -150,4 +150,11 @@/* Get the valid iova range */ #define VHOST_VDPA_GET_IOVA_RANGE _IOR(VHOST_VIRTIO, 0x78, \ struct vhost_vdpa_iova_range) + +/* Get the config size */ +#define VHOST_VDPA_GET_CONFIG_SIZE _IOR(VHOST_VIRTIO, 0x79, __u32) + +/* Get the count of all virtqueues */ +#define VHOST_VDPA_GET_VQS_COUNT _IOR(VHOST_VIRTIO, 0x80, __u32) +#endif
Signature · 1 line
2.23.0
PATCH v5 2/4 virtio: get class_id and pci device id by the virtio id
In reply to Longpeng(Mike) viaFrom: Longpeng <[email protected]>
Add helpers to get the "Transitional PCI Device ID" and "class_id" of the device specified by the "Virtio Device ID".
These helpers will be used to build the generic vDPA device later.
--- hw/virtio/virtio-pci.c | 77 ++++++++++++++++++++++++++++++++++++++++++ hw/virtio/virtio-pci.h | 5 +++ 2 files changed, 82 insertions(+)
hw/virtio/virtio-pci.c +77 -0
@@ -19,6 +19,7 @@#include "exec/memop.h" #include "standard-headers/linux/virtio_pci.h" +#include "standard-headers/linux/virtio_ids.h" #include "hw/boards.h" #include "hw/virtio/virtio.h" #include "migration/qemu-file-types.h"
@@ -212,6 +213,79 @@ static int virtio_pci_load_queue(DeviceState *d, int n, QEMUFile *f)return 0 ; } +typedef struct VirtIOPCIIDInfo { +/* virtio id */ +uint16_t vdev_id; +/* pci device id for the transitional device */ +uint16_t trans_devid; +uint16_t class_id; +}VirtIOPCIIDInfo ; + +#define VIRTIO_TRANS_DEV_ID_INFO (name, class) \ + { \ + .vdev_id = VIRTIO_ID_##name, \ + .trans_devid = PCI_DEVICE_ID_VIRTIO_##name, \ + .class_id = class, \ + } + +#define VIRTIO_MODERN_DEV_ID_NFO (name, class) \ + { \ + .vdev_id = VIRTIO_ID_##name, \ + .class_id = class, \ + } + +static const VirtIOPCIIDInfo virtio_pci_id_info[] = { +/* Non-transitional devices */ +VIRTIO_MODERN_DEV_ID_NFO (CRYPTO ,PCI_CLASS_OTHERS ), +VIRTIO_MODERN_DEV_ID_NFO (FS ,PCI_CLASS_STORAGE_OTHER ), +/* Transitional devices */ +VIRTIO_TRANS_DEV_ID_INFO (NET ,PCI_CLASS_NETWORK_ETHERNET ), +VIRTIO_TRANS_DEV_ID_INFO (BLOCK ,PCI_CLASS_STORAGE_SCSI ), +VIRTIO_TRANS_DEV_ID_INFO (CONSOLE ,PCI_CLASS_COMMUNICATION_OTHER ), +VIRTIO_TRANS_DEV_ID_INFO (SCSI ,PCI_CLASS_STORAGE_SCSI ), +VIRTIO_TRANS_DEV_ID_INFO (9P ,PCI_BASE_CLASS_NETWORK ), +VIRTIO_TRANS_DEV_ID_INFO (BALLOON ,PCI_CLASS_OTHERS ), +VIRTIO_TRANS_DEV_ID_INFO (RNG ,PCI_CLASS_OTHERS ), +}; + +static const VirtIOPCIIDInfo *virtio_pci_get_id_info (uint16_t vdev_id) +{ +const VirtIOPCIIDInfo *info =NULL ; +int i; + +for (i =0 ; i <ARRAY_SIZE (virtio_pci_id_info); i++) { +if (virtio_pci_id_info[i].vdev_id == vdev_id) { + info = &virtio_pci_id_info[i]; +break ; + } + } + +if (!info) { +/* The device id is invalid or not added to the id_info yet. */ +error_report ("Invalid virtio device(id %u)" , vdev_id); +abort (); + } + +return info; +} + +/* +* Get the Transitional Device ID for the specific device, return +* zero if the device is non-transitional. +*/ +uint16_t virtio_pci_get_trans_devid (uint16_t device_id) +{ +return virtio_pci_get_id_info (device_id)->trans_devid; +} + +/* +* Get the Class ID for the specific device. +*/ +uint16_t virtio_pci_get_class_id (uint16_t device_id) +{ +return virtio_pci_get_id_info (device_id)->class_id; +} +static bool virtio_pci_ioeventfd_enabled (DeviceState *d) {VirtIOPCIProxy *proxy =to_virtio_pci_proxy (d);
@@ -1675,6 +1749,9 @@ static void virtio_pci_device_plugged(DeviceState *d, Error **errp)* is set to PCI_SUBVENDOR_ID_REDHAT_QUMRANET by default. */ pci_set_word (config +PCI_SUBSYSTEM_ID ,virtio_bus_get_vdev_id (bus)); +if (proxy->trans_devid) { +pci_config_set_device_id (config, proxy->trans_devid); + } }else {/* pure virtio-1.0 */ pci_set_word(config + PCI_VENDOR_ID,
hw/virtio/virtio-pci.h +5 -0
@@ -146,6 +146,8 @@ struct VirtIOPCIProxy {bool disable_modern;bool ignore_backend_features;OnOffAuto disable_legacy; +/* Transitional device id */ +uint16_t trans_devid;uint32_t class_code;uint32_t nvectors;uint32_t dfselect;
@@ -158,6 +160,9 @@ struct VirtIOPCIProxy {VirtioBusState bus; }; +uint16_t virtio_pci_get_trans_devid (uint16_t device_id); +uint16_t virtio_pci_get_class_id (uint16_t device_id); +static inline bool virtio_pci_modern (VirtIOPCIProxy *proxy) {return !proxy->disable_modern;
Signature · 1 line
2.23.0
Re: [PATCH v5 2/4] virtio: get class_id and pci device id by the virtio id
Quoted 8 lines · Longpeng(Mike)
From: Longpeng <[email protected]>
Add helpers to get the "Transitional PCI Device ID" and "
class_id" of the device specified by the "Virtio Device ID".These helpers will be used to build the generic vDPA device later.
Signed-off-by Longpeng <[email protected]>
Except, the IDs in the current header a broken. I just fixed them and they will be hopefully OK in the next version.
Quoted 32 lines
--- hw/virtio/virtio-pci.c | 77 ++++++++++++++++++++++++++++++++++++++++++ hw/virtio/virtio-pci.h | 5 +++ 2 files changed, 82 insertions(+)hw/virtio/virtio-pci.c +12
-0@@ -19,6 +19,7 @@#include "exec/memop.h" #include "standard-headers/linux/virtio_pci.h" +#include "standard-headers/linux/virtio_ids.h" #include "hw/boards.h" #include "hw/virtio/virtio.h" #include "migration/qemu-file-types.h" @@ -212,6 +213,79 @@ static int virtio_pci_load_queue(DeviceState *d, int n, QEMUFile *f)return 0 ; } +typedef struct VirtIOPCIIDInfo { +/* virtio id */ +uint16_t vdev_id; +/* pci device id for the transitional device */ +uint16_t trans_devid; +uint16_t class_id; +}VirtIOPCIIDInfo ; + +#define VIRTIO_TRANS_DEV_ID_INFO (name, class) \ + { \ + .vdev_id = VIRTIO_ID_##name, \
hw/virtio/virtio-pci.c static int virtio_pci_load_queue(DeviceState *d, int n, QEMUFile *f) · line 236
Quoted 10 lines
hw/virtio/virtio-pci.c +10
-0+ .trans_devid = PCI_DEVICE_ID_VIRTIO_##name, \ + .class_id = class, \ + } + +#define VIRTIO_MODERN_DEV_ID_NFO (name, class) \ + { \ + .vdev_id = VIRTIO_ID_##name, \ + .class_id = class, \ + } +
No, I think I liked the original approach in the RFC better, even though it duplicates a tiny bit of code.
This trick does not save a lot of typing and obscures the ID use in case it's wrong (as was the case just recently).
Quoted 90 lines
hw/virtio/virtio-pci.c +55
-0+static const VirtIOPCIIDInfo virtio_pci_id_info[] = { +/* Non-transitional devices */ +VIRTIO_MODERN_DEV_ID_NFO (CRYPTO ,PCI_CLASS_OTHERS ), +VIRTIO_MODERN_DEV_ID_NFO (FS ,PCI_CLASS_STORAGE_OTHER ), +/* Transitional devices */ +VIRTIO_TRANS_DEV_ID_INFO (NET ,PCI_CLASS_NETWORK_ETHERNET ), +VIRTIO_TRANS_DEV_ID_INFO (BLOCK ,PCI_CLASS_STORAGE_SCSI ), +VIRTIO_TRANS_DEV_ID_INFO (CONSOLE ,PCI_CLASS_COMMUNICATION_OTHER ), +VIRTIO_TRANS_DEV_ID_INFO (SCSI ,PCI_CLASS_STORAGE_SCSI ), +VIRTIO_TRANS_DEV_ID_INFO (9P ,PCI_BASE_CLASS_NETWORK ), +VIRTIO_TRANS_DEV_ID_INFO (BALLOON ,PCI_CLASS_OTHERS ), +VIRTIO_TRANS_DEV_ID_INFO (RNG ,PCI_CLASS_OTHERS ), +}; + +static const VirtIOPCIIDInfo *virtio_pci_get_id_info (uint16_t vdev_id) +{ +const VirtIOPCIIDInfo *info =NULL ; +int i; + +for (i =0 ; i <ARRAY_SIZE (virtio_pci_id_info); i++) { +if (virtio_pci_id_info[i].vdev_id == vdev_id) { + info = &virtio_pci_id_info[i]; +break ; + } + } + +if (!info) { +/* The device id is invalid or not added to the id_info yet. */ +error_report ("Invalid virtio device(id %u)" , vdev_id); +abort (); + } + +return info; +} + +/* +* Get the Transitional Device ID for the specific device, return +* zero if the device is non-transitional. +*/ +uint16_t virtio_pci_get_trans_devid (uint16_t device_id) +{ +return virtio_pci_get_id_info (device_id)->trans_devid; +} + +/* +* Get the Class ID for the specific device. +*/ +uint16_t virtio_pci_get_class_id (uint16_t device_id) +{ +return virtio_pci_get_id_info (device_id)->class_id; +} +static bool virtio_pci_ioeventfd_enabled (DeviceState *d) {VirtIOPCIProxy *proxy =to_virtio_pci_proxy (d);@@ -1675,6 +1749,9 @@ static void virtio_pci_device_plugged(DeviceState *d, Error **errp)* is set to PCI_SUBVENDOR_ID_REDHAT_QUMRANET by default. */ pci_set_word (config +PCI_SUBSYSTEM_ID ,virtio_bus_get_vdev_id (bus)); +if (proxy->trans_devid) { +pci_config_set_device_id (config, proxy->trans_devid); + } }else {/* pure virtio-1.0 */ pci_set_word(config + PCI_VENDOR_ID,hw/virtio/virtio-pci.h +5
-0@@ -146,6 +146,8 @@ struct VirtIOPCIProxy {bool disable_modern;bool ignore_backend_features;OnOffAuto disable_legacy; +/* Transitional device id */ +uint16_t trans_devid;uint32_t class_code;uint32_t nvectors;uint32_t dfselect;@@ -158,6 +160,9 @@ struct VirtIOPCIProxy {VirtioBusState bus; }; +uint16_t virtio_pci_get_trans_devid (uint16_t device_id); +uint16_t virtio_pci_get_class_id (uint16_t device_id); +static inline bool virtio_pci_modern (VirtIOPCIProxy *proxy) {return !proxy->disable_modern;Signature · 1 line
2.23.0
Re: [PATCH v5 2/4] virtio: get class_id and pci device id by the virtio id
Quoted 6 lines · Michael S. Tsirkin
Quoted 5 lines · Longpeng(Mike)
From: Longpeng <[email protected]>
Add helpers to get the "Transitional PCI Device ID" and "
class_id" of the device specified by the "Virtio Device ID".
Quoted 8 lines
Quoted 3 lines
These helpers will be used to build the generic vDPA device later.
Signed-off-by Longpeng <[email protected]>Except, the IDs in the current header a broken. I just fixed them and they will be hopefully OK in the next version.
Do you mean this patch ?
```
virtio: fix virtio transitional ids
This commit fixes the transitional PCI device ID.
Fixes: d61914ea6ada ("virtio: update virtio id table, add transitional ids")
Signed-off-by: Shunsuke Mie <[email protected]>
Link: https://lore.kernel.org/r/[email protected]
Signed-off-by: Michael S. Tsirkin <[email protected]>
```Luckily, we use PCI_DEVICE_ID_VIRTIO_xxx instead of VIRTIO_TRANS_ID_xxx here.
Quoted 39 lines
Quoted 39 lines
--- hw/virtio/virtio-pci.c | 77 ++++++++++++++++++++++++++++++++++++++++++ hw/virtio/virtio-pci.h | 5 +++ 2 files changed, 82 insertions(+)hw/virtio/virtio-pci.c +19
-0@@ -19,6 +19,7 @@#include "exec/memop.h" #include "standard-headers/linux/virtio_pci.h" +#include "standard-headers/linux/virtio_ids.h" #include "hw/boards.h" #include "hw/virtio/virtio.h" #include "migration/qemu-file-types.h" @@ -212,6 +213,79 @@ static int virtio_pci_load_queue(DeviceState *d, int n, QEMUFile *f)return 0 ; } +typedef struct VirtIOPCIIDInfo { +/* virtio id */ +uint16_t vdev_id; +/* pci device id for the transitional device */ +uint16_t trans_devid; +uint16_t class_id; +}VirtIOPCIIDInfo ; + +#define VIRTIO_TRANS_DEV_ID_INFO (name, class) \ + { \ + .vdev_id = VIRTIO_ID_##name, \ + .trans_devid = PCI_DEVICE_ID_VIRTIO_##name, \ + .class_id = class, \ + } + +#define VIRTIO_MODERN_DEV_ID_NFO (name, class) \ + { \ + .vdev_id = VIRTIO_ID_##name, \
Quoted 9 lines
hw/virtio/virtio-pci.c static int virtio_pci_load_queue(DeviceState *d, int n, QEMUFile *f) · line 230
Quoted 3 lines
hw/virtio/virtio-pci.c +3
-0+ .class_id = class, \ + } +No, I think I liked the original approach in the RFC better, even though it duplicates a tiny bit of code.
This trick does not save a lot of typing and obscures the ID use in case it's wrong (as was the case just recently).
OK, will do in v6.
Thanks.
Quoted 94 lines
Quoted 91 lines
hw/virtio/virtio-pci.c +55
-0+static const VirtIOPCIIDInfo virtio_pci_id_info[] = { +/* Non-transitional devices */ +VIRTIO_MODERN_DEV_ID_NFO (CRYPTO ,PCI_CLASS_OTHERS ), +VIRTIO_MODERN_DEV_ID_NFO (FS ,PCI_CLASS_STORAGE_OTHER ), +/* Transitional devices */ +VIRTIO_TRANS_DEV_ID_INFO (NET ,PCI_CLASS_NETWORK_ETHERNET ), +VIRTIO_TRANS_DEV_ID_INFO (BLOCK ,PCI_CLASS_STORAGE_SCSI ), +VIRTIO_TRANS_DEV_ID_INFO (CONSOLE ,PCI_CLASS_COMMUNICATION_OTHER ), +VIRTIO_TRANS_DEV_ID_INFO (SCSI ,PCI_CLASS_STORAGE_SCSI ), +VIRTIO_TRANS_DEV_ID_INFO (9P ,PCI_BASE_CLASS_NETWORK ), +VIRTIO_TRANS_DEV_ID_INFO (BALLOON ,PCI_CLASS_OTHERS ), +VIRTIO_TRANS_DEV_ID_INFO (RNG ,PCI_CLASS_OTHERS ), +}; + +static const VirtIOPCIIDInfo *virtio_pci_get_id_info (uint16_t vdev_id) +{ +const VirtIOPCIIDInfo *info =NULL ; +int i; + +for (i =0 ; i <ARRAY_SIZE (virtio_pci_id_info); i++) { +if (virtio_pci_id_info[i].vdev_id == vdev_id) { + info = &virtio_pci_id_info[i]; +break ; + } + } + +if (!info) { +/* The device id is invalid or not added to the id_info yet. */ +error_report ("Invalid virtio device(id %u)" , vdev_id); +abort (); + } + +return info; +} + +/* +* Get the Transitional Device ID for the specific device, return +* zero if the device is non-transitional. +*/ +uint16_t virtio_pci_get_trans_devid (uint16_t device_id) +{ +return virtio_pci_get_id_info (device_id)->trans_devid; +} + +/* +* Get the Class ID for the specific device. +*/ +uint16_t virtio_pci_get_class_id (uint16_t device_id) +{ +return virtio_pci_get_id_info (device_id)->class_id; +} +static bool virtio_pci_ioeventfd_enabled (DeviceState *d) {VirtIOPCIProxy *proxy =to_virtio_pci_proxy (d);@@ -1675,6 +1749,9 @@ static void virtio_pci_device_plugged(DeviceState *d, Error **errp)* is set to PCI_SUBVENDOR_ID_REDHAT_QUMRANET by default. */ pci_set_word (config +PCI_SUBSYSTEM_ID ,virtio_bus_get_vdev_id (bus)); +if (proxy->trans_devid) { +pci_config_set_device_id (config, proxy->trans_devid); + } }else {/* pure virtio-1.0 */ pci_set_word(config + PCI_VENDOR_ID,hw/virtio/virtio-pci.h +5
-0@@ -146,6 +146,8 @@ struct VirtIOPCIProxy {bool disable_modern;bool ignore_backend_features;OnOffAuto disable_legacy; +/* Transitional device id */ +uint16_t trans_devid;uint32_t class_code;uint32_t nvectors;uint32_t dfselect;@@ -158,6 +160,9 @@ struct VirtIOPCIProxy {VirtioBusState bus; }; +uint16_t virtio_pci_get_trans_devid (uint16_t device_id); +uint16_t virtio_pci_get_class_id (uint16_t device_id); +static inline bool virtio_pci_modern (VirtIOPCIProxy *proxy) {return !proxy->disable_modern;Signature · 1 line
2.23.0.
PATCH v5 3/4 vdpa: add vdpa-dev support
In reply to Longpeng(Mike) viaFrom: Longpeng <[email protected]>
Supports vdpa-dev, we can use the deivce directly:
-M microvm -m 512m -smp 2 -kernel ... -initrd ... -device \
vhost-vdpa-device,vhostdev=/dev/vhost-vdpa-x
--- hw/virtio/Kconfig | 5 + hw/virtio/meson.build | 1 + hw/virtio/vdpa-dev.c | 377 +++++++++++++++++++++++++++++++++++ include/hw/virtio/vdpa-dev.h | 43 ++++ 4 files changed, 426 insertions(+) create mode 100644 hw/virtio/vdpa-dev.c create mode 100644 include/hw/virtio/vdpa-dev.h
hw/virtio/Kconfig +5 -0
@@ -68,3 +68,8 @@ config VHOST_USER_RNG bool default y depends on VIRTIO && VHOST_USER + +config VHOST_VDPA_DEV + bool + default y + depends on VIRTIO && VHOST_VDPA && LINUX
hw/virtio/meson.build +1 -0
@@ -29,6 +29,7 @@ virtio_ss.add(when: 'CONFIG_VHOST_USER_I2C', if_true: files('vhost-user-i2c.c')) virtio_ss.add(when: ['CONFIG_VIRTIO_PCI', 'CONFIG_VHOST_USER_I2C'], if_true: files('vhost-user-i2c-pci.c')) virtio_ss.add(when: 'CONFIG_VHOST_USER_RNG', if_true: files('vhost-user-rng.c')) virtio_ss.add(when: ['CONFIG_VHOST_USER_RNG', 'CONFIG_VIRTIO_PCI'], if_true: files('vhost-user-rng-pci.c')) +virtio_ss.add(when: 'CONFIG_VHOST_VDPA_DEV', if_true: files('vdpa-dev.c')) virtio_pci_ss = ss.source_set() virtio_pci_ss.add(when: 'CONFIG_VHOST_VSOCK', if_true: files('vhost-vsock-pci.c'))
hw/virtio/vdpa-dev.c new file +377 -0
377 lines not on this page · raw message
include/hw/virtio/vdpa-dev.h new file +43 -0
43 lines not on this page · raw message
Signature · 1 line
2.23.0
Re: [PATCH v5 3/4] vdpa: add vdpa-dev support
Quoted 162 lines · Longpeng(Mike)
From: Longpeng <[email protected]>
Supports vdpa-dev, we can use the deivce directly:
-M microvm -m 512m -smp 2 -kernel ... -initrd ... -device \
vhost-vdpa-device,vhostdev=/dev/vhost-vdpa-xSigned-off-by Longpeng <[email protected]>---
hw/virtio/Kconfig | 5 + hw/virtio/meson.build | 1 + hw/virtio/vdpa-dev.c | 377 +++++++++++++++++++++++++++++++++++ include/hw/virtio/vdpa-dev.h | 43 ++++ 4 files changed, 426 insertions(+)create mode 100644
hw/virtio/vdpa-dev.c
create mode 100644include/hw/virtio/vdpa-dev.hhw/virtio/Kconfig +5
-0@@ -68,3 +68,8 @@ config VHOST_USER_RNG bool default y depends on VIRTIO && VHOST_USER + +config VHOST_VDPA_DEV + bool + default y + depends on VIRTIO && VHOST_VDPA && LINUXhw/virtio/meson.build +1
-0@@ -29,6 +29,7 @@ virtio_ss.add(when: 'CONFIG_VHOST_USER_I2C', if_true: files('vhost-user-i2c.c')) virtio_ss.add(when: ['CONFIG_VIRTIO_PCI', 'CONFIG_VHOST_USER_I2C'], if_true: files('vhost-user-i2c-pci.c')) virtio_ss.add(when: 'CONFIG_VHOST_USER_RNG', if_true: files('vhost-user-rng.c')) virtio_ss.add(when: ['CONFIG_VHOST_USER_RNG', 'CONFIG_VIRTIO_PCI'], if_true: files('vhost-user-rng-pci.c')) +virtio_ss.add(when: 'CONFIG_VHOST_VDPA_DEV', if_true: files('vdpa-dev.c')) virtio_pci_ss = ss.source_set() virtio_pci_ss.add(when: 'CONFIG_VHOST_VSOCK', if_true: files('vhost-vsock-pci.c'))hw/virtio/vdpa-dev.c new file +114
-0@@ -0,0 +1,377 @@ +/* +* Vhost Vdpa Device +* +* Copyright (c) Huawei Technologies Co., Ltd. 2022. All Rights Reserved. +* +* Authors: +* Longpeng <[email protected]> +* +* Largely based on the "vhost-user-blk-pci.c" and "vhost-user-blk.c" implemented by: +* Changpeng Liu <[email protected]> +* +* This work is licensed under the terms of the GNU LGPL, version 2 or later. +* See the COPYING.LIB file in the top-level directory. +*/ +#include "qemu/osdep.h" +#include <sys/ioctl.h> +#include <linux/vhost.h> +#include "qapi/error.h" +#include "qemu/error-report.h" +#include "qemu/cutils.h" +#include "hw/qdev-core.h" +#include "hw/qdev-properties.h" +#include "hw/qdev-properties-system.h" +#include "hw/virtio/vhost.h" +#include "hw/virtio/virtio.h" +#include "hw/virtio/virtio-bus.h" +#include "hw/virtio/virtio-access.h" +#include "hw/virtio/vdpa-dev.h" +#include "sysemu/sysemu.h" +#include "sysemu/runstate.h" + +static void +vhost_vdpa_device_dummy_handle_output (VirtIODevice *vdev,VirtQueue *vq) +{ +/* Nothing to do */ +} + +static uint32_t +vhost_vdpa_device_get_u32 (int fd,unsigned long int cmd,Error **errp) +{ +uint32_t val = (uint32_t )-1 ; + +if (ioctl (fd, cmd, &val) <0 ) { +error_setg (errp,"vhost-vdpa-device: cmd 0x%lx failed: %s" , + cmd,strerror (errno)); + } + +return val; +} + +static void vhost_vdpa_device_realize (DeviceState *dev,Error **errp) +{ +VirtIODevice *vdev =VIRTIO_DEVICE (dev); +VhostVdpaDevice *v =VHOST_VDPA_DEVICE (vdev); +uint16_t max_queue_size; +struct vhost_virtqueue *vqs; +int i, ret; + +if (!v->vhostdev) { +error_setg (errp,"vhost-vdpa-device: vhostdev are missing" ); +return ; + } + + v->vhostfd =qemu_open (v->vhostdev,O_RDWR , errp); +if (*errp) { +return ; + } + v->vdpa.device_fd = v->vhostfd; + + v->vdev_id =vhost_vdpa_device_get_u32 (v->vhostfd, +VHOST_VDPA_GET_DEVICE_ID , errp); +if (*errp) { + goto out; + } + + max_queue_size =vhost_vdpa_device_get_u32 (v->vhostfd, +VHOST_VDPA_GET_VRING_NUM , errp); +if (*errp) { + goto out; + } + +if (v->queue_size > max_queue_size) { +error_setg (errp,"vhost-vdpa-device: invalid queue_size: %u (max:%u)" , + v->queue_size, max_queue_size); + goto out; + }else if (!v->queue_size) { + v->queue_size = max_queue_size; + } + + v->num_queues =vhost_vdpa_device_get_u32 (v->vhostfd, +VHOST_VDPA_GET_VQS_COUNT , errp); +if (*errp) { + goto out; + } + +if (!v->num_queues || v->num_queues >VIRTIO_QUEUE_MAX ) { +error_setg (errp,"invalid number of virtqueues: %u (max:%u)" , + v->num_queues,VIRTIO_QUEUE_MAX ); + goto out; + } + + v->dev.nvqs = v->num_queues; + vqs = g_new0(struct vhost_virtqueue, v->dev.nvqs); + v->dev.vqs = vqs; + v->dev.vq_index =0 ; + v->dev.vq_index_end = v->dev.nvqs; + v->dev.backend_features =0 ; + v->started = false; + + ret =vhost_dev_init (&v->dev, &v->vdpa,VHOST_BACKEND_TYPE_VDPA ,0 ,NULL ); +if (ret <0 ) { +error_setg (errp,"vhost-vdpa-device: vhost initialization failed: %s" , +strerror (-ret)); + goto free_vqs;
hw/virtio/vdpa-dev.c line 124
Quoted 10 lines
hw/virtio/vdpa-dev.c new file +10
-0+ } + + v->config_size =vhost_vdpa_device_get_u32 (v->vhostfd, +VHOST_VDPA_GET_CONFIG_SIZE , errp); +if (*errp) { + goto vhost_cleanup; + } + +if (v->post_init && v->post_init (v, errp) <0 ) { + goto free_virtio;
This goto seems wrong.
Either we use the vhost_cleanup label or we move this block just before return as it seems to me was in v3.
Out of curiosity, is there a reason for this change from v3 or was a simple mistake as a consequence of patch reorganization?
Since I think v6 is needed, checkpatch has some warnings for exceeding 80 characters, maybe some of them can be easily fixed.
The rest LGTM, I think we are in a good shape :-)
Thanks,
Stefano
Quoted 305 lines
hw/virtio/vdpa-dev.c new file +253
-0+ } + + v->config =g_malloc0 (v->config_size); + + ret =vhost_dev_get_config (&v->dev, v->config, v->config_size,NULL ); +if (ret <0 ) { +error_setg (errp,"vhost-vdpa-device: get config failed" ); + goto free_config; + } + +virtio_init (vdev,"vhost-vdpa" , v->vdev_id, v->config_size); + + v->virtqs = g_new0(VirtQueue *, v->dev.nvqs); +for (i =0 ; i < v->dev.nvqs; i++) { + v->virtqs[i] =virtio_add_queue (vdev, v->queue_size, + vhost_vdpa_device_dummy_handle_output); + } + +return ; + +free_virtio: +for (i =0 ; i < v->num_queues; i++) { +virtio_delete_queue (v->virtqs[i]); + } +g_free (v->virtqs); +virtio_cleanup (vdev); +free_config: +g_free (v->config); +vhost_cleanup: +vhost_dev_cleanup (&v->dev); +free_vqs: +g_free (vqs); +out: +qemu_close (v->vhostfd); + v->vhostfd =-1 ; +} + +static void vhost_vdpa_device_unrealize (DeviceState *dev) +{ +VirtIODevice *vdev =VIRTIO_DEVICE (dev); +VhostVdpaDevice *s =VHOST_VDPA_DEVICE (vdev); +int i; + +virtio_set_status (vdev,0 ); + +for (i =0 ; i < s->num_queues; i++) { +virtio_delete_queue (s->virtqs[i]); + } +g_free (s->virtqs); +virtio_cleanup (vdev); + +g_free (s->config); +g_free (s->dev.vqs); +vhost_dev_cleanup (&s->dev); +qemu_close (s->vhostfd); + s->vhostfd =-1 ; +} + +static void +vhost_vdpa_device_get_config (VirtIODevice *vdev,uint8_t *config) +{ +VhostVdpaDevice *s =VHOST_VDPA_DEVICE (vdev); + +memcpy (config, s->config, s->config_size); +} + +static void +vhost_vdpa_device_set_config (VirtIODevice *vdev,const uint8_t *config) +{ +VhostVdpaDevice *s =VHOST_VDPA_DEVICE (vdev); +int ret; + + ret =vhost_dev_set_config (&s->dev, s->config,0 , s->config_size, +VHOST_SET_CONFIG_TYPE_MASTER ); +if (ret) { +error_report ("set device config space failed" ); +return ; + } +} + +static uint64_t vhost_vdpa_device_get_features (VirtIODevice *vdev, +uint64_t features, +Error **errp) +{ +VhostVdpaDevice *s =VHOST_VDPA_DEVICE (vdev); +uint64_t backend_features = s->dev.features; + +if (!virtio_has_feature (features,VIRTIO_F_IOMMU_PLATFORM )) { +virtio_clear_feature (&backend_features,VIRTIO_F_IOMMU_PLATFORM ); + } + +return backend_features; +} + +static int vhost_vdpa_device_start (VirtIODevice *vdev,Error **errp) +{ +VhostVdpaDevice *s =VHOST_VDPA_DEVICE (vdev); +BusState *qbus =BUS (qdev_get_parent_bus (DEVICE (vdev))); +VirtioBusClass *k =VIRTIO_BUS_GET_CLASS (qbus); +int i, ret; + +if (!k->set_guest_notifiers) { +error_setg (errp,"binding does not support guest notifiers" ); +return -ENOSYS ; + } + + ret =vhost_dev_enable_notifiers (&s->dev, vdev); +if (ret <0 ) { +error_setg_errno (errp, -ret,"Error enabling host notifiers" ); +return ret; + } + + ret = k->set_guest_notifiers (qbus->parent, s->dev.nvqs, true); +if (ret <0 ) { +error_setg_errno (errp, -ret,"Error binding guest notifier" ); + goto err_host_notifiers; + } + + s->dev.acked_features = vdev->guest_features; + + ret =vhost_dev_start (&s->dev, vdev); +if (ret <0 ) { +error_setg_errno (errp, -ret,"Error starting vhost" ); + goto err_guest_notifiers; + } + s->started = true; + +/* +* guest_notifier_mask/pending not used yet, so just unmask +* everything here. virtio-pci will do the right thing by +* enabling/disabling irqfd. +*/ +for (i =0 ; i < s->dev.nvqs; i++) { +vhost_virtqueue_mask (&s->dev, vdev, i, false); + } + +return ret; + +err_guest_notifiers: + k->set_guest_notifiers (qbus->parent, s->dev.nvqs, false); +err_host_notifiers: +vhost_dev_disable_notifiers (&s->dev, vdev); +return ret; +} + +static void vhost_vdpa_device_stop (VirtIODevice *vdev) +{ +VhostVdpaDevice *s =VHOST_VDPA_DEVICE (vdev); +BusState *qbus =BUS (qdev_get_parent_bus (DEVICE (vdev))); +VirtioBusClass *k =VIRTIO_BUS_GET_CLASS (qbus); +int ret; + +if (!s->started) { +return ; + } + s->started = false; + +if (!k->set_guest_notifiers) { +return ; + } + +vhost_dev_stop (&s->dev, vdev); + + ret = k->set_guest_notifiers (qbus->parent, s->dev.nvqs, false); +if (ret <0 ) { +error_report ("vhost guest notifier cleanup failed: %d" , ret); +return ; + } + +vhost_dev_disable_notifiers (&s->dev, vdev); +} + +static void vhost_vdpa_device_set_status (VirtIODevice *vdev,uint8_t status) +{ +VhostVdpaDevice *s =VHOST_VDPA_DEVICE (vdev); +bool should_start =virtio_device_started (vdev, status); +Error *local_err =NULL ; +int ret; + +if (!vdev->vm_running) { + should_start = false; + } + +if (s->started == should_start) { +return ; + } + +if (should_start) { + ret =vhost_vdpa_device_start (vdev, &local_err); +if (ret <0 ) { +error_reportf_err (local_err,"vhost-vdpa-device: start failed: " ); + } + }else { +vhost_vdpa_device_stop (vdev); + } +} + +static Property vhost_vdpa_device_properties[] = { +DEFINE_PROP_STRING ("vhostdev" , VhostVdpaDevice, vhostdev), +DEFINE_PROP_UINT16 ("queue-size" , VhostVdpaDevice, queue_size,0 ), +DEFINE_PROP_END_OF_LIST (), +}; + +static const VMStateDescription vmstate_vhost_vdpa_device = { + .name ="vhost-vdpa-device" , + .unmigratable =1 , + .minimum_version_id =1 , + .version_id =1 , + .fields = (VMStateField []) { +VMSTATE_VIRTIO_DEVICE , +VMSTATE_END_OF_LIST () + }, +}; + +static void vhost_vdpa_device_class_init (ObjectClass *klass,void *data) +{ +DeviceClass *dc =DEVICE_CLASS (klass); +VirtioDeviceClass *vdc =VIRTIO_DEVICE_CLASS (klass); + +device_class_set_props (dc, vhost_vdpa_device_properties); + dc->desc ="VDPA-based generic device assignment" ; + dc->vmsd = &vmstate_vhost_vdpa_device; +set_bit (DEVICE_CATEGORY_MISC , dc->categories); + vdc->realize = vhost_vdpa_device_realize; + vdc->unrealize = vhost_vdpa_device_unrealize; + vdc->get_config = vhost_vdpa_device_get_config; + vdc->set_config = vhost_vdpa_device_set_config; + vdc->get_features = vhost_vdpa_device_get_features; + vdc->set_status = vhost_vdpa_device_set_status; +} + +static void vhost_vdpa_device_instance_init (Object *obj) +{ +VhostVdpaDevice *s =VHOST_VDPA_DEVICE (obj); + +device_add_bootindex_property (obj, &s->bootindex,"bootindex" , +NULL ,DEVICE (obj)); +} + +static const TypeInfo vhost_vdpa_device_info = { + .name =TYPE_VHOST_VDPA_DEVICE , + .parent =TYPE_VIRTIO_DEVICE , + .instance_size =sizeof (VhostVdpaDevice), + .class_init = vhost_vdpa_device_class_init, + .instance_init = vhost_vdpa_device_instance_init, +}; + +static void register_vhost_vdpa_device_type (void ) +{ +type_register_static (&vhost_vdpa_device_info); +} + +type_init (register_vhost_vdpa_device_type);include/hw/virtio/vdpa-dev.h new file +43
-0@@ -0,0 +1,43 @@ +/* +* Vhost Vdpa Device +* +* Copyright (c) Huawei Technologies Co., Ltd. 2022. All Rights Reserved. +* +* Authors: +* Longpeng <[email protected]> +* +* Largely based on the "vhost-user-blk.h" implemented by: +* Changpeng Liu <[email protected]> +* +* This work is licensed under the terms of the GNU LGPL, version 2 or later. +* See the COPYING.LIB file in the top-level directory. +*/ +#ifndef _VHOST_VDPA_DEVICE_H +#define _VHOST_VDPA_DEVICE_H + +#include "hw/virtio/vhost.h" +#include "hw/virtio/vhost-vdpa.h" +#include "qom/object.h" + + +#define TYPE_VHOST_VDPA_DEVICE "vhost-vdpa-device" +OBJECT_DECLARE_SIMPLE_TYPE(VhostVdpaDevice, VHOST_VDPA_DEVICE) + +struct VhostVdpaDevice { +VirtIODevice parent_obj; +char *vhostdev; +int vhostfd; +int32_t bootindex; +uint32_t vdev_id; +uint32_t num_queues; +struct vhost_dev dev; +struct vhost_vdpa vdpa; +VirtQueue **virtqs; +uint8_t *config; +int config_size; +uint16_t queue_size; +bool started; +int (*post_init)(VhostVdpaDevice *v,Error **errp); +}; + +#endif Signature · 1 line
2.23.0
Re: [PATCH v5 3/4] vdpa: add vdpa-dev support
Quoted 188 lines · Stefano Garzarella
Quoted 177 lines · Longpeng(Mike)
From: Longpeng <[email protected]>
Supports vdpa-dev, we can use the deivce directly:
-M microvm -m 512m -smp 2 -kernel ... -initrd ... -device \
vhost-vdpa-device,vhostdev=/dev/vhost-vdpa-xSigned-off-by Longpeng <[email protected]>---
hw/virtio/Kconfig | 5 +
hw/virtio/meson.build | 1 +
hw/virtio/vdpa-dev.c | 377 +++++++++++++++++++++++++++++++++++include/hw/virtio/vdpa-dev.h| 43 ++++4 files changed, 426 insertions(+)create mode 100644
hw/virtio/vdpa-dev.c
create mode 100644include/hw/virtio/vdpa-dev.hhw/virtio/Kconfig +0
-0@@ -68,3 +68,8 @@ config VHOST_USER_RNGbool
default y
depends on VIRTIO &&VHOST_USERhw/virtio/Kconfig +5
-0+ +config VHOST_VDPA_DEV + bool + default y + depends on VIRTIO && VHOST_VDPA && LINUXhw/virtio/meson.build +1
-0@@ -29,6 +29,7 @@ virtio_ss.add(when: 'CONFIG_VHOST_USER_I2C', if_true: files('vhost-user-i2c.c')) virtio_ss.add(when: ['CONFIG_VIRTIO_PCI', 'CONFIG_VHOST_USER_I2C'], if_true: files('vhost-user-i2c-pci.c')) virtio_ss.add(when: 'CONFIG_VHOST_USER_RNG', if_true: files('vhost-user-rng.c')) virtio_ss.add(when: ['CONFIG_VHOST_USER_RNG', 'CONFIG_VIRTIO_PCI'], if_true: files('vhost-user-rng-pci.c')) +virtio_ss.add(when: 'CONFIG_VHOST_VDPA_DEV', if_true: files('vdpa-dev.c'))
virtio_pci_ss=ss.source_set()virtio_pci_ss.add(when: 'CONFIG_VHOST_VSOCK', if_true: files('vhost-vsock-pci.c'))hw/virtio/vdpa-dev.c new file +4
-0@@ -0,0 +1,377 @@ +/* +* Vhost Vdpa Device +* +* Copyright (c) Huawei Technologies Co., Ltd. 2022. All Rights Reserved.
+ *
+ * Authors:
+ * Longpeng <[email protected]>
+ *
+ * Largely based on the "vhost-user-blk-pci.c" and "vhost-user-blk.c" implemented by:
+ * Changpeng Liu <[email protected]>
+ *
+ * This work is licensed under the terms of the GNU LGPL, version 2 or later.
+ * See the COPYING.LIB file in the top-level directory.
+ */hw/virtio/vdpa-dev.c new file +57
-0+#include "qemu/osdep.h" +#include <sys/ioctl.h> +#include <linux/vhost.h> +#include "qapi/error.h" +#include "qemu/error-report.h" +#include "qemu/cutils.h" +#include "hw/qdev-core.h" +#include "hw/qdev-properties.h" +#include "hw/qdev-properties-system.h" +#include "hw/virtio/vhost.h" +#include "hw/virtio/virtio.h" +#include "hw/virtio/virtio-bus.h" +#include "hw/virtio/virtio-access.h" +#include "hw/virtio/vdpa-dev.h" +#include "sysemu/sysemu.h" +#include "sysemu/runstate.h" + +static void +vhost_vdpa_device_dummy_handle_output (VirtIODevice *vdev,VirtQueue *vq) +{ + /* Nothing to do */ +} + +static uint32_t +vhost_vdpa_device_get_u32 (int fd,unsigned long int cmd,Error **errp) +{ + uint32_t val = (uint32_t)-1; + + if (ioctl(fd, cmd, &val) < 0) { + error_setg(errp, "vhost-vdpa-device: cmd 0x%lx failed: %s", + cmd, strerror(errno)); + } + + return val; +} + +static void vhost_vdpa_device_realize (DeviceState *dev,Error **errp) +{ + VirtIODevice *vdev = VIRTIO_DEVICE(dev); + VhostVdpaDevice *v = VHOST_VDPA_DEVICE(vdev); + uint16_t max_queue_size; + struct vhost_virtqueue *vqs; + int i, ret; + + if (!v->vhostdev) { + error_setg(errp, "vhost-vdpa-device: vhostdev are missing"); + return; + } + + v->vhostfd = qemu_open(v->vhostdev, O_RDWR, errp); + if (*errp) { + return; + } + v->vdpa.device_fd = v->vhostfd; + + v->vdev_id = vhost_vdpa_device_get_u32(v->vhostfd, + VHOST_VDPA_GET_DEVICE_ID,errp);
+ if (*errp) {
+ goto out;
+ }hw/virtio/vdpa-dev.c new file +3
-0+ + max_queue_size = vhost_vdpa_device_get_u32(v->vhostfd, +
VHOST_VDPA_GET_VRING_NUM, errp);
+ if (*errp) {
+ goto out;
+ }
+
+ if (v->queue_size>max_queue_size) {
+ error_setg(errp, "vhost-vdpa-device: invalidqueue_size: %u (max:%u)",
+v->queue_size,max_queue_size);
+ goto out;
+ } else if (!v->queue_size) {
+v->queue_size=max_queue_size;
+ }hw/virtio/vdpa-dev.c new file +3
-0+ + v->num_queues = vhost_vdpa_device_get_u32(v->vhostfd, +
VHOST_VDPA_GET_VQS_COUNT, errp);
+ if (*errp) {
+ goto out;
+ }hw/virtio/vdpa-dev.c new file +16
-0+ + if (!v->num_queues || v->num_queues > VIRTIO_QUEUE_MAX) { + error_setg(errp, "invalid number of virtqueues: %u (max:%u)", + v->num_queues, VIRTIO_QUEUE_MAX); + goto out; + } + + v->dev.nvqs = v->num_queues; + vqs = g_new0(struct vhost_virtqueue, v->dev.nvqs); + v->dev.vqs = vqs; + v->dev.vq_index = 0; + v->dev.vq_index_end = v->dev.nvqs; + v->dev.backend_features = 0; + v->started = false; + + ret = vhost_dev_init(&v->dev, &v->vdpa, VHOST_BACKEND_TYPE_VDPA,0, NULL);
+ if (ret < 0) {
+ error_setg(errp, "vhost-vdpa-device: vhost initialization
failed: %s",
+ strerror(-ret));
+ gotofree_vqs;
+ }hw/virtio/vdpa-dev.c new file +3
-0+ + v->config_size = vhost_vdpa_device_get_u32(v->vhostfd, +Quoted 7 lines
VHOST_VDPA_GET_CONFIG_SIZE, errp);
+ if (*errp) {
+ gotovhost_cleanup;
+ }
+
+ if (v->post_init&&v->post_init(v, errp)< 0) { + gotofree_virtio;This goto seems wrong.
Quoted 2 lines
Either we use the
vhost_cleanuplabel or we move this block just before return as it seems to me was in v3.
Good catch. We should use 'vhost_cleanup' label here.
Quoted 2 lines
Out of curiosity, is there a reason for this change from v3 or was a simple mistake as a consequence of patch reorganization?
We should invoke .post_init() before calling virtio_init() which would use the fields of VirtIOPCIProxy.
Quoted 2 lines
Since I think v6 is needed, checkpatch has some warnings for exceeding 80 characters, maybe some of them can be easily fixed.
OK, I'll fix them.
Quoted 319 lines
The rest LGTM, I think we are in a good shape :-)
Thanks,
StefanoQuoted 312 lines
+ }
hw/virtio/vdpa-dev.c new file +4
-0+ + v->config = g_malloc0(v->config_size); + + ret = vhost_dev_get_config(&v->dev, v->config, v->config_size,NULL);
+ if (ret < 0) {
+error_setg(errp, "vhost-vdpa-device: get config failed"); + gotofree_config;
+ }hw/virtio/vdpa-dev.c new file +7
-0+ + virtio_init(vdev, "vhost-vdpa", v->vdev_id, v->config_size); + + v->virtqs = g_new0(VirtQueue *, v->dev.nvqs); + for (i = 0; i < v->dev.nvqs; i++) { + v->virtqs[i] = virtio_add_queue(vdev, v->queue_size, +
vhost_vdpa_device_dummy_handle_output);
+ }hw/virtio/vdpa-dev.c new file +72
-0+ + return; + +free_virtio: + for (i = 0; i < v->num_queues; i++) { + virtio_delete_queue(v->virtqs[i]); + } + g_free(v->virtqs); + virtio_cleanup(vdev); +free_config: + g_free(v->config); +vhost_cleanup: + vhost_dev_cleanup(&v->dev); +free_vqs: + g_free(vqs); +out: + qemu_close(v->vhostfd); + v->vhostfd = -1; +} + +static void vhost_vdpa_device_unrealize (DeviceState *dev) +{ + VirtIODevice *vdev = VIRTIO_DEVICE(dev); + VhostVdpaDevice *s = VHOST_VDPA_DEVICE(vdev); + int i; + + virtio_set_status(vdev, 0); + + for (i = 0; i < s->num_queues; i++) { + virtio_delete_queue(s->virtqs[i]); + } + g_free(s->virtqs); + virtio_cleanup(vdev); + + g_free(s->config); + g_free(s->dev.vqs); + vhost_dev_cleanup(&s->dev); + qemu_close(s->vhostfd); + s->vhostfd = -1; +} + +static void +vhost_vdpa_device_get_config (VirtIODevice *vdev,uint8_t *config) +{ + VhostVdpaDevice *s = VHOST_VDPA_DEVICE(vdev); + + memcpy(config, s->config, s->config_size); +} + +static void +vhost_vdpa_device_set_config (VirtIODevice *vdev,const uint8_t *config) +{ + VhostVdpaDevice *s = VHOST_VDPA_DEVICE(vdev); + int ret; + + ret = vhost_dev_set_config(&s->dev, s->config, 0, s->config_size, + VHOST_SET_CONFIG_TYPE_MASTER); + if (ret) { + error_report("set device config space failed"); + return; + } +} + +static uint64_t vhost_vdpa_device_get_features (VirtIODevice *vdev, + uint64_t features, + Error **errp) +{ + VhostVdpaDevice *s = VHOST_VDPA_DEVICE(vdev); + uint64_t backend_features = s->dev.features; + + if (!virtio_has_feature(features, VIRTIO_F_IOMMU_PLATFORM)) { + virtio_clear_feature(&backend_features,
VIRTIO_F_IOMMU_PLATFORM);
+ }hw/virtio/vdpa-dev.c new file +83
-0+ + return backend_features; +} + +static int vhost_vdpa_device_start (VirtIODevice *vdev,Error **errp) +{ + VhostVdpaDevice *s = VHOST_VDPA_DEVICE(vdev); + BusState *qbus = BUS(qdev_get_parent_bus(DEVICE(vdev))); + VirtioBusClass *k = VIRTIO_BUS_GET_CLASS(qbus); + int i, ret; + + if (!k->set_guest_notifiers) { + error_setg(errp, "binding does not support guest notifiers"); + return -ENOSYS; + } + + ret = vhost_dev_enable_notifiers(&s->dev, vdev); + if (ret < 0) { + error_setg_errno(errp, -ret, "Error enabling host notifiers"); + return ret; + } + + ret = k->set_guest_notifiers(qbus->parent, s->dev.nvqs, true); + if (ret < 0) { + error_setg_errno(errp, -ret, "Error binding guest notifier"); + goto err_host_notifiers; + } + + s->dev.acked_features = vdev->guest_features; + + ret = vhost_dev_start(&s->dev, vdev); + if (ret < 0) { + error_setg_errno(errp, -ret, "Error starting vhost"); + goto err_guest_notifiers; + } + s->started = true; + + /* + * guest_notifier_mask/pending not used yet, so just unmask + * everything here. virtio-pci will do the right thing by + * enabling/disabling irqfd. + */ + for (i = 0; i < s->dev.nvqs; i++) { + vhost_virtqueue_mask(&s->dev, vdev, i, false); + } + + return ret; + +err_guest_notifiers: + k->set_guest_notifiers(qbus->parent, s->dev.nvqs, false); +err_host_notifiers: + vhost_dev_disable_notifiers(&s->dev, vdev); + return ret; +} + +static void vhost_vdpa_device_stop (VirtIODevice *vdev) +{ + VhostVdpaDevice *s = VHOST_VDPA_DEVICE(vdev); + BusState *qbus = BUS(qdev_get_parent_bus(DEVICE(vdev))); + VirtioBusClass *k = VIRTIO_BUS_GET_CLASS(qbus); + int ret; + + if (!s->started) { + return; + } + s->started = false; + + if (!k->set_guest_notifiers) { + return; + } + + vhost_dev_stop(&s->dev, vdev); + + ret = k->set_guest_notifiers(qbus->parent, s->dev.nvqs, false); + if (ret < 0) { + error_report("vhost guest notifier cleanup failed: %d", ret); + return; + } + + vhost_dev_disable_notifiers(&s->dev, vdev); +} + +static void vhost_vdpa_device_set_status(VirtIODevice *vdev, uint8_tstatus)
hw/virtio/vdpa-dev.c new file +18
-0+{ + VhostVdpaDevice *s = VHOST_VDPA_DEVICE(vdev); + bool should_start = virtio_device_started(vdev, status); + Error *local_err = NULL; + int ret; + + if (!vdev->vm_running) { + should_start = false; + } + + if (s->started == should_start) { + return; + } + + if (should_start) { + ret = vhost_vdpa_device_start(vdev, &local_err); + if (ret < 0) { + error_reportf_err(local_err, "vhost-vdpa-device: startfailed: ");
+ }
+ } else {
+vhost_vdpa_device_stop(vdev);
+ }hw/virtio/vdpa-dev.c new file +58
-0+} + +static Property vhost_vdpa_device_properties[] = { + DEFINE_PROP_STRING("vhostdev", VhostVdpaDevice, vhostdev), + DEFINE_PROP_UINT16("queue-size", VhostVdpaDevice, queue_size, 0), + DEFINE_PROP_END_OF_LIST(), +}; + +static const VMStateDescription vmstate_vhost_vdpa_device = { + .name = "vhost-vdpa-device", + .unmigratable = 1, + .minimum_version_id = 1, + .version_id = 1, + .fields = (VMStateField[]) { + VMSTATE_VIRTIO_DEVICE, + VMSTATE_END_OF_LIST() + }, +}; + +static void vhost_vdpa_device_class_init (ObjectClass *klass,void *data) +{ + DeviceClass *dc = DEVICE_CLASS(klass); + VirtioDeviceClass *vdc = VIRTIO_DEVICE_CLASS(klass); + + device_class_set_props(dc, vhost_vdpa_device_properties); + dc->desc = "VDPA-based generic device assignment"; + dc->vmsd = &vmstate_vhost_vdpa_device; + set_bit(DEVICE_CATEGORY_MISC, dc->categories); + vdc->realize = vhost_vdpa_device_realize; + vdc->unrealize = vhost_vdpa_device_unrealize; + vdc->get_config = vhost_vdpa_device_get_config; + vdc->set_config = vhost_vdpa_device_set_config; + vdc->get_features = vhost_vdpa_device_get_features; + vdc->set_status = vhost_vdpa_device_set_status; +} + +static void vhost_vdpa_device_instance_init (Object *obj) +{ + VhostVdpaDevice *s = VHOST_VDPA_DEVICE(obj); + + device_add_bootindex_property(obj, &s->bootindex, "bootindex", + NULL, DEVICE(obj)); +} + +static const TypeInfo vhost_vdpa_device_info = { + .name = TYPE_VHOST_VDPA_DEVICE, + .parent = TYPE_VIRTIO_DEVICE, + .instance_size = sizeof(VhostVdpaDevice), + .class_init = vhost_vdpa_device_class_init, + .instance_init = vhost_vdpa_device_instance_init, +}; + +static void register_vhost_vdpa_device_type (void ) +{ + type_register_static(&vhost_vdpa_device_info); +} + +type_init (register_vhost_vdpa_device_type);include/hw/virtio/vdpa-dev.h new file +4
-0@@ -0,0 +1,43 @@ +/* +* Vhost Vdpa Device +* +* Copyright (c) Huawei Technologies Co., Ltd. 2022. All Rights Reserved.
+ *
+ * Authors:
+ * Longpeng <[email protected]>
+ *
+ * Largely based on the "vhost-user-blk.h" implemented by:
+ * Changpeng Liu <[email protected]>
+ *
+ * This work is licensed under the terms of the GNU LGPL, version 2 or later.
+ * See the COPYING.LIB file in the top-level directory.
+ */include/hw/virtio/vdpa-dev.h new file +29
-0+#ifndef _VHOST_VDPA_DEVICE_H +#define _VHOST_VDPA_DEVICE_H + +#include "hw/virtio/vhost.h" +#include "hw/virtio/vhost-vdpa.h" +#include "qom/object.h" + + +#define TYPE_VHOST_VDPA_DEVICE "vhost-vdpa-device" +OBJECT_DECLARE_SIMPLE_TYPE(VhostVdpaDevice, VHOST_VDPA_DEVICE) + +struct VhostVdpaDevice { + VirtIODevice parent_obj; + char *vhostdev; + int vhostfd; + int32_t bootindex; + uint32_t vdev_id; + uint32_t num_queues; + struct vhost_dev dev; + struct vhost_vdpa vdpa; + VirtQueue **virtqs; + uint8_t *config; + int config_size; + uint16_t queue_size; + bool started; + int (*post_init)(VhostVdpaDevice *v, Error **errp); +}; + +#endif Signature · 1 line
2.23.0.
Re: [PATCH v5 3/4] vdpa: add vdpa-dev support
Quoted 191 lines · Cloud Infrastructure Service Product Dept.)
Quoted 188 lines · Stefano Garzarella
Quoted 177 lines · Longpeng(Mike)
From: Longpeng <[email protected]>
Supports vdpa-dev, we can use the deivce directly:
-M microvm -m 512m -smp 2 -kernel ... -initrd ... -device \
vhost-vdpa-device,vhostdev=/dev/vhost-vdpa-xSigned-off-by Longpeng <[email protected]>---
hw/virtio/Kconfig | 5 +
hw/virtio/meson.build | 1 +
hw/virtio/vdpa-dev.c | 377 +++++++++++++++++++++++++++++++++++include/hw/virtio/vdpa-dev.h| 43 ++++4 files changed, 426 insertions(+)create mode 100644
hw/virtio/vdpa-dev.c
create mode 100644include/hw/virtio/vdpa-dev.hhw/virtio/Kconfig +0
-0@@ -68,3 +68,8 @@ config VHOST_USER_RNGbool
default y
depends on VIRTIO &&VHOST_USERhw/virtio/Kconfig +5
-0+ +config VHOST_VDPA_DEV + bool + default y + depends on VIRTIO && VHOST_VDPA && LINUXhw/virtio/meson.build +1
-0@@ -29,6 +29,7 @@ virtio_ss.add(when: 'CONFIG_VHOST_USER_I2C', if_true: files('vhost-user-i2c.c')) virtio_ss.add(when: ['CONFIG_VIRTIO_PCI', 'CONFIG_VHOST_USER_I2C'], if_true: files('vhost-user-i2c-pci.c')) virtio_ss.add(when: 'CONFIG_VHOST_USER_RNG', if_true: files('vhost-user-rng.c')) virtio_ss.add(when: ['CONFIG_VHOST_USER_RNG', 'CONFIG_VIRTIO_PCI'], if_true: files('vhost-user-rng-pci.c')) +virtio_ss.add(when: 'CONFIG_VHOST_VDPA_DEV', if_true: files('vdpa-dev.c'))
virtio_pci_ss=ss.source_set()virtio_pci_ss.add(when: 'CONFIG_VHOST_VSOCK', if_true: files('vhost-vsock-pci.c'))hw/virtio/vdpa-dev.c new file +4
-0@@ -0,0 +1,377 @@ +/* +* Vhost Vdpa Device +* +* Copyright (c) Huawei Technologies Co., Ltd. 2022. All Rights Reserved.
+ *
+ * Authors:
+ * Longpeng <[email protected]>
+ *
+ * Largely based on the "vhost-user-blk-pci.c" and "vhost-user-blk.c" implemented by:
+ * Changpeng Liu <[email protected]>
+ *
+ * This work is licensed under the terms of the GNU LGPL, version 2 or later.
+ * See the COPYING.LIB file in the top-level directory.
+ */hw/virtio/vdpa-dev.c new file +57
-0+#include "qemu/osdep.h" +#include <sys/ioctl.h> +#include <linux/vhost.h> +#include "qapi/error.h" +#include "qemu/error-report.h" +#include "qemu/cutils.h" +#include "hw/qdev-core.h" +#include "hw/qdev-properties.h" +#include "hw/qdev-properties-system.h" +#include "hw/virtio/vhost.h" +#include "hw/virtio/virtio.h" +#include "hw/virtio/virtio-bus.h" +#include "hw/virtio/virtio-access.h" +#include "hw/virtio/vdpa-dev.h" +#include "sysemu/sysemu.h" +#include "sysemu/runstate.h" + +static void +vhost_vdpa_device_dummy_handle_output (VirtIODevice *vdev,VirtQueue *vq) +{ + /* Nothing to do */ +} + +static uint32_t +vhost_vdpa_device_get_u32 (int fd,unsigned long int cmd,Error **errp) +{ + uint32_t val = (uint32_t)-1; + + if (ioctl(fd, cmd, &val) < 0) { + error_setg(errp, "vhost-vdpa-device: cmd 0x%lx failed: %s", + cmd, strerror(errno)); + } + + return val; +} + +static void vhost_vdpa_device_realize (DeviceState *dev,Error **errp) +{ + VirtIODevice *vdev = VIRTIO_DEVICE(dev); + VhostVdpaDevice *v = VHOST_VDPA_DEVICE(vdev); + uint16_t max_queue_size; + struct vhost_virtqueue *vqs; + int i, ret; + + if (!v->vhostdev) { + error_setg(errp, "vhost-vdpa-device: vhostdev are missing"); + return; + } + + v->vhostfd = qemu_open(v->vhostdev, O_RDWR, errp); + if (*errp) { + return; + } + v->vdpa.device_fd = v->vhostfd; + + v->vdev_id = vhost_vdpa_device_get_u32(v->vhostfd, +
VHOST_VDPA_GET_DEVICE_ID, errp);
+ if (*errp) {
+ goto out;
+ }hw/virtio/vdpa-dev.c new file +3
-0+ + max_queue_size = vhost_vdpa_device_get_u32(v->vhostfd, +
VHOST_VDPA_GET_VRING_NUM, errp);
+ if (*errp) {
+ goto out;
+ }
+
+ if (v->queue_size>max_queue_size) {
+ error_setg(errp, "vhost-vdpa-device: invalidqueue_size:
%u (max:%u)",
+v->queue_size,max_queue_size);
+ goto out;
+ } else if (!v->queue_size) {
+v->queue_size=max_queue_size;
+ }hw/virtio/vdpa-dev.c new file +3
-0+ + v->num_queues = vhost_vdpa_device_get_u32(v->vhostfd, +
VHOST_VDPA_GET_VQS_COUNT, errp);
+ if (*errp) {
+ goto out;
+ }hw/virtio/vdpa-dev.c new file +16
-0+ + if (!v->num_queues || v->num_queues > VIRTIO_QUEUE_MAX) { + error_setg(errp, "invalid number of virtqueues: %u (max:%u)", + v->num_queues, VIRTIO_QUEUE_MAX); + goto out; + } + + v->dev.nvqs = v->num_queues; + vqs = g_new0(struct vhost_virtqueue, v->dev.nvqs); + v->dev.vqs = vqs; + v->dev.vq_index = 0; + v->dev.vq_index_end = v->dev.nvqs; + v->dev.backend_features = 0; + v->started = false; + + ret = vhost_dev_init(&v->dev, &v->vdpa,
VHOST_BACKEND_TYPE_VDPA, 0, NULL);
+ if (ret < 0) {
+ error_setg(errp, "vhost-vdpa-device: vhost initialization
failed: %s",
+ strerror(-ret));
+ gotofree_vqs;
+ }hw/virtio/vdpa-dev.c new file +3
-0+ + v->config_size = vhost_vdpa_device_get_u32(v->vhostfd, +Quoted 7 lines
VHOST_VDPA_GET_CONFIG_SIZE, errp);
+ if (*errp) {
+ gotovhost_cleanup;
+ }
+
+ if (v->post_init&&v->post_init(v, errp)< 0) { + gotofree_virtio;This goto seems wrong.
Quoted 4 lines
Quoted 2 lines
Either we use the
vhost_cleanuplabel or we move this block just before return as it seems to me was in v3.Good catch. We should use '
vhost_cleanup' label here.
Yep, so in that case we can remove the free_virtio label and the related block.
Quoted 5 lines
Quoted 3 lines
Out of curiosity, is there a reason for this change from v3 or was a simple mistake as a consequence of patch reorganization?
We should invoke .post_init() before calling
virtio_init()which would use the fields of VirtIOPCIProxy.
Okay, so maybe a comment there would be helpful.
Thanks,
Stefano
PATCH v5 4/4 vdpa: add vdpa-dev-pci support
In reply to Longpeng(Mike) viaFrom: Longpeng <[email protected]>
Supports vdpa-dev-pci, we can use the device as follow:
-device vhost-vdpa-device-pci,vhostdev=/dev/vhost-vdpa-X
--- hw/virtio/meson.build | 1 + hw/virtio/vdpa-dev-pci.c | 101 +++++++++++++++++++++++++++++++++++++++ 2 files changed, 102 insertions(+) create mode 100644 hw/virtio/vdpa-dev-pci.c
hw/virtio/meson.build +1 -0
@@ -50,6 +50,7 @@ virtio_pci_ss.add(when: 'CONFIG_VIRTIO_SERIAL', if_true: files('virtio-serial-pc virtio_pci_ss.add(when: 'CONFIG_VIRTIO_PMEM', if_true: files('virtio-pmem-pci.c')) virtio_pci_ss.add(when: 'CONFIG_VIRTIO_IOMMU', if_true: files('virtio-iommu-pci.c')) virtio_pci_ss.add(when: 'CONFIG_VIRTIO_MEM', if_true: files('virtio-mem-pci.c')) +virtio_pci_ss.add(when: 'CONFIG_VHOST_VDPA_DEV', if_true: files('vdpa-dev-pci.c')) virtio_ss.add_all(when: 'CONFIG_VIRTIO_PCI', if_true: virtio_pci_ss)
hw/virtio/vdpa-dev-pci.c new file +101 -0
101 lines not on this page · raw message
Signature · 1 line
2.23.0
Re: [PATCH v5 4/4] vdpa: add vdpa-dev-pci support
Quoted 4 lines · Longpeng(Mike)
From: Longpeng <[email protected]>
Supports vdpa-dev-pci, we can use the device as follow:
Quoted 8 lines
-device vhost-vdpa-device-pci,vhostdev=/dev/vhost-vdpa-X
Signed-off-by Longpeng <[email protected]>---
hw/virtio/meson.build | 1 +hw/virtio/vdpa-dev-pci.c| 101 +++++++++++++++++++++++++++++++++++++++2 files changed, 102 insertions(+)create mode 100644
hw/virtio/vdpa-dev-pci.c