Support VIRTIO_F_RING_RESET for virtio-net, vhost-user, vhost-kernel in virtio pci-modern
cover Support VIRTIO_F_RING_RESET for virtio-net, vhost-user, vhost-kernel in virtio pci-modern
The virtio queue reset function has already been defined in the virtio spec 1.2. The relevant virtio spec information is here:
https://github.com/oasis-tcs/virtio-spec/issues/124 https://github.com/oasis-tcs/virtio-spec/issues/139
This patch set is to support this function in QEMU. It consists of several parts:
1. Patches 1-7 are the basic interfaces for vq reset in virtio and virtio-pci.
2. Patches 8-12 support vq stop and vq restart for vhost-kernel.
3. Patches 13-19 support vq stop and vq restart for vhost-user.
4. Patches 20-22 support vq reset and re-enable for virtio-net.
5. Patches 23-24 enable the vq reset feature for vhost-kernel and vhost-user.
The process of virtqueue reset can be concluded as:
1. The virtqueue is disabled when VIRTIO_PCI_COMMON_Q_RESET is written.
2. Then the virtqueue can be optionally restarted(re-enabled).
Since this patch set involves multiple modules and seems a bit messy, we briefly describe the calling process for different modes below.
virtio-net:
1. VIRTIO_PCI_COMMON_Q_RESET is written [virtio-pci]
-> virtio_queue_reset() [virtio]
-> virtio_net_queue_reset() [virtio-net]
-> __virtio_queue_reset()
2. VIRTIO_PCI_COMMON_Q_ENABLE is written [virtio-pci]
-> set enabled, reset status of vq.vhost-kernel:
1. VIRTIO_PCI_COMMON_Q_RESET is written [virtio-pci]
-> virtio_queue_reset() [virtio]
-> virtio_net_queue_reset() [virtio-net]
-> vhost_net_virtqueue_stop() [vhost-net]
-> vhost_net_set_backend() [vhost]
-> vhost_dev_virtqueue_stop()
-> vhost_virtqueue_unmap()
-> __virtio_queue_reset()
2. VIRTIO_PCI_COMMON_Q_ENABLE is written [virtio-pci]
-> virtio_queue_enable() [virtio]
-> virtio_net_queue_enable() [virtio-net]
-> vhost_net_virtqueue_restart() [vhost-net]
-> vhost_dev_virtqueue_restart() [vhost]
-> vhost_virtqueue_start()
-> vhost_net_set_backend()
-> set enabled, reset status of vq.vhost-user:
1. VIRTIO_PCI_COMMON_Q_RESET is written [virtio-pci]
-> virtio_queue_reset() [virtio]
-> virtio_net_queue_reset() [virtio-net]
-> vhost_net_virtqueue_stop() [vhost-net]
-> vhost_dev_virtqueue_stop() [vhost]
-> vhost_user_reset_vring() [vhost-user]
-> send VHOST_USER_RESET_VRING to the device
-> vhost_virtqueue_unmap()
-> __virtio_queue_reset()
2. VIRTIO_PCI_COMMON_Q_ENABLE is written [virtio-pci]
-> virtio_queue_enable() [virtio]
-> virtio_net_queue_enable() [virtio-net]
-> vhost_net_virtqueue_restart() [vhost-net]
-> vhost_dev_virtqueue_restart() [vhost]
-> vhost_virtqueue_start()
-> vhost_user_set_single_vring_enable [vhost-user]
-> send VHOST_USER_SET_VRING_ENABLE to the device
-> set enabled, reset status of vq.Test environment:
Host: 5.19.0-rc3 (With vq reset support)
Qemu: QEMU emulator version 7.0.50
Guest: 5.19.0-rc3 (With vq reset support)
DPDK: 22.07-rc1 (With vq reset support)
Test Cmd: ethtool -g eth1; ethtool -G eth1 rx $1 tx $2; ethtool -g eth1;
The drvier can resize the virtio queue, then virtio queue reset function should be triggered.
The default is split mode, modify Qemu virtio-net to add PACKED feature to test packed mode.
Guest Kernel Patch:
https://lore.kernel.org/bpf/[email protected]/
DPDK Patch:
https://github.com/middaywords/dpdk/compare/72206323a5dd3182b13f61b25a64abdddfee595c...eabadfac7953da66bc10ffb8284b490d09bb7ec7
Host Kernel Patch:
https://github.com/middaywords/linux/commit/19a91e0d7167b2031e46078c6215c213b89cb2c3
Looking forward to your review and comments. Thanks.
Kangjie Xu (19):
virtio: introduce virtio_queue_enable()
virtio: core: vq reset feature negotation support
virtio-pci: support queue enable
vhost: extract the logic of unmapping the vrings and desc
vhost: introduce vhost_dev_virtqueue_stop()
vhost: introduce vhost_dev_virtqueue_restart()
vhost-net: vhost-kernel: introduce vhost_net_virtqueue_stop()
vhost-net: vhost-kernel: introduce vhost_net_virtqueue_restart()
docs: vhost-user: add VHOST_USER_RESET_VRING message
vhost-user: introduce vhost_reset_vring() interface
vhost-user: add op to enable or disable a single vring
vhost: vhost-user: update vhost_dev_virtqueue_stop()
vhost: vhost-user: update vhost_dev_virtqueue_restart()
vhost-net: vhost-user: update vhost_net_virtqueue_stop()
vhost-net: vhost-user: update vhost_net_virtqueue_restart()
virtio-net: introduce flush_or_purge_queued_packets()
virtio-net: support queue_enable
vhost: vhost-kernel: enable vq reset feature
vhost: vhost-user: enable vq reset feature
Xuan Zhuo (5):
virtio: sync relevant definitions with linux
virtio: introduce __virtio_queue_reset()
virtio: introduce virtio_queue_reset()
virtio-pci: support queue reset
virtio-net: support queue reset
docs/interop/vhost-user.rst | 10 +++ hw/core/machine.c | 1 + hw/net/vhost_net.c | 79 +++++++++++++++++++ hw/net/virtio-net.c | 58 ++++++++++++-- hw/virtio/vhost-user.c | 67 ++++++++++++++-- hw/virtio/vhost.c | 79 +++++++++++++++++-- hw/virtio/virtio-pci.c | 20 +++++ hw/virtio/virtio.c | 62 +++++++++++---- include/hw/virtio/vhost-backend.h | 6 ++ include/hw/virtio/vhost.h | 5 ++ include/hw/virtio/virtio-pci.h | 1 + include/hw/virtio/virtio.h | 8 +- include/net/vhost_net.h | 4 + .../standard-headers/linux/virtio_config.h | 5 ++ include/standard-headers/linux/virtio_pci.h | 2 + 15 files changed, 371 insertions(+), 36 deletions(-)
Signature · 1 line
2.32.0
PATCH v2 1/24 virtio: sync relevant definitions with linux
From: Xuan Zhuo <[email protected]>
This is updated using scripts/update-linux-headers.sh.
Added VIRTIO_F_RING_RESET, VIRTIO_PCI_COMMON_Q_RESET. It came from here:
https://github.com/oasis-tcs/virtio-spec/issues/124
https://github.com/oasis-tcs/virtio-spec/issues/139
Add VIRTIO_PCI_COMMON_Q_NDATA, which comes from here:
https://github.com/oasis-tcs/virtio-spec/issues/89
--- include/standard-headers/linux/virtio_config.h | 5 +++++ include/standard-headers/linux/virtio_pci.h | 2 ++ 2 files changed, 7 insertions(+)
include/standard-headers/linux/virtio_config.h +5 -0
@@ -96,4 +96,9 @@* Does the device support Single Root I/O Virtualization? */ #define VIRTIO_F_SR_IOV 37 + +/* +* This feature indicates that the driver can reset a queue individually. +*/ +#define VIRTIO_F_RING_RESET 40#endif /* _LINUX_VIRTIO_CONFIG_H */
include/standard-headers/linux/virtio_pci.h +2 -0
@@ -202,6 +202,8 @@ struct virtio_pci_cfg_cap {#define VIRTIO_PCI_COMMON_Q_AVAILHI 44#define VIRTIO_PCI_COMMON_Q_USEDLO 48#define VIRTIO_PCI_COMMON_Q_USEDHI 52 +#define VIRTIO_PCI_COMMON_Q_NDATA 56 +#define VIRTIO_PCI_COMMON_Q_RESET 58#endif /* VIRTIO_PCI_NO_MODERN */
Signature · 1 line
2.32.0
PATCH v2 2/24 virtio: introduce __virtio_queue_reset()
In reply to Kangjie XuFrom: Xuan Zhuo <[email protected]>
Separate the logic of vq reset. This logic will be called directly later.
--- hw/virtio/virtio.c | 37 +++++++++++++++++++++---------------- 1 file changed, 21 insertions(+), 16 deletions(-)
hw/virtio/virtio.c +21 -16
@@ -2019,6 +2019,26 @@ static enum virtio_device_endian virtio_current_cpu_endian(void) } } +static void __virtio_queue_reset (VirtIODevice *vdev,uint32_t i) +{ + vdev->vq[i].vring.desc =0 ; + vdev->vq[i].vring.avail =0 ; + vdev->vq[i].vring.used =0 ; + vdev->vq[i].last_avail_idx =0 ; + vdev->vq[i].shadow_avail_idx =0 ; + vdev->vq[i].used_idx =0 ; + vdev->vq[i].last_avail_wrap_counter = true; + vdev->vq[i].shadow_avail_wrap_counter = true; + vdev->vq[i].used_wrap_counter = true; +virtio_queue_set_vector (vdev, i,VIRTIO_NO_VECTOR ); + vdev->vq[i].signalled_used =0 ; + vdev->vq[i].signalled_used_valid = false; + vdev->vq[i].notification = true; + vdev->vq[i].vring.num = vdev->vq[i].vring.num_default; + vdev->vq[i].inuse =0 ; +virtio_virtqueue_reset_region_cache (&vdev->vq[i]); +} +void virtio_reset (void *opaque) {VirtIODevice *vdev = opaque;
@@ -2050,22 +2070,7 @@ void virtio_reset(void *opaque)virtio_notify_vector (vdev, vdev->config_vector);for (i =0 ; i <VIRTIO_QUEUE_MAX ; i++) { - vdev->vq[i].vring.desc =0 ; - vdev->vq[i].vring.avail =0 ; - vdev->vq[i].vring.used =0 ; - vdev->vq[i].last_avail_idx =0 ; - vdev->vq[i].shadow_avail_idx =0 ; - vdev->vq[i].used_idx =0 ; - vdev->vq[i].last_avail_wrap_counter = true; - vdev->vq[i].shadow_avail_wrap_counter = true; - vdev->vq[i].used_wrap_counter = true; -virtio_queue_set_vector (vdev, i,VIRTIO_NO_VECTOR ); - vdev->vq[i].signalled_used =0 ; - vdev->vq[i].signalled_used_valid = false; - vdev->vq[i].notification = true; - vdev->vq[i].vring.num = vdev->vq[i].vring.num_default; - vdev->vq[i].inuse =0 ; -virtio_virtqueue_reset_region_cache (&vdev->vq[i]); +__virtio_queue_reset (vdev, i); } }
Signature · 1 line
2.32.0
Re: [PATCH v2 02/24] virtio: introduce __virtio_queue_reset()
Quoted 6 lines · Kangjie Xu
From: Xuan Zhuo <[email protected]>
Separate the logic of vq reset. This logic will be called directly later.
Signed-off-by Xuan Zhuo <[email protected]>
Quoted 59 lines
--- hw/virtio/virtio.c | 37 +++++++++++++++++++++---------------- 1 file changed, 21 insertions(+), 16 deletions(-)hw/virtio/virtio.c +21
-16@@ -2019,6 +2019,26 @@ static enum virtio_device_endian virtio_current_cpu_endian(void) } } +static void __virtio_queue_reset (VirtIODevice *vdev,uint32_t i) +{ + vdev->vq[i].vring.desc =0 ; + vdev->vq[i].vring.avail =0 ; + vdev->vq[i].vring.used =0 ; + vdev->vq[i].last_avail_idx =0 ; + vdev->vq[i].shadow_avail_idx =0 ; + vdev->vq[i].used_idx =0 ; + vdev->vq[i].last_avail_wrap_counter = true; + vdev->vq[i].shadow_avail_wrap_counter = true; + vdev->vq[i].used_wrap_counter = true; +virtio_queue_set_vector (vdev, i,VIRTIO_NO_VECTOR ); + vdev->vq[i].signalled_used =0 ; + vdev->vq[i].signalled_used_valid = false; + vdev->vq[i].notification = true; + vdev->vq[i].vring.num = vdev->vq[i].vring.num_default; + vdev->vq[i].inuse =0 ; +virtio_virtqueue_reset_region_cache (&vdev->vq[i]); +} +void virtio_reset (void *opaque) {VirtIODevice *vdev = opaque;@@ -2050,22 +2070,7 @@ void virtio_reset(void *opaque)virtio_notify_vector (vdev, vdev->config_vector);for (i =0 ; i <VIRTIO_QUEUE_MAX ; i++) { - vdev->vq[i].vring.desc =0 ; - vdev->vq[i].vring.avail =0 ; - vdev->vq[i].vring.used =0 ; - vdev->vq[i].last_avail_idx =0 ; - vdev->vq[i].shadow_avail_idx =0 ; - vdev->vq[i].used_idx =0 ; - vdev->vq[i].last_avail_wrap_counter = true; - vdev->vq[i].shadow_avail_wrap_counter = true; - vdev->vq[i].used_wrap_counter = true; -virtio_queue_set_vector (vdev, i,VIRTIO_NO_VECTOR ); - vdev->vq[i].signalled_used =0 ; - vdev->vq[i].signalled_used_valid = false; - vdev->vq[i].notification = true; - vdev->vq[i].vring.num = vdev->vq[i].vring.num_default; - vdev->vq[i].inuse =0 ; -virtio_virtqueue_reset_region_cache (&vdev->vq[i]); +__virtio_queue_reset (vdev, i); } }
PATCH v2 3/24 virtio: introduce virtio_queue_reset()
In reply to Kangjie XuFrom: Xuan Zhuo <[email protected]>
Introduce a new interface function virtio_queue_reset() to implement reset for vq.
Add a new callback to VirtioDeviceClass for queue reset operation for each child device.
--- hw/virtio/virtio.c | 11 +++++++++++ include/hw/virtio/virtio.h | 2 ++ 2 files changed, 13 insertions(+)
hw/virtio/virtio.c +11 -0
@@ -2039,6 +2039,17 @@ static void __virtio_queue_reset(VirtIODevice *vdev, uint32_t i)virtio_virtqueue_reset_region_cache (&vdev->vq[i]); } +void virtio_queue_reset (VirtIODevice *vdev,uint32_t queue_index) +{ +VirtioDeviceClass *k =VIRTIO_DEVICE_GET_CLASS (vdev); + +if (k->queue_reset) { + k->queue_reset (vdev, queue_index); + } + +__virtio_queue_reset (vdev, queue_index); +} +void virtio_reset (void *opaque) {VirtIODevice *vdev = opaque;
include/hw/virtio/virtio.h +2 -0
@@ -130,6 +130,7 @@ struct VirtioDeviceClass {void (*set_config)(VirtIODevice *vdev,const uint8_t *config);void (*reset)(VirtIODevice *vdev);void (*set_status)(VirtIODevice *vdev,uint8_t val); +void (*queue_reset)(VirtIODevice *vdev,uint32_t queue_index);/* For transitional devices, this is a bitmap of features * that are only exposed on the legacy interface but not * the modern one.
@@ -268,6 +269,7 @@ int virtio_queue_set_host_notifier_mr(VirtIODevice *vdev, int n, MemoryRegion *mr, bool assign);int virtio_set_status (VirtIODevice *vdev,uint8_t val);void virtio_reset (void *opaque); +void virtio_queue_reset (VirtIODevice *vdev,uint32_t queue_index);void virtio_update_irq (VirtIODevice *vdev);int virtio_set_features (VirtIODevice *vdev,uint64_t val);
Signature · 1 line
2.32.0
Re: [PATCH v2 03/24] virtio: introduce virtio_queue_reset()
Quoted 9 lines · Kangjie Xu
From: Xuan Zhuo <[email protected]>
Introduce a new interface function
virtio_queue_reset()to implement reset for vq.Add a new callback to VirtioDeviceClass for queue reset operation for each child device.
Signed-off-by Xuan Zhuo <[email protected]>
Quoted 47 lines
--- hw/virtio/virtio.c | 11 +++++++++++ include/hw/virtio/virtio.h | 2 ++ 2 files changed, 13 insertions(+)hw/virtio/virtio.c +11
-0@@ -2039,6 +2039,17 @@ static void __virtio_queue_reset(VirtIODevice *vdev, uint32_t i)virtio_virtqueue_reset_region_cache (&vdev->vq[i]); } +void virtio_queue_reset (VirtIODevice *vdev,uint32_t queue_index) +{ +VirtioDeviceClass *k =VIRTIO_DEVICE_GET_CLASS (vdev); + +if (k->queue_reset) { + k->queue_reset (vdev, queue_index); + } + +__virtio_queue_reset (vdev, queue_index); +} +void virtio_reset (void *opaque) {VirtIODevice *vdev = opaque;include/hw/virtio/virtio.h +2
-0@@ -130,6 +130,7 @@ struct VirtioDeviceClass {void (*set_config)(VirtIODevice *vdev,const uint8_t *config);void (*reset)(VirtIODevice *vdev);void (*set_status)(VirtIODevice *vdev,uint8_t val); +void (*queue_reset)(VirtIODevice *vdev,uint32_t queue_index);/* For transitional devices, this is a bitmap of features * that are only exposed on the legacy interface but not * the modern one. @@ -268,6 +269,7 @@ int virtio_queue_set_host_notifier_mr(VirtIODevice *vdev, int n, MemoryRegion *mr, bool assign);int virtio_set_status (VirtIODevice *vdev,uint8_t val);void virtio_reset (void *opaque); +void virtio_queue_reset (VirtIODevice *vdev,uint32_t queue_index);void virtio_update_irq (VirtIODevice *vdev);int virtio_set_features (VirtIODevice *vdev,uint64_t val);
PATCH v2 4/24 virtio: introduce virtio_queue_enable()
In reply to Kangjie XuIntroduce the interface queue_enable() in VirtioDeviceClass and the fucntion virtio_queue_enable() in virtio, it can be called when VIRTIO_PCI_COMMON_Q_ENABLE is written and related virtqueue can be started. It only supports the devices of virtio 1 or later. The not-supported devices can only start the virtqueue when DRIVER_OK.
--- hw/virtio/virtio.c | 14 ++++++++++++++ include/hw/virtio/virtio.h | 2 ++ 2 files changed, 16 insertions(+)
hw/virtio/virtio.c +14 -0
@@ -2050,6 +2050,20 @@ void virtio_queue_reset(VirtIODevice *vdev, uint32_t queue_index)__virtio_queue_reset (vdev, queue_index); } +void virtio_queue_enable (VirtIODevice *vdev,uint32_t queue_index) +{ +VirtioDeviceClass *k =VIRTIO_DEVICE_GET_CLASS (vdev); + +if (!virtio_vdev_has_feature (vdev,VIRTIO_F_VERSION_1 )) { +error_report ("queue_enable is only suppported in devices of virtio " +"1.0 or later." ); + } + +if (k->queue_enable) { + k->queue_enable (vdev, queue_index); + } +} +void virtio_reset (void *opaque) {VirtIODevice *vdev = opaque;
include/hw/virtio/virtio.h +2 -0
@@ -131,6 +131,7 @@ struct VirtioDeviceClass {void (*reset)(VirtIODevice *vdev);void (*set_status)(VirtIODevice *vdev,uint8_t val);void (*queue_reset)(VirtIODevice *vdev,uint32_t queue_index); +void (*queue_enable)(VirtIODevice *vdev,uint32_t queue_index);/* For transitional devices, this is a bitmap of features * that are only exposed on the legacy interface but not * the modern one.
@@ -270,6 +271,7 @@ int virtio_queue_set_host_notifier_mr(VirtIODevice *vdev, int n,int virtio_set_status (VirtIODevice *vdev,uint8_t val);void virtio_reset (void *opaque);void virtio_queue_reset (VirtIODevice *vdev,uint32_t queue_index); +void virtio_queue_enable (VirtIODevice *vdev,uint32_t queue_index);void virtio_update_irq (VirtIODevice *vdev);int virtio_set_features (VirtIODevice *vdev,uint64_t val);
Signature · 1 line
2.32.0
Re: [PATCH v2 04/24] virtio: introduce virtio_queue_enable()
Quoted 9 lines · Kangjie Xu
Introduce the interface
queue_enable()in VirtioDeviceClass and the fucntionvirtio_queue_enable()in virtio, it can be called whenVIRTIO_PCI_COMMON_Q_ENABLEis written and related virtqueue can be started. It only supports the devices of virtio 1 or later. The not-supported devices can only start the virtqueue whenDRIVER_OK.Signed-off-by Kangjie Xu <[email protected]>Signed-off-by Xuan Zhuo <[email protected]>---
Quoted 49 lines
hw/virtio/virtio.c | 14 ++++++++++++++ include/hw/virtio/virtio.h | 2 ++ 2 files changed, 16 insertions(+)hw/virtio/virtio.c +14
-0@@ -2050,6 +2050,20 @@ void virtio_queue_reset(VirtIODevice *vdev, uint32_t queue_index)__virtio_queue_reset (vdev, queue_index); } +void virtio_queue_enable (VirtIODevice *vdev,uint32_t queue_index) +{ +VirtioDeviceClass *k =VIRTIO_DEVICE_GET_CLASS (vdev); + +if (!virtio_vdev_has_feature (vdev,VIRTIO_F_VERSION_1 )) { +error_report ("queue_enable is only suppported in devices of virtio " +"1.0 or later." ); + } + +if (k->queue_enable) { + k->queue_enable (vdev, queue_index); + } +} +void virtio_reset (void *opaque) {VirtIODevice *vdev = opaque;include/hw/virtio/virtio.h +2
-0@@ -131,6 +131,7 @@ struct VirtioDeviceClass {void (*reset)(VirtIODevice *vdev);void (*set_status)(VirtIODevice *vdev,uint8_t val);void (*queue_reset)(VirtIODevice *vdev,uint32_t queue_index); +void (*queue_enable)(VirtIODevice *vdev,uint32_t queue_index);/* For transitional devices, this is a bitmap of features * that are only exposed on the legacy interface but not * the modern one. @@ -270,6 +271,7 @@ int virtio_queue_set_host_notifier_mr(VirtIODevice *vdev, int n,int virtio_set_status (VirtIODevice *vdev,uint8_t val);void virtio_reset (void *opaque);void virtio_queue_reset (VirtIODevice *vdev,uint32_t queue_index); +void virtio_queue_enable (VirtIODevice *vdev,uint32_t queue_index);void virtio_update_irq (VirtIODevice *vdev);int virtio_set_features (VirtIODevice *vdev,uint64_t val);
PATCH v2 5/24 virtio: core: vq reset feature negotation support
In reply to Kangjie XuA a new command line parameter "queue_reset" is added.
Meanwhile, the vq reset feature is disabled for pre-7.1 machines.
--- hw/core/machine.c | 1 + include/hw/virtio/virtio.h | 4 +++- 2 files changed, 4 insertions(+), 1 deletion(-)
hw/core/machine.c +1 -0
@@ -43,6 +43,7 @@GlobalProperty hw_compat_7_0[] = { {"arm-gicv3-common" ,"force-8-bit-prio" ,"on" }, {"nvme-ns" ,"eui64-default" ,"on" }, + {"virtio-device" ,"queue_reset" ,"false" }, };const size_t hw_compat_7_0_len =G_N_ELEMENTS (hw_compat_7_0);
include/hw/virtio/virtio.h +3 -1
@@ -295,7 +295,9 @@ typedef struct VirtIORNGConf VirtIORNGConf;DEFINE_PROP_BIT64 ("iommu_platform" , _state, _field, \VIRTIO_F_IOMMU_PLATFORM , false), \DEFINE_PROP_BIT64 ("packed" , _state, _field, \ - VIRTIO_F_RING_PACKED, false) +VIRTIO_F_RING_PACKED , false), \ +DEFINE_PROP_BIT64 ("queue_reset" , _state, _field, \ + VIRTIO_F_RING_RESET, true)hwaddr virtio_queue_get_desc_addr (VirtIODevice *vdev,int n);bool virtio_queue_enabled_legacy (VirtIODevice *vdev,int n);
Signature · 1 line
2.32.0
Re: [PATCH v2 05/24] virtio: core: vq reset feature negotation support
Quoted 16 lines · Kangjie Xu
A a new command line parameter "
queue_reset" is added.Meanwhile, the vq reset feature is disabled for pre-7.1 machines.
Signed-off-by Kangjie Xu <[email protected]>Signed-off-by Xuan Zhuo <[email protected]>--- hw/core/machine.c | 1 + include/hw/virtio/virtio.h | 4 +++- 2 files changed, 4 insertions(+), 1 deletion(-)hw/core/machine.c +0
-0
hw/core/machine.c line 46
Quoted 4 lines
hw/core/machine.c +1
-0@@ -43,6 +43,7 @@GlobalProperty hw_compat_7_0[] = { {"arm-gicv3-common" ,"force-8-bit-prio" ,"on" }, {"nvme-ns" ,"eui64-default" ,"on" }, + {"virtio-device" ,"queue_reset" ,"false" },
7.1 is about to release so we need to do it for pre-7.2.
Thanks
Quoted 18 lines
hw/core/machine.c +0
-0};const size_t hw_compat_7_0_len =G_N_ELEMENTS (hw_compat_7_0);include/hw/virtio/virtio.h +3
-1@@ -295,7 +295,9 @@ typedef struct VirtIORNGConf VirtIORNGConf;DEFINE_PROP_BIT64 ("iommu_platform" , _state, _field, \VIRTIO_F_IOMMU_PLATFORM , false), \DEFINE_PROP_BIT64 ("packed" , _state, _field, \ - VIRTIO_F_RING_PACKED, false) +VIRTIO_F_RING_PACKED , false), \ +DEFINE_PROP_BIT64 ("queue_reset" , _state, _field, \ + VIRTIO_F_RING_RESET, true)hwaddr virtio_queue_get_desc_addr (VirtIODevice *vdev,int n);bool virtio_queue_enabled_legacy (VirtIODevice *vdev,int n);
Re: [PATCH v2 05/24] virtio: core: vq reset feature negotation support
Quoted 26 lines · Jason Wang
Quoted 16 lines · Kangjie Xu
A a new command line parameter "
queue_reset" is added.Meanwhile, the vq reset feature is disabled for pre-7.1 machines.
Signed-off-by Kangjie Xu <[email protected]>Signed-off-by Xuan Zhuo <[email protected]>---
hw/core/machine.c | 1 +
include/hw/virtio/virtio.h| 4 +++-2 files changed, 4 insertions(+), 1 deletion(-)hw/core/machine.c +0
-0hw/core/machine.c line 46
Quoted 4 lines
hw/core/machine.c +1
-0@@ -43,6 +43,7 @@ GlobalProperty hw_compat_7_0[] = { { "arm-gicv3-common", "force-8-bit-prio", "on" }, { "nvme-ns", "eui64-default", "on"}, + { "virtio-device", "queue_reset", "false" },7.1 is about to release so we need to do it for pre-7.2.
Quoted 2 lines
Thanks
The current version is 7.0.92, should I wait until 7.1 to be released and submit my patch after that? or just add hw_compat_7_1[] and related support.
Thanks
Quoted 17 lines
Quoted 17 lines
hw/core/machine.c +0
-0}; const size_t hw_compat_7_0_len = G_N_ELEMENTS(hw_compat_7_0);diff --git
a/include/hw/virtio/virtio.hb/include/hw/virtio/virtio.hindex085997d8f3..ed3ecbef80100644include/hw/virtio/virtio.h +3
-1@@ -295,7 +295,9 @@ typedef struct VirtIORNGConf VirtIORNGConf; DEFINE_PROP_BIT64("iommu_platform", _state, _field, \ VIRTIO_F_IOMMU_PLATFORM, false), \ DEFINE_PROP_BIT64("packed", _state, _field, \ - VIRTIO_F_RING_PACKED, false) + VIRTIO_F_RING_PACKED, false), \ + DEFINE_PROP_BIT64("queue_reset", _state, _field, \ + VIRTIO_F_RING_RESET, true) hwaddr virtio_queue_get_desc_addr(VirtIODevice *vdev, int n); bool virtio_queue_enabled_legacy(VirtIODevice *vdev, int n);
PATCH v2 6/24 virtio-pci: support queue reset
In reply to Kangjie XuFrom: Xuan Zhuo <[email protected]>
PCI devices support vq reset.
Based on this function, the driver can adjust the size of the ring, and quickly recycle the buffer in the ring.
The migration of the virtio devices will not happen during a reset operation. This is becuase the global iothread lock is held. Migration thread also needs the lock. As a result, we do not need to migrate the reset state of VirtIOPCIQueue.
--- hw/virtio/virtio-pci.c | 19 +++++++++++++++++++ include/hw/virtio/virtio-pci.h | 1 + 2 files changed, 20 insertions(+)
hw/virtio/virtio-pci.c +19 -0
@@ -1246,6 +1246,9 @@ static uint64_t virtio_pci_common_read(void *opaque, hwaddr addr,case VIRTIO_PCI_COMMON_Q_USEDHI : val = proxy->vqs[vdev->queue_sel].used[1 ];break ; +case VIRTIO_PCI_COMMON_Q_RESET : + val = proxy->vqs[vdev->queue_sel].reset; +break ;default : val =0 ; }
@@ -1333,6 +1336,7 @@ static void virtio_pci_common_write(void *opaque, hwaddr addr, ((uint64_t )proxy->vqs[vdev->queue_sel].used[1 ]) <<32 | proxy->vqs[vdev->queue_sel].used[0]); proxy->vqs[vdev->queue_sel].enabled =1 ; + proxy->vqs[vdev->queue_sel].reset =0 ; }else {virtio_error (vdev,"wrong value for queue_enable %" PRIx64, val); }
@@ -1355,6 +1359,20 @@ static void virtio_pci_common_write(void *opaque, hwaddr addr,case VIRTIO_PCI_COMMON_Q_USEDHI : proxy->vqs[vdev->queue_sel].used[1 ] = val;break ; +case VIRTIO_PCI_COMMON_Q_RESET : +if (val ==1 ) { +/* +* With the global iothread lock taken, the migration will not +* happen until the virtqueue reset is done. +*/ + proxy->vqs[vdev->queue_sel].reset =1 ; + +virtio_queue_reset (vdev, vdev->queue_sel); + + proxy->vqs[vdev->queue_sel].reset =0 ; + proxy->vqs[vdev->queue_sel].enabled =0 ; + } +break ;default :break ; }
@@ -1950,6 +1968,7 @@ static void virtio_pci_reset(DeviceState *qdev)for (i =0 ; i <VIRTIO_QUEUE_MAX ; i++) { proxy->vqs[i].enabled =0 ; + proxy->vqs[i].reset =0 ; proxy->vqs[i].num =0 ; proxy->vqs[i].desc[0 ] = proxy->vqs[i].desc[1 ] =0 ; proxy->vqs[i].avail[0 ] = proxy->vqs[i].avail[1 ] =0 ;
include/hw/virtio/virtio-pci.h +1 -0
@@ -117,6 +117,7 @@ typedef struct VirtIOPCIRegion { typedef struct VirtIOPCIQueue { uint16_t num; bool enabled; + bool reset; uint32_t desc[2]; uint32_t avail[2]; uint32_t used[2];
Signature · 1 line
2.32.0
Re: [PATCH v2 06/24] virtio-pci: support queue reset
Quoted 42 lines · Kangjie Xu
From: Xuan Zhuo <[email protected]>
PCI devices support vq reset.
Based on this function, the driver can adjust the size of the ring, and quickly recycle the buffer in the ring.
The migration of the virtio devices will not happen during a reset operation. This is becuase the global iothread lock is held. Migration thread also needs the lock. As a result, we do not need to migrate the reset state of VirtIOPCIQueue.
Signed-off-by Xuan Zhuo <[email protected]>Signed-off-by Kangjie Xu <[email protected]>--- hw/virtio/virtio-pci.c | 19 +++++++++++++++++++ include/hw/virtio/virtio-pci.h | 1 + 2 files changed, 20 insertions(+)hw/virtio/virtio-pci.c +4
-0@@ -1246,6 +1246,9 @@ static uint64_t virtio_pci_common_read(void *opaque, hwaddr addr,case VIRTIO_PCI_COMMON_Q_USEDHI : val = proxy->vqs[vdev->queue_sel].used[1 ];break ; +case VIRTIO_PCI_COMMON_Q_RESET : + val = proxy->vqs[vdev->queue_sel].reset; +break ;default : val =0 ; }@@ -1333,6 +1336,7 @@ static void virtio_pci_common_write(void *opaque, hwaddr addr, ((uint64_t )proxy->vqs[vdev->queue_sel].used[1 ]) <<32 | proxy->vqs[vdev->queue_sel].used[0]); proxy->vqs[vdev->queue_sel].enabled =1 ; + proxy->vqs[vdev->queue_sel].reset =0 ; }else {virtio_error (vdev,"wrong value for queue_enable %" PRIx64, val); }
hw/virtio/virtio-pci.c static void virtio_pci_common_write(void *opaque, hwaddr addr, · line 1367
Quoted 9 lines
hw/virtio/virtio-pci.c +6
-0@@ -1355,6 +1359,20 @@ static void virtio_pci_common_write(void *opaque, hwaddr addr,case VIRTIO_PCI_COMMON_Q_USEDHI : proxy->vqs[vdev->queue_sel].used[1 ] = val;break ; +case VIRTIO_PCI_COMMON_Q_RESET : +if (val ==1 ) { +/* +* With the global iothread lock taken, the migration will not +* happen until the virtqueue reset is done. +*/
This comment applies to all other common cfg operation as well, So it looks not necessary?
Quoted 24 lines
hw/virtio/virtio-pci.c +9
-0+ proxy->vqs[vdev->queue_sel].reset =1 ; + +virtio_queue_reset (vdev, vdev->queue_sel); + + proxy->vqs[vdev->queue_sel].reset =0 ; + proxy->vqs[vdev->queue_sel].enabled =0 ; + } +break ;default :break ; }@@ -1950,6 +1968,7 @@ static void virtio_pci_reset(DeviceState *qdev)for (i =0 ; i <VIRTIO_QUEUE_MAX ; i++) { proxy->vqs[i].enabled =0 ; + proxy->vqs[i].reset =0 ; proxy->vqs[i].num =0 ; proxy->vqs[i].desc[0 ] = proxy->vqs[i].desc[1 ] =0 ; proxy->vqs[i].avail[0 ] = proxy->vqs[i].avail[1 ] =0 ;include/hw/virtio/virtio-pci.h +0
-0
include/hw/virtio/virtio-pci.h typedef struct VirtIOPCIRegion { · line 120
Quoted 4 lines
include/hw/virtio/virtio-pci.h +1
-0@@ -117,6 +117,7 @@ typedef struct VirtIOPCIRegion { typedef struct VirtIOPCIQueue { uint16_t num; bool enabled; + bool reset;
Do we need to migrate this?
Thanks
Quoted 3 lines
include/hw/virtio/virtio-pci.h +0
-0uint32_t desc[2 ];uint32_t avail[2 ];uint32_t used[2 ];
Re: [PATCH v2 06/24] virtio-pci: support queue reset
Quoted 52 lines · Jason Wang
Quoted 50 lines · Kangjie Xu
From: Xuan Zhuo <[email protected]>
PCI devices support vq reset.
Based on this function, the driver can adjust the size of the ring, and quickly recycle the buffer in the ring.
The migration of the virtio devices will not happen during a reset operation. This is becuase the global iothread lock is held. Migration thread also needs the lock. As a result, we do not need to migrate the reset state of VirtIOPCIQueue.
Signed-off-by Xuan Zhuo <[email protected]>Signed-off-by Kangjie Xu <[email protected]>---
hw/virtio/virtio-pci.c | 19 +++++++++++++++++++include/hw/virtio/virtio-pci.h| 1 +2 files changed, 20 insertions(+)hw/virtio/virtio-pci.c +0
-0@@ -1246,6 +1246,9 @@ static uint64_t virtio_pci_common_read(void *opaque, hwaddr addr,case
VIRTIO_PCI_COMMON_Q_USEDHI:
val = proxy->vqs[vdev->queue_sel].used[1]; break;
+ caseVIRTIO_PCI_COMMON_Q_RESET:
+ val = proxy->vqs[vdev->queue_sel].reset; + break;
default:
val = 0;
}hw/virtio/virtio-pci.c +0
-0@@ -1333,6 +1336,7 @@ static void virtio_pci_common_write(void *opaque, hwaddr addr,((uint64_t)proxy->vqs[vdev->queue_sel].used[1]) << 32 | proxy->vqs[vdev->queue_sel].used[0]); proxy->vqs[vdev->queue_sel].enabled = 1;
+ proxy->vqs[vdev->queue_sel].reset = 0;
} else {
virtio_error(vdev, "wrong value for queue_enable %"PRIx64, val); }hw/virtio/virtio-pci.c +0
-0@@ -1355,6 +1359,20 @@ static void virtio_pci_common_write(void *opaque, hwaddr addr,case
VIRTIO_PCI_COMMON_Q_USEDHI:
proxy->vqs[vdev->queue_sel].used[1] = val; break;
+ caseVIRTIO_PCI_COMMON_Q_RESET:
+ if (val == 1) {
Quoted 9 lines
Quoted 5 lines
+ /*
+ * With the global iothread lock taken, the migration will not
+ * happen until the virtqueue reset is done. + */This comment applies to all other common cfg operation as well, So it looks not necessary?
Get it.
Quoted 32 lines
Quoted 24 lines
+ proxy->vqs[vdev->queue_sel].reset = 1;
hw/virtio/virtio-pci.c +7
-0+ + virtio_queue_reset(vdev, vdev->queue_sel); + + proxy->vqs[vdev->queue_sel].reset = 0; + proxy->vqs[vdev->queue_sel].enabled = 0; + } + break;default:
break;
}hw/virtio/virtio-pci.c +1
-0@@ -1950,6 +1968,7 @@ static void virtio_pci_reset(DeviceState *qdev) for (i = 0; i < VIRTIO_QUEUE_MAX; i++) { proxy->vqs[i].enabled = 0; + proxy->vqs[i].reset = 0; proxy->vqs[i].num = 0; proxy->vqs[i].desc[0] = proxy->vqs[i].desc[1] = 0; proxy->vqs[i].avail[0] = proxy->vqs[i].avail[1] = 0;include/hw/virtio/virtio-pci.h +0
-0include/hw/virtio/virtio-pci.h typedef struct VirtIOPCIRegion { · line 120
Quoted 4 lines
include/hw/virtio/virtio-pci.h +1
-0@@ -117,6 +117,7 @@ typedef struct VirtIOPCIRegion { typedef struct VirtIOPCIQueue { uint16_t num; bool enabled; + bool reset;Do we need to migrate this?
Quoted 2 lines
Thanks
I think we do not need to migrate this because we hold the global iothread lock when virtqueue reset is triggered. The migration of these device states also needs this lock.
On the other hand, the 'reset' state of virtqueue is same(is 0) before and after the process of resetting a virtqueue.
Thus, the migration will not happen when we are resetting a virtqueue and we do not to migrate it.
Thanks
Quoted 4 lines
Quoted 4 lines
include/hw/virtio/virtio-pci.h +0
-0uint32_t desc[2]; uint32_t avail[2]; uint32_t used[2];
Re: [PATCH v2 06/24] virtio-pci: support queue reset
Quoted 97 lines · Kangjie Xu
Quoted 52 lines · Jason Wang
Quoted 50 lines · Kangjie Xu
From: Xuan Zhuo <[email protected]>
PCI devices support vq reset.
Based on this function, the driver can adjust the size of the ring, and quickly recycle the buffer in the ring.
The migration of the virtio devices will not happen during a reset operation. This is becuase the global iothread lock is held. Migration thread also needs the lock. As a result, we do not need to migrate the reset state of VirtIOPCIQueue.
Signed-off-by Xuan Zhuo <[email protected]>Signed-off-by Kangjie Xu <[email protected]>---
hw/virtio/virtio-pci.c | 19 +++++++++++++++++++include/hw/virtio/virtio-pci.h| 1 +2 files changed, 20 insertions(+)hw/virtio/virtio-pci.c +0
-0@@ -1246,6 +1246,9 @@ static uint64_t virtio_pci_common_read(void *opaque, hwaddr addr,case
VIRTIO_PCI_COMMON_Q_USEDHI:
val = proxy->vqs[vdev->queue_sel].used[1]; break;
+ caseVIRTIO_PCI_COMMON_Q_RESET:
+ val = proxy->vqs[vdev->queue_sel].reset; + break;
default:
val = 0;
}hw/virtio/virtio-pci.c +0
-0@@ -1333,6 +1336,7 @@ static void virtio_pci_common_write(void *opaque, hwaddr addr,((uint64_t)proxy->vqs[vdev->queue_sel].used[1]) << 32 | proxy->vqs[vdev->queue_sel].used[0]); proxy->vqs[vdev->queue_sel].enabled = 1;
+ proxy->vqs[vdev->queue_sel].reset = 0;
} else {
virtio_error(vdev, "wrong value for queue_enable %"PRIx64, val); }hw/virtio/virtio-pci.c +0
-0@@ -1355,6 +1359,20 @@ static void virtio_pci_common_write(void *opaque, hwaddr addr,case
VIRTIO_PCI_COMMON_Q_USEDHI:
proxy->vqs[vdev->queue_sel].used[1] = val; break;
+ caseVIRTIO_PCI_COMMON_Q_RESET:
+ if (val == 1) {Quoted 9 lines
Quoted 5 lines
+ /*
+ * With the global iothread lock taken, the migration will not
+ * happen until the virtqueue reset is done. + */This comment applies to all other common cfg operation as well, So it looks not necessary?
Get it.
Quoted 32 lines
Quoted 24 lines
+ proxy->vqs[vdev->queue_sel].reset = 1;
hw/virtio/virtio-pci.c +7
-0+ + virtio_queue_reset(vdev, vdev->queue_sel); + + proxy->vqs[vdev->queue_sel].reset = 0; + proxy->vqs[vdev->queue_sel].enabled = 0; + } + break;default:
break;
}hw/virtio/virtio-pci.c +1
-0@@ -1950,6 +1968,7 @@ static void virtio_pci_reset(DeviceState *qdev) for (i = 0; i < VIRTIO_QUEUE_MAX; i++) { proxy->vqs[i].enabled = 0; + proxy->vqs[i].reset = 0; proxy->vqs[i].num = 0; proxy->vqs[i].desc[0] = proxy->vqs[i].desc[1] = 0; proxy->vqs[i].avail[0] = proxy->vqs[i].avail[1] = 0;include/hw/virtio/virtio-pci.h +0
-0include/hw/virtio/virtio-pci.h typedef struct VirtIOPCIRegion { · line 120
Quoted 4 lines
include/hw/virtio/virtio-pci.h +1
-0@@ -117,6 +117,7 @@ typedef struct VirtIOPCIRegion { typedef struct VirtIOPCIQueue { uint16_t num; bool enabled; + bool reset;Do we need to migrate this?
Quoted 7 lines
Quoted 2 lines
Thanks
I think we do not need to migrate this because we hold the global iothread lock when virtqueue reset is triggered. The migration of these device states also needs this lock.
On the other hand, the 'reset' state of virtqueue is same(is 0) before and after the process of resetting a virtqueue.
Thus, the migration will not happen when we are resetting a virtqueue and we do not to migrate it.
Ok, let's add a comment above reset to explain this.
Thanks
Quoted 8 lines
Thanks
Quoted 4 lines
Quoted 4 lines
include/hw/virtio/virtio-pci.h +0
-0uint32_t desc[2]; uint32_t avail[2]; uint32_t used[2];
PATCH v2 7/24 virtio-pci: support queue enable
In reply to Kangjie XuPCI devices support vq enable.
Based on this function, the driver can re-enable the virtqueue after the virtqueue is reset.
--- hw/virtio/virtio-pci.c | 1 + 1 file changed, 1 insertion(+)
hw/virtio/virtio-pci.c +1 -0
@@ -1335,6 +1335,7 @@ static void virtio_pci_common_write(void *opaque, hwaddr addr, proxy->vqs[vdev->queue_sel].avail[0 ], ((uint64_t )proxy->vqs[vdev->queue_sel].used[1 ]) <<32 | proxy->vqs[vdev->queue_sel].used[0]); +virtio_queue_enable (vdev, vdev->queue_sel); proxy->vqs[vdev->queue_sel].enabled =1 ; proxy->vqs[vdev->queue_sel].reset =0 ; }else {
Signature · 1 line
2.32.0
Re: [PATCH v2 07/24] virtio-pci: support queue enable
Quoted 1 line · Kangjie Xu
PCI devices support vq enable.
Nit: it might be "support device specific vq enable"
Quoted 15 lines
Based on this function, the driver can re-enable the virtqueue after the virtqueue is reset.
Signed-off-by Kangjie Xu <[email protected]>Signed-off-by Xuan Zhuo <[email protected]>--- hw/virtio/virtio-pci.c | 1 + 1 file changed, 1 insertion(+)hw/virtio/virtio-pci.c +0
-0
hw/virtio/virtio-pci.c static void virtio_pci_common_write(void *opaque, hwaddr addr, · line 1340
Quoted 6 lines
hw/virtio/virtio-pci.c +1
-0@@ -1335,6 +1335,7 @@ static void virtio_pci_common_write(void *opaque, hwaddr addr, proxy->vqs[vdev->queue_sel].avail[0 ], ((uint64_t )proxy->vqs[vdev->queue_sel].used[1 ]) <<32 | proxy->vqs[vdev->queue_sel].used[0]); +virtio_queue_enable (vdev, vdev->queue_sel); proxy->vqs[vdev->queue_sel].enabled =1 ; proxy->vqs[vdev->queue_sel].reset =0 ;
Any reason we do it before the assignment of 1? It probably means the device specific method can't depend on virtio_queue_enabled()?
Thanks
Quoted 1 line
hw/virtio/virtio-pci.c +0
-0}else {
Re: [PATCH v2 07/24] virtio-pci: support queue enable
Quoted 8 lines · Jason Wang
Quoted 1 line · Kangjie Xu
PCI devices support vq enable.
Nit: it might be "support device specific vq enable"
Get it.
Quoted 26 lines
Quoted 17 lines
Based on this function, the driver can re-enable the virtqueue after the virtqueue is reset.
Signed-off-by Kangjie Xu <[email protected]>Signed-off-by Xuan Zhuo <[email protected]>---
hw/virtio/virtio-pci.c| 1 +1 file changed, 1 insertion(+)hw/virtio/virtio-pci.c +0
-0@@ -1335,6 +1335,7 @@ static void virtio_pci_common_write(void *opaque, hwaddr addr, proxy->vqs[vdev->queue_sel].avail[0],Quoted 5 lines
((uint64_t)proxy->vqs[vdev->queue_sel].used[1]) << 32 | proxy->vqs[vdev->queue_sel].used[0]);
+virtio_queue_enable(vdev, vdev->queue_sel); proxy->vqs[vdev->queue_sel].enabled = 1; proxy->vqs[vdev->queue_sel].reset = 0;Any reason we do it before the assignment of 1? It probably means the device specific method can't depend on
virtio_queue_enabled()?
Quoted 2 lines
Thanks
Sorry, I don't get why device specific method can't depend on virtio_queue_enabled().
Before virtio_queue_enable() is done, virtqueue should always be not ready and disabled.
Otherwise, If we put it after the assignment of enabled to 1, the virtqueue may be accessed illegally and may cause panic, because the virtqueue is still being intialized and being configured.
Thanks
Quoted 2 lines
Quoted 2 lines
} else {
Re: [PATCH v2 07/24] virtio-pci: support queue enable
Quoted 37 lines · Kangjie Xu
Quoted 8 lines · Jason Wang
Quoted 1 line · Kangjie Xu
PCI devices support vq enable.
Nit: it might be "support device specific vq enable"
Get it.
Quoted 26 lines
Quoted 17 lines
Based on this function, the driver can re-enable the virtqueue after the virtqueue is reset.
Signed-off-by Kangjie Xu <[email protected]>Signed-off-by Xuan Zhuo <[email protected]>---
hw/virtio/virtio-pci.c| 1 +1 file changed, 1 insertion(+)hw/virtio/virtio-pci.c +0
-0@@ -1335,6 +1335,7 @@ static void virtio_pci_common_write(void *opaque, hwaddr addr, proxy->vqs[vdev->queue_sel].avail[0],Quoted 5 lines
((uint64_t)proxy->vqs[vdev->queue_sel].used[1]) << 32 | proxy->vqs[vdev->queue_sel].used[0]);
+virtio_queue_enable(vdev, vdev->queue_sel); proxy->vqs[vdev->queue_sel].enabled = 1; proxy->vqs[vdev->queue_sel].reset = 0;Any reason we do it before the assignment of 1? It probably means the device specific method can't depend on
virtio_queue_enabled()?
Quoted 3 lines
Quoted 2 lines
Thanks
Sorry, I don't get why device specific method can't depend on
virtio_queue_enabled().
I meant if the device specific method call virtio_queue_enabled() it will return false in this case, is this intended?
Quoted 4 lines
Before
virtio_queue_enable()is done, virtqueue should always be not ready and disabled.Otherwise, If we put it after the assignment of enabled to 1, the virtqueue may be accessed illegally and may cause panic, because the virtqueue is still being intialized and being configured.
How? Shouldn't we make transport ready before making device virtqueue(device) ready?
Thanks
Quoted 6 lines
Thanks
Quoted 2 lines
Quoted 2 lines
} else {
Re: [PATCH v2 07/24] virtio-pci: support queue enable
Quoted 39 lines · Jason Wang
Quoted 37 lines · Kangjie Xu
Quoted 8 lines · Jason Wang
Quoted 1 line · Kangjie Xu
PCI devices support vq enable.
Nit: it might be "support device specific vq enable"
Get it.
Quoted 26 lines
Quoted 17 lines
Based on this function, the driver can re-enable the virtqueue after the virtqueue is reset.
Signed-off-by Kangjie Xu <[email protected]>Signed-off-by Xuan Zhuo <[email protected]>---
hw/virtio/virtio-pci.c| 1 +1 file changed, 1 insertion(+)hw/virtio/virtio-pci.c +0
-0@@ -1335,6 +1335,7 @@ static void virtio_pci_common_write(void *opaque, hwaddr addr, proxy->vqs[vdev->queue_sel].avail[0],Quoted 5 lines
((uint64_t)proxy->vqs[vdev->queue_sel].used[1]) << 32 | proxy->vqs[vdev->queue_sel].used[0]);
+virtio_queue_enable(vdev, vdev->queue_sel); proxy->vqs[vdev->queue_sel].enabled = 1; proxy->vqs[vdev->queue_sel].reset = 0;Any reason we do it before the assignment of 1? It probably means the device specific method can't depend on
virtio_queue_enabled()?
Quoted 7 lines
Quoted 3 lines
Quoted 2 lines
Thanks
Sorry, I don't get why device specific method can't depend on
virtio_queue_enabled().I meant if the device specific method call
virtio_queue_enabled()it will return false in this case, is this intended?
Yes, I intend it to behave in this way.
Quoted 11 lines
Quoted 5 lines
Before
virtio_queue_enable()is done, virtqueue should always be not ready and disabled.Otherwise, If we put it after the assignment of enabled to 1, the virtqueue may be accessed illegally and may cause panic, because the virtqueue is still being intialized and being configured.
How? Shouldn't we make transport ready before making device virtqueue(device) ready?
Thanks
I am not experienced in this field, could you tell me why we should make the transport ready first?
I make the transport ready later than making device ready for two aspects:
1. In QEMU, the virtio_queue_enabled() is used only when we start the device/queue pair (vhost_dev_start_one), or reading VIRTIO_PCI_COMMON_Q_ENABLE. These two operations and resetting the queue will *be synchronized* using iothread lock, so we do not need to worry about the case currently.
2. Suppose we use virtio_queue_enabled() or access the enabled status asynchronously, and we make the transport ready first.
After enabled set to true, and before virtio_queue_enable() is finished, somewhere that call virtio_queue_enabled() or access the enabled status of VirtIOPCIQueue. Then the caller will consider the virtqueue as enabled(enabled = true in VirtIOPCIQueue). The caller might access the virtqueue(access avail ring / desc table). But I think *the access here is illegal* because the virtqueue might still be *unintialized* status.
Thus, from my perspective, to prevent illegal access, we need to make transport ready after virtio_queue_enable().
Thanks
Quoted 7 lines
Quoted 7 lines
Thanks
Quoted 2 lines
Quoted 2 lines
} else {
Re: [PATCH v2 07/24] virtio-pci: support queue enable
Quoted 66 lines · Kangjie Xu
在 2022/8/24 16:59, Jason Wang 写道:
在 2022/8/23 16:20, Kangjie Xu 写道:
在 2022/8/23 15:44, Jason Wang 写道:
在 2022/8/16 09:06, Kangjie Xu 写道:
PCI devices support vq enable.
Nit: it might be "support device specific vq enable"
Get it.
Based on this function, the driver can re-enable the virtqueue after the virtqueue is reset.
Signed-off-by Kangjie Xu <[email protected]>Signed-off-by Xuan Zhuo <[email protected]>--- hw/virtio/virtio-pci.c | 1 + 1 file changed, 1 insertion(+)hw/virtio/virtio-pci.c +0
-0@@ -1335,6 +1335,7 @@ static void virtio_pci_common_write(void *opaque, hwaddr addr, proxy->vqs[vdev->queue_sel].avail[0],((uint64_t)proxy->vqs[vdev->queue_sel].used[1]) << 32 | proxy->vqs[vdev->queue_sel].used[0]);
+ virtio_queue_enable(vdev, vdev->queue_sel); proxy->vqs[vdev->queue_sel].enabled = 1; proxy->vqs[vdev->queue_sel].reset = 0;Any reason we do it before the assignment of 1? It probably means the device specific method can't depend on
virtio_queue_enabled()?Thanks
Sorry, I don't get why device specific method can't depend on
virtio_queue_enabled().I meant if the device specific method call
virtio_queue_enabled()it will return false in this case, is this intended?Yes, I intend it to behave in this way.
Before
virtio_queue_enable()is done, virtqueue should always be not ready and disabled.Otherwise, If we put it after the assignment of enabled to 1, the virtqueue may be accessed illegally and may cause panic, because the virtqueue is still being intialized and being configured.
Quoted 5 lines
How? Shouldn't we make transport ready before making device virtqueue(device) ready?
Thanks
I am not experienced in this field, could you tell me why we should make the transport ready first?
That's a must for the device to work.
E.g for PCI device, I can't image the device is ready to work before PCI is ready.
Quoted 4 lines
I make the transport ready later than making device ready for two aspects:
1. In QEMU, the
virtio_queue_enabled()is used only when we start the device/queue pair (vhost_dev_start_one), or readingVIRTIO_PCI_COMMON_Q_ENABLE. These two operations and resetting the queue will be synchronized using iothread lock, so we do not need to worry about the case currently.
Note that virtio_queue_enabled() is an exported helper, you can't assume how it will be used in the future.
Quoted 5 lines
2. Suppose we use
virtio_queue_enabled()or access the enabled status asynchronously, and we make the transport ready first.After enabled set to true, and before
virtio_queue_enable()is finished, somewhere that callvirtio_queue_enabled()or access the enabled status of VirtIOPCIQueue. Then the caller will consider the virtqueue as enabled(enabled = true in VirtIOPCIQueue). The caller might access the virtqueue(access avail ring / desc table). But I think the access here is illegal because the virtqueue might still be unintialized status.
How can this happen, won't we call device specific method after we set queue_enabled as true? It's the charge of the device specific method to synchronize the necessary external I/O in this case.
Thanks
Quoted 12 lines
Thus, from my perspective, to prevent illegal access, we need to make transport ready after
virtio_queue_enable().Thanks
Thanks
} else {
Re: [PATCH v2 07/24] virtio-pci: support queue enable
Quoted 84 lines · Jason Wang
Quoted 65 lines · Kangjie Xu
在 2022/8/24 16:59, Jason Wang 写道:
在 2022/8/23 16:20, Kangjie Xu 写道:
在 2022/8/23 15:44, Jason Wang 写道:
在 2022/8/16 09:06, Kangjie Xu 写道:
PCI devices support vq enable.
Nit: it might be "support device specific vq enable"
Get it.
Based on this function, the driver can re-enable the virtqueue after the virtqueue is reset.
Signed-off-by Kangjie Xu <[email protected]>Signed-off-by Xuan Zhuo <[email protected]>--- hw/virtio/virtio-pci.c | 1 + 1 file changed, 1 insertion(+)hw/virtio/virtio-pci.c +0
-0@@ -1335,6 +1335,7 @@ static void virtio_pci_common_write(void *opaque, hwaddr addr, proxy->vqs[vdev->queue_sel].avail[0],((uint64_t)proxy->vqs[vdev->queue_sel].used[1]) << 32 | proxy->vqs[vdev->queue_sel].used[0]);
+ virtio_queue_enable(vdev, vdev->queue_sel); proxy->vqs[vdev->queue_sel].enabled = 1; proxy->vqs[vdev->queue_sel].reset = 0;Any reason we do it before the assignment of 1? It probably means the device specific method can't depend on
virtio_queue_enabled()?Thanks
Sorry, I don't get why device specific method can't depend on
virtio_queue_enabled().I meant if the device specific method call
virtio_queue_enabled()it will return false in this case, is this intended?Yes, I intend it to behave in this way.
Before
virtio_queue_enable()is done, virtqueue should always be not ready and disabled.Otherwise, If we put it after the assignment of enabled to 1, the virtqueue may be accessed illegally and may cause panic, because the virtqueue is still being intialized and being configured.
Quoted 5 lines
How? Shouldn't we make transport ready before making device virtqueue(device) ready?
Thanks
I am not experienced in this field, could you tell me why we should make the transport ready first?
That's a must for the device to work.
E.g for PCI device, I can't image the device is ready to work before PCI is ready.
Quoted 3 lines
I make the transport ready later than making device ready for two aspects:
1. In QEMU, the
virtio_queue_enabled()is used only when we start the device/queue pair (vhost_dev_start_one), or readingVIRTIO_PCI_COMMON_Q_ENABLE. These two operations and resetting the queue will be synchronized using iothread lock, so we do not need to worry about the case currently.Note that
virtio_queue_enabled()is an exported helper, you can't assume how it will be used in the future.Quoted 2 lines
2. Suppose we use
virtio_queue_enabled()or access the enabled status asynchronously, and we make the transport ready first.
Quoted 7 lines
Quoted 2 lines
After enabled set to true, and before
virtio_queue_enable()is finished, somewhere that callvirtio_queue_enabled()or access the enabled status of VirtIOPCIQueue. Then the caller will consider the virtqueue as enabled(enabled = true in VirtIOPCIQueue). The caller might access the virtqueue(access avail ring / desc table). But I think the access here is illegal because the virtqueue might still be unintialized status.How can this happen, won't we call device specific method after we set
queue_enabledas true? It's the charge of the device specific method to synchronize the necessary external I/O in this case.Thanks
I get your point.
You are right. I realized that I ignored the external I/O.
Will fix it.
Thanks
Quoted 13 lines
Quoted 13 lines
Thus, from my perspective, to prevent illegal access, we need to make transport ready after
virtio_queue_enable().Thanks
Thanks
} else {
PATCH v2 8/24 vhost: extract the logic of unmapping the vrings and desc
In reply to Kangjie XuIntroduce vhost_virtqueue_unmap() to ummap the vrings and desc of a virtqueue.
The function will be used later.
--- hw/virtio/vhost.c | 20 ++++++++++++++------ 1 file changed, 14 insertions(+), 6 deletions(-)
hw/virtio/vhost.c +14 -6
@@ -1197,6 +1197,19 @@ fail_alloc_desc:return r; } +static void vhost_virtqueue_unmap (struct vhost_dev *dev, +struct VirtIODevice *vdev, +struct vhost_virtqueue *vq, +unsigned idx) +{ +vhost_memory_unmap (dev, vq->used,virtio_queue_get_used_size (vdev, idx), +1 ,virtio_queue_get_used_size (vdev, idx)); +vhost_memory_unmap (dev, vq->avail,virtio_queue_get_avail_size (vdev, idx), +0 ,virtio_queue_get_avail_size (vdev, idx)); +vhost_memory_unmap (dev, vq->desc,virtio_queue_get_desc_size (vdev, idx), +0 ,virtio_queue_get_desc_size (vdev, idx)); +} + static void vhost_virtqueue_stop(struct vhost_dev *dev, struct VirtIODevice *vdev, struct vhost_virtqueue *vq,
@@ -1235,12 +1248,7 @@ static void vhost_virtqueue_stop(struct vhost_dev *dev, vhost_vq_index); } -vhost_memory_unmap (dev, vq->used,virtio_queue_get_used_size (vdev, idx), -1 ,virtio_queue_get_used_size (vdev, idx)); -vhost_memory_unmap (dev, vq->avail,virtio_queue_get_avail_size (vdev, idx), -0 ,virtio_queue_get_avail_size (vdev, idx)); -vhost_memory_unmap (dev, vq->desc,virtio_queue_get_desc_size (vdev, idx), -0 ,virtio_queue_get_desc_size (vdev, idx)); +vhost_virtqueue_unmap (dev, vdev, vq, idx); } static void vhost_eventfd_add(MemoryListener *listener,
Signature · 1 line
2.32.0
Re: [PATCH v2 08/24] vhost: extract the logic of unmapping the vrings and desc
Quoted 7 lines · Kangjie Xu
Introduce
vhost_virtqueue_unmap()to ummap the vrings and desc of a virtqueue.The function will be used later.
Signed-off-by Kangjie Xu <[email protected]>Signed-off-by Xuan Zhuo <[email protected]>
Quoted 42 lines
--- hw/virtio/vhost.c | 20 ++++++++++++++------ 1 file changed, 14 insertions(+), 6 deletions(-)hw/virtio/vhost.c +14
-6@@ -1197,6 +1197,19 @@ fail_alloc_desc:return r; } +static void vhost_virtqueue_unmap (struct vhost_dev *dev, +struct VirtIODevice *vdev, +struct vhost_virtqueue *vq, +unsigned idx) +{ +vhost_memory_unmap (dev, vq->used,virtio_queue_get_used_size (vdev, idx), +1 ,virtio_queue_get_used_size (vdev, idx)); +vhost_memory_unmap (dev, vq->avail,virtio_queue_get_avail_size (vdev, idx), +0 ,virtio_queue_get_avail_size (vdev, idx)); +vhost_memory_unmap (dev, vq->desc,virtio_queue_get_desc_size (vdev, idx), +0 ,virtio_queue_get_desc_size (vdev, idx)); +} + static void vhost_virtqueue_stop(struct vhost_dev *dev, struct VirtIODevice *vdev, struct vhost_virtqueue *vq,@@ -1235,12 +1248,7 @@ static void vhost_virtqueue_stop(struct vhost_dev *dev, vhost_vq_index); } -vhost_memory_unmap (dev, vq->used,virtio_queue_get_used_size (vdev, idx), -1 ,virtio_queue_get_used_size (vdev, idx)); -vhost_memory_unmap (dev, vq->avail,virtio_queue_get_avail_size (vdev, idx), -0 ,virtio_queue_get_avail_size (vdev, idx)); -vhost_memory_unmap (dev, vq->desc,virtio_queue_get_desc_size (vdev, idx), -0 ,virtio_queue_get_desc_size (vdev, idx)); +vhost_virtqueue_unmap (dev, vdev, vq, idx); } static void vhost_eventfd_add(MemoryListener *listener,
PATCH v2 9/24 vhost: introduce vhost_dev_virtqueue_stop()
In reply to Kangjie XuIntroduce vhost_dev_virtqueue_stop(), which can ummap the vrings and the desc of it.
--- hw/virtio/vhost.c | 9 +++++++++ include/hw/virtio/vhost.h | 3 +++ 2 files changed, 12 insertions(+)
hw/virtio/vhost.c +9 -0
@@ -1904,3 +1904,12 @@ int vhost_net_set_backend(struct vhost_dev *hdev,return -ENOSYS ; } + +void vhost_dev_virtqueue_stop (struct vhost_dev *hdev,VirtIODevice *vdev, +int idx) +{ +vhost_virtqueue_unmap (hdev, + vdev, + hdev->vqs + idx, + idx); +}
include/hw/virtio/vhost.h +3 -0
@@ -288,4 +288,7 @@ int vhost_dev_set_inflight(struct vhost_dev *dev, struct vhost_inflight *inflight);int vhost_dev_get_inflight (struct vhost_dev *dev,uint16_t queue_size,struct vhost_inflight *inflight); + +void vhost_dev_virtqueue_stop (struct vhost_dev *hdev,VirtIODevice *vdev, +int idx);#endif
Signature · 1 line
2.32.0
Re: [PATCH v2 09/24] vhost: introduce vhost_dev_virtqueue_stop()
Quoted 16 lines · Kangjie Xu
Introduce
vhost_dev_virtqueue_stop(), which can ummap the vrings and the desc of it.Signed-off-by Kangjie Xu <[email protected]>Signed-off-by Xuan Zhuo <[email protected]>--- hw/virtio/vhost.c | 9 +++++++++ include/hw/virtio/vhost.h | 3 +++ 2 files changed, 12 insertions(+)hw/virtio/vhost.c +0
-0@@ -1904,3 +1904,12 @@ int vhost_net_set_backend(struct vhost_dev *hdev,
hw/virtio/vhost.c int vhost_net_set_backend(struct vhost_dev *hdev, · line 1914
Quoted 10 lines
hw/virtio/vhost.c +8
-0return -ENOSYS ; } + +void vhost_dev_virtqueue_stop (struct vhost_dev *hdev,VirtIODevice *vdev, +int idx) +{ +vhost_virtqueue_unmap (hdev, + vdev, + hdev->vqs + idx, + idx);
So I think the unmap is not sufficient, we need backend specific support. E.g for vhost kernel, need a SET_BACKEND here?
Thanks
Quoted 13 lines
hw/virtio/vhost.c +1
-0+}include/hw/virtio/vhost.h +3
-0@@ -288,4 +288,7 @@ int vhost_dev_set_inflight(struct vhost_dev *dev, struct vhost_inflight *inflight);int vhost_dev_get_inflight (struct vhost_dev *dev,uint16_t queue_size,struct vhost_inflight *inflight); + +void vhost_dev_virtqueue_stop (struct vhost_dev *hdev,VirtIODevice *vdev, +int idx);#endif
Re: [PATCH v2 09/24] vhost: introduce vhost_dev_virtqueue_stop()
Quoted 32 lines · Jason Wang
Quoted 16 lines · Kangjie Xu
Introduce
vhost_dev_virtqueue_stop(), which can ummap the vrings and the desc of it.Signed-off-by Kangjie Xu <[email protected]>Signed-off-by Xuan Zhuo <[email protected]>---
hw/virtio/vhost.c | 9 +++++++++
include/hw/virtio/vhost.h| 3 +++2 files changed, 12 insertions(+)hw/virtio/vhost.c +0
-0@@ -1904,3 +1904,12 @@ int vhost_net_set_backend(struct vhost_dev *hdev, return -ENOSYS;hw/virtio/vhost.c int vhost_net_set_backend(struct vhost_dev *hdev, · line 1914
Quoted 10 lines
hw/virtio/vhost.c +8
-0} + +void vhost_dev_virtqueue_stop (struct vhost_dev *hdev,VirtIODevice *vdev, + int idx) +{ + vhost_virtqueue_unmap(hdev, + vdev, + hdev->vqs + idx, + idx);So I think the unmap is not sufficient, we need backend specific support. E.g for vhost kernel, need a
SET_BACKENDhere?
Quoted 2 lines
Thanks
But SET_BACKEND of vhost-net needs a parameter fd in vhost_vring_file: that is net->backend of VHostNetState.
If we add the fd parameter or struct vhost_vring_file to vhost_dev_virtqueue_stop/restart, it exposes some implementation details in the parameter list.
And that seems not good? So I put SET_BACKEND in the vhost-net module. The workflow is similar to vhost_net_start_one().
Thanks
Quoted 15 lines
Quoted 15 lines
hw/virtio/vhost.c +1
-0+}include/hw/virtio/vhost.h +3
-0@@ -288,4 +288,7 @@ int vhost_dev_set_inflight(struct vhost_dev *dev, struct vhost_inflight *inflight); int vhost_dev_get_inflight(struct vhost_dev *dev, uint16_t queue_size, struct vhost_inflight *inflight); + +void vhost_dev_virtqueue_stop (struct vhost_dev *hdev,VirtIODevice *vdev, + int idx);#endif
Re: [PATCH v2 09/24] vhost: introduce vhost_dev_virtqueue_stop()
Quoted 35 lines · Kangjie Xu
Quoted 33 lines · Jason Wang
Quoted 20 lines · Kangjie Xu
Introduce
vhost_dev_virtqueue_stop(), which can ummap the vrings and the desc of it.Signed-off-by Kangjie Xu <[email protected]>Signed-off-by Xuan Zhuo <[email protected]>---
hw/virtio/vhost.c | 9 +++++++++
include/hw/virtio/vhost.h| 3 +++2 files changed, 12 insertions(+)hw/virtio/vhost.c +2
-0@@ -1904,3 +1904,12 @@ int vhost_net_set_backend(struct vhost_dev *hdev, return -ENOSYS; } + +void vhost_dev_virtqueue_stop(struct vhost_dev *hdev, VirtIODeviceQuoted 7 lines
*vdev,
+ int idx)
+{
+ vhost_virtqueue_unmap(hdev,
+ vdev,
+hdev->vqs+ idx,
+ idx);So I think the unmap is not sufficient, we need backend specific support. E.g for vhost kernel, need a
SET_BACKENDhere?
Quoted 7 lines
Quoted 2 lines
Thanks
But
SET_BACKENDof vhost-net needs a parameter fd invhost_vring_file: that isnet->backendof VHostNetState.If we add the fd parameter or struct
vhost_vring_fileto vhost_dev_virtqueue_stop/restart, it exposes some implementation details in the parameter list.And that seems not good? So I put
SET_BACKENDin the vhost-net module. The workflow is similar tovhost_net_start_one().
That looks fine.
Thanks
Quoted 19 lines
Thanks
Quoted 15 lines
Quoted 15 lines
hw/virtio/vhost.c +1
-0+}include/hw/virtio/vhost.h +3
-0@@ -288,4 +288,7 @@ int vhost_dev_set_inflight(struct vhost_dev *dev, struct vhost_inflight *inflight); int vhost_dev_get_inflight(struct vhost_dev *dev, uint16_t queue_size, struct vhost_inflight *inflight); + +void vhost_dev_virtqueue_stop (struct vhost_dev *hdev,VirtIODevice *vdev, + int idx);#endif
PATCH v2 10/24 vhost: introduce vhost_dev_virtqueue_restart()
In reply to Kangjie XuIntroduce vhost_dev_virtqueue_restart(), which can restart the virtqueue when the vhost has already started running.
--- hw/virtio/vhost.c | 13 +++++++++++++ include/hw/virtio/vhost.h | 2 ++ 2 files changed, 15 insertions(+)
hw/virtio/vhost.c +13 -0
@@ -1913,3 +1913,16 @@ void vhost_dev_virtqueue_stop(struct vhost_dev *hdev, VirtIODevice *vdev, hdev->vqs + idx, idx); } + +int vhost_dev_virtqueue_restart (struct vhost_dev *hdev,VirtIODevice *vdev, +int idx) +{ +const VhostOps *vhost_ops = hdev->vhost_ops; + +assert (vhost_ops); + +return vhost_virtqueue_start (hdev, + vdev, + hdev->vqs + idx, + hdev->vq_index + idx); +}
include/hw/virtio/vhost.h +2 -0
@@ -291,4 +291,6 @@ int vhost_dev_get_inflight(struct vhost_dev *dev, uint16_t queue_size,void vhost_dev_virtqueue_stop (struct vhost_dev *hdev,VirtIODevice *vdev,int idx); +int vhost_dev_virtqueue_restart (struct vhost_dev *hdev,VirtIODevice *vdev, +int idx);#endif
Signature · 1 line
2.32.0
Re: [PATCH v2 10/24] vhost: introduce vhost_dev_virtqueue_restart()
Quoted 15 lines · Kangjie Xu
Introduce
vhost_dev_virtqueue_restart(), which can restart the virtqueue when the vhost has already started running.Signed-off-by Kangjie Xu <[email protected]>Signed-off-by Xuan Zhuo <[email protected]>--- hw/virtio/vhost.c | 13 +++++++++++++ include/hw/virtio/vhost.h | 2 ++ 2 files changed, 15 insertions(+)hw/virtio/vhost.c +0
-0
hw/virtio/vhost.c void vhost_dev_virtqueue_stop(struct vhost_dev *hdev, VirtIODevice *vdev, · line 1922
Quoted 10 lines
hw/virtio/vhost.c +7
-0@@ -1913,3 +1913,16 @@ void vhost_dev_virtqueue_stop(struct vhost_dev *hdev, VirtIODevice *vdev, hdev->vqs + idx, idx); } + +int vhost_dev_virtqueue_restart (struct vhost_dev *hdev,VirtIODevice *vdev, +int idx) +{ +const VhostOps *vhost_ops = hdev->vhost_ops; + +assert (vhost_ops);
So we had the comment like:
/* should only be called after backend is connected */
in vhost_virtqueue_mask().
If this assert has the same reason, let's add a comment here.
hw/virtio/vhost.c void vhost_dev_virtqueue_stop(struct vhost_dev *hdev, VirtIODevice *vdev, · line 1927
Quoted 5 lines
hw/virtio/vhost.c +5
-0+ +return vhost_virtqueue_start (hdev, + vdev, + hdev->vqs + idx, + hdev->vq_index + idx);
So it just a wrapper of vhost_virtqueue_start(), any value to have a re-start wrapper?
Thanks
Quoted 12 lines
hw/virtio/vhost.c +1
-0+}include/hw/virtio/vhost.h +2
-0@@ -291,4 +291,6 @@ int vhost_dev_get_inflight(struct vhost_dev *dev, uint16_t queue_size,void vhost_dev_virtqueue_stop (struct vhost_dev *hdev,VirtIODevice *vdev,int idx); +int vhost_dev_virtqueue_restart (struct vhost_dev *hdev,VirtIODevice *vdev, +int idx);#endif
Re: [PATCH v2 10/24] vhost: introduce vhost_dev_virtqueue_restart()
Quoted 33 lines · Jason Wang
Quoted 23 lines · Kangjie Xu
Introduce
vhost_dev_virtqueue_restart(), which can restart the virtqueue when the vhost has already started running.Signed-off-by Kangjie Xu <[email protected]>Signed-off-by Xuan Zhuo <[email protected]>---
hw/virtio/vhost.c | 13 +++++++++++++
include/hw/virtio/vhost.h| 2 ++2 files changed, 15 insertions(+)hw/virtio/vhost.c +0
-0@@ -1913,3 +1913,16 @@ void vhost_dev_virtqueue_stop(struct vhost_dev *hdev, VirtIODevice *vdev, hdev->vqs + idx, idx);}
hw/virtio/vhost.c +2
-0+ +int vhost_dev_virtqueue_restart(struct vhost_dev *hdev, VirtIODevice*vdev,
+ int idx)hw/virtio/vhost.c void vhost_dev_virtqueue_stop(struct vhost_dev *hdev, VirtIODevice *vdev, · line 1922
Quoted 4 lines
hw/virtio/vhost.c +4
-0+{ + const VhostOps *vhost_ops = hdev->vhost_ops; + + assert(vhost_ops);So we had the comment like:
Quoted 7 lines
/* should only be called after backend is connected */
in
vhost_virtqueue_mask().If this assert has the same reason, let's add a comment here.
Get it.
Quoted 11 lines
Quoted 5 lines
+
+ return vhost_virtqueue_start(hdev,
+ vdev,
+hdev->vqs+ idx, +hdev->vq_index+ idx);So it just a wrapper of
vhost_virtqueue_start(), any value to have a re-start wrapper?Thanks
Because in subsequent patches, to support vhost-user, we will add vhost_ops->vhost_set_single_vring_enable() in this function.
Thanks.
Quoted 14 lines
Quoted 14 lines
hw/virtio/vhost.c +1
-0+}include/hw/virtio/vhost.h +2
-0@@ -291,4 +291,6 @@ int vhost_dev_get_inflight(struct vhost_dev *dev,uint16_t queue_size, void vhost_dev_virtqueue_stop(struct vhost_dev *hdev, VirtIODevice *vdev, int idx); +int vhost_dev_virtqueue_restart (struct vhost_dev *hdev,VirtIODevice *vdev, + int idx);#endif
PATCH v2 11/24 vhost-net: vhost-kernel: introduce vhost_net_virtqueue_stop()
In reply to Kangjie XuIntroduce vhost_virtqueue_stop(), which can reset the virtqueue in the device. Then it will unmap vrings and the desc of the virtqueue.
This patch only considers the case for vhost-kernel, when NetClientDriver is NET_CLIENT_DRIVER_TAP.
--- hw/net/vhost_net.c | 21 +++++++++++++++++++++ include/net/vhost_net.h | 2 ++ 2 files changed, 23 insertions(+)
hw/net/vhost_net.c +21 -0
@@ -514,3 +514,24 @@ int vhost_net_set_mtu(struct vhost_net *net, uint16_t mtu)return vhost_ops->vhost_net_set_mtu (&net->dev, mtu); } + +void vhost_net_virtqueue_stop (VirtIODevice *vdev,NetClientState *nc, +int vq_index) +{ +VHostNetState *net =get_vhost_net (nc->peer); +const VhostOps *vhost_ops = net->dev.vhost_ops; +struct vhost_vring_file file = { .fd =-1 }; +int idx; + +assert (vhost_ops); + + idx = vhost_ops->vhost_get_vq_index (&net->dev, vq_index); + +if (net->nc->info->type ==NET_CLIENT_DRIVER_TAP ) { + file.index = idx; +int r =vhost_net_set_backend (&net->dev, &file); +assert (r >=0 ); + } + +vhost_dev_virtqueue_stop (&net->dev, vdev, idx); +}
include/net/vhost_net.h +2 -0
@@ -48,4 +48,6 @@ uint64_t vhost_net_get_acked_features(VHostNetState *net);int vhost_net_set_mtu (struct vhost_net *net,uint16_t mtu); +void vhost_net_virtqueue_stop (VirtIODevice *vdev,NetClientState *nc, +int vq_index);#endif
Signature · 1 line
2.32.0
Re: [PATCH v2 11/24] vhost-net: vhost-kernel: introduce vhost_net_virtqueue_stop()
Quoted 30 lines · Kangjie Xu
Introduce
vhost_virtqueue_stop(), which can reset the virtqueue in the device. Then it will unmap vrings and the desc of the virtqueue.This patch only considers the case for vhost-kernel, when NetClientDriver is
NET_CLIENT_DRIVER_TAP.Signed-off-by Kangjie Xu <[email protected]>Signed-off-by Xuan Zhuo <[email protected]>--- hw/net/vhost_net.c | 21 +++++++++++++++++++++ include/net/vhost_net.h | 2 ++ 2 files changed, 23 insertions(+)hw/net/vhost_net.c +8
-0@@ -514,3 +514,24 @@ int vhost_net_set_mtu(struct vhost_net *net, uint16_t mtu)return vhost_ops->vhost_net_set_mtu (&net->dev, mtu); } + +void vhost_net_virtqueue_stop (VirtIODevice *vdev,NetClientState *nc, +int vq_index) +{ +VHostNetState *net =get_vhost_net (nc->peer); +const VhostOps *vhost_ops = net->dev.vhost_ops; +struct vhost_vring_file file = { .fd =-1 }; +int idx;
hw/net/vhost_net.c int vhost_net_set_mtu(struct vhost_net *net, uint16_t mtu) · line 534
Quoted 10 lines
hw/net/vhost_net.c +10
-0+ +assert (vhost_ops); + + idx = vhost_ops->vhost_get_vq_index (&net->dev, vq_index); + +if (net->nc->info->type ==NET_CLIENT_DRIVER_TAP ) { + file.index = idx; +int r =vhost_net_set_backend (&net->dev, &file); +assert (r >=0 ); + }
Let's have a vhost_ops here instead of open code it.
Thanks
Quoted 14 lines
hw/net/vhost_net.c +3
-0+ +vhost_dev_virtqueue_stop (&net->dev, vdev, idx); +}include/net/vhost_net.h +2
-0@@ -48,4 +48,6 @@ uint64_t vhost_net_get_acked_features(VHostNetState *net);int vhost_net_set_mtu (struct vhost_net *net,uint16_t mtu); +void vhost_net_virtqueue_stop (VirtIODevice *vdev,NetClientState *nc, +int vq_index);#endif
Re: [PATCH v2 11/24] vhost-net: vhost-kernel: introduce vhost_net_virtqueue_stop()
Quoted 46 lines · Jason Wang
Quoted 30 lines · Kangjie Xu
Introduce
vhost_virtqueue_stop(), which can reset the virtqueue in the device. Then it will unmap vrings and the desc of the virtqueue.This patch only considers the case for vhost-kernel, when NetClientDriver is
NET_CLIENT_DRIVER_TAP.Signed-off-by Kangjie Xu <[email protected]>Signed-off-by Xuan Zhuo <[email protected]>---
hw/net/vhost_net.c | 21 +++++++++++++++++++++include/net/vhost_net.h| 2 ++2 files changed, 23 insertions(+)hw/net/vhost_net.c +8
-0@@ -514,3 +514,24 @@ int vhost_net_set_mtu(struct vhost_net *net, uint16_t mtu) return vhost_ops->vhost_net_set_mtu(&net->dev, mtu); } + +void vhost_net_virtqueue_stop (VirtIODevice *vdev,NetClientState *nc, + int vq_index) +{ + VHostNetState *net = get_vhost_net(nc->peer); + const VhostOps *vhost_ops = net->dev.vhost_ops; + struct vhost_vring_file file = { .fd = -1 }; + int idx;hw/net/vhost_net.c int vhost_net_set_mtu(struct vhost_net *net, · line 534
Quoted 10 lines
hw/net/vhost_net.c +10
-0+ + assert(vhost_ops); + + idx = vhost_ops->vhost_get_vq_index(&net->dev, vq_index); + + if (net->nc->info->type == NET_CLIENT_DRIVER_TAP) { + file.index = idx; + int r = vhost_net_set_backend(&net->dev, &file); + assert(r >= 0); + }Let's have a
vhost_opshere instead of open code it.
Quoted 2 lines
Thanks
I share your view.
Thanks
Quoted 14 lines
Quoted 14 lines
hw/net/vhost_net.c +3
-0+ + vhost_dev_virtqueue_stop(&net->dev, vdev, idx); +}include/net/vhost_net.h +1
-0@@ -48,4 +48,6 @@ uint64_t vhost_net_get_acked_features(VHostNetState *net); int vhost_net_set_mtu(struct vhost_net *net, uint16_t mtu); +void vhost_net_virtqueue_stop(VirtIODevice *vdev, NetClientState *nc, + int vq_index); #endif
Re: [PATCH v2 11/24] vhost-net: vhost-kernel: introduce vhost_net_virtqueue_stop()
In reply to Jason WangQuoted 46 lines · Jason Wang
Quoted 30 lines · Kangjie Xu
Introduce
vhost_virtqueue_stop(), which can reset the virtqueue in the device. Then it will unmap vrings and the desc of the virtqueue.This patch only considers the case for vhost-kernel, when NetClientDriver is
NET_CLIENT_DRIVER_TAP.Signed-off-by Kangjie Xu <[email protected]>Signed-off-by Xuan Zhuo <[email protected]>---
hw/net/vhost_net.c | 21 +++++++++++++++++++++include/net/vhost_net.h| 2 ++2 files changed, 23 insertions(+)hw/net/vhost_net.c +8
-0@@ -514,3 +514,24 @@ int vhost_net_set_mtu(struct vhost_net *net, uint16_t mtu) return vhost_ops->vhost_net_set_mtu(&net->dev, mtu); } + +void vhost_net_virtqueue_stop (VirtIODevice *vdev,NetClientState *nc, + int vq_index) +{ + VHostNetState *net = get_vhost_net(nc->peer); + const VhostOps *vhost_ops = net->dev.vhost_ops; + struct vhost_vring_file file = { .fd = -1 }; + int idx;hw/net/vhost_net.c int vhost_net_set_mtu(struct vhost_net *net, · line 534
Quoted 10 lines
hw/net/vhost_net.c +10
-0+ + assert(vhost_ops); + + idx = vhost_ops->vhost_get_vq_index(&net->dev, vq_index); + + if (net->nc->info->type == NET_CLIENT_DRIVER_TAP) { + file.index = idx; + int r = vhost_net_set_backend(&net->dev, &file); + assert(r >= 0); + }Let's have a
vhost_opshere instead of open code it.
Quoted 2 lines
Thanks
I double-checked it, vhost_net_set_backend is already a wrapper of vhost_ops->vhost_net_set_backend().
It seems that, to further simplify it, we can only add idx and fd to the parameter list of vhost_net_set_backend().
Thanks
Quoted 14 lines
Quoted 14 lines
hw/net/vhost_net.c +3
-0+ + vhost_dev_virtqueue_stop(&net->dev, vdev, idx); +}include/net/vhost_net.h +1
-0@@ -48,4 +48,6 @@ uint64_t vhost_net_get_acked_features(VHostNetState *net); int vhost_net_set_mtu(struct vhost_net *net, uint16_t mtu); +void vhost_net_virtqueue_stop(VirtIODevice *vdev, NetClientState *nc, + int vq_index); #endif
Re: [PATCH v2 11/24] vhost-net: vhost-kernel: introduce vhost_net_virtqueue_stop()
Quoted 48 lines · Kangjie Xu
Quoted 46 lines · Jason Wang
Quoted 30 lines · Kangjie Xu
Introduce
vhost_virtqueue_stop(), which can reset the virtqueue in the device. Then it will unmap vrings and the desc of the virtqueue.This patch only considers the case for vhost-kernel, when NetClientDriver is
NET_CLIENT_DRIVER_TAP.Signed-off-by Kangjie Xu <[email protected]>Signed-off-by Xuan Zhuo <[email protected]>---
hw/net/vhost_net.c | 21 +++++++++++++++++++++include/net/vhost_net.h| 2 ++2 files changed, 23 insertions(+)hw/net/vhost_net.c +8
-0@@ -514,3 +514,24 @@ int vhost_net_set_mtu(struct vhost_net *net, uint16_t mtu) return vhost_ops->vhost_net_set_mtu(&net->dev, mtu); } + +void vhost_net_virtqueue_stop (VirtIODevice *vdev,NetClientState *nc, + int vq_index) +{ + VHostNetState *net = get_vhost_net(nc->peer); + const VhostOps *vhost_ops = net->dev.vhost_ops; + struct vhost_vring_file file = { .fd = -1 }; + int idx;hw/net/vhost_net.c int vhost_net_set_mtu(struct vhost_net *net, · line 534
Quoted 10 lines
hw/net/vhost_net.c +10
-0+ + assert(vhost_ops); + + idx = vhost_ops->vhost_get_vq_index(&net->dev, vq_index); + + if (net->nc->info->type == NET_CLIENT_DRIVER_TAP) { + file.index = idx; + int r = vhost_net_set_backend(&net->dev, &file); + assert(r >= 0); + }Let's have a
vhost_opshere instead of open code it.
Quoted 5 lines
Quoted 2 lines
Thanks
I double-checked it,
vhost_net_set_backendis already a wrapper ofvhost_ops->vhost_net_set_backend().It seems that, to further simplify it, we can only add idx and fd to the parameter list of
vhost_net_set_backend().
Ok, so we can leave it as is. (Probably need a vhost_net_ops in the future).
Thanks
Quoted 18 lines
Thanks
Quoted 14 lines
Quoted 14 lines
hw/net/vhost_net.c +3
-0+ + vhost_dev_virtqueue_stop(&net->dev, vdev, idx); +}include/net/vhost_net.h +1
-0@@ -48,4 +48,6 @@ uint64_tvhost_net_get_acked_features (VHostNetState *net); int vhost_net_set_mtu(struct vhost_net *net, uint16_t mtu); +void vhost_net_virtqueue_stop(VirtIODevice *vdev, NetClientState *nc, + int vq_index); #endif
PATCH v2 12/24 vhost-net: vhost-kernel: introduce vhost_net_virtqueue_restart()
In reply to Kangjie XuIntroduce vhost_net_virtqueue_restart(), which can restart the virtqueue when the vhost net started running before. If it fails to restart the virtqueue, the device will be stopped.
This patch only considers the case for vhost-kernel, when NetClientDriver is NET_CLIENT_DRIVER_TAP.
--- hw/net/vhost_net.c | 48 +++++++++++++++++++++++++++++++++++++++++ include/net/vhost_net.h | 2 ++ 2 files changed, 50 insertions(+)
hw/net/vhost_net.c +48 -0
@@ -535,3 +535,51 @@ void vhost_net_virtqueue_stop(VirtIODevice *vdev, NetClientState *nc,vhost_dev_virtqueue_stop (&net->dev, vdev, idx); } + +int vhost_net_virtqueue_restart (VirtIODevice *vdev,NetClientState *nc, +int vq_index) +{ +VHostNetState *net =get_vhost_net (nc->peer); +const VhostOps *vhost_ops = net->dev.vhost_ops; +struct vhost_vring_file file = { }; +int idx, r; + +if (!net->dev.started) { +return 0 ; + } + +assert (vhost_ops); + + idx = vhost_ops->vhost_get_vq_index (&net->dev, vq_index); + + r =vhost_dev_virtqueue_restart (&net->dev, vdev, idx); +if (r <0 ) { + goto err_start; + } + +if (net->nc->info->type ==NET_CLIENT_DRIVER_TAP ) { + file.index = idx; + file.fd = net->backend; + r =vhost_net_set_backend (&net->dev, &file); +if (r <0 ) { + r = -errno; + goto err_start; + } + } + +return 0 ; + +err_start: +error_report ("Error when restarting the queue." ); + +if (net->nc->info->type ==NET_CLIENT_DRIVER_TAP ) { + file.fd =-1 ; + file.index = idx; +int r =vhost_net_set_backend (&net->dev, &file); +assert (r >=0 ); + } + +vhost_dev_stop (&net->dev, vdev); + +return r; +}
include/net/vhost_net.h +2 -0
@@ -50,4 +50,6 @@ int vhost_net_set_mtu(struct vhost_net *net, uint16_t mtu);void vhost_net_virtqueue_stop (VirtIODevice *vdev,NetClientState *nc,int vq_index); +int vhost_net_virtqueue_restart (VirtIODevice *vdev,NetClientState *nc, +int vq_index);#endif
Signature · 1 line
2.32.0
Re: [PATCH v2 12/24] vhost-net: vhost-kernel: introduce vhost_net_virtqueue_restart()
Quoted 9 lines · Kangjie Xu
Introduce
vhost_net_virtqueue_restart(), which can restart the virtqueue when the vhost net started running before. If it fails to restart the virtqueue, the device will be stopped.This patch only considers the case for vhost-kernel, when NetClientDriver is
NET_CLIENT_DRIVER_TAP.Signed-off-by Kangjie Xu <[email protected]>Signed-off-by Xuan Zhuo <[email protected]>
I would explain why current vhost_net_start_one()/vhost_net_stop_one() can't work. Is it because it works at queue pair level? If yes can we restructure the code and try to reuse ?
Thanks
Quoted 72 lines
--- hw/net/vhost_net.c | 48 +++++++++++++++++++++++++++++++++++++++++ include/net/vhost_net.h | 2 ++ 2 files changed, 50 insertions(+)hw/net/vhost_net.c +48
-0@@ -535,3 +535,51 @@ void vhost_net_virtqueue_stop(VirtIODevice *vdev, NetClientState *nc,vhost_dev_virtqueue_stop (&net->dev, vdev, idx); } + +int vhost_net_virtqueue_restart (VirtIODevice *vdev,NetClientState *nc, +int vq_index) +{ +VHostNetState *net =get_vhost_net (nc->peer); +const VhostOps *vhost_ops = net->dev.vhost_ops; +struct vhost_vring_file file = { }; +int idx, r; + +if (!net->dev.started) { +return 0 ; + } + +assert (vhost_ops); + + idx = vhost_ops->vhost_get_vq_index (&net->dev, vq_index); + + r =vhost_dev_virtqueue_restart (&net->dev, vdev, idx); +if (r <0 ) { + goto err_start; + } + +if (net->nc->info->type ==NET_CLIENT_DRIVER_TAP ) { + file.index = idx; + file.fd = net->backend; + r =vhost_net_set_backend (&net->dev, &file); +if (r <0 ) { + r = -errno; + goto err_start; + } + } + +return 0 ; + +err_start: +error_report ("Error when restarting the queue." ); + +if (net->nc->info->type ==NET_CLIENT_DRIVER_TAP ) { + file.fd =-1 ; + file.index = idx; +int r =vhost_net_set_backend (&net->dev, &file); +assert (r >=0 ); + } + +vhost_dev_stop (&net->dev, vdev); + +return r; +}include/net/vhost_net.h +2
-0@@ -50,4 +50,6 @@ int vhost_net_set_mtu(struct vhost_net *net, uint16_t mtu);void vhost_net_virtqueue_stop (VirtIODevice *vdev,NetClientState *nc,int vq_index); +int vhost_net_virtqueue_restart (VirtIODevice *vdev,NetClientState *nc, +int vq_index);#endif
Re: [PATCH v2 12/24] vhost-net: vhost-kernel: introduce vhost_net_virtqueue_restart()
Quoted 9 lines · Jason Wang
Quoted 7 lines · Kangjie Xu
Introduce
vhost_net_virtqueue_restart(), which can restart the virtqueue when the vhost net started running before. If it fails to restart the virtqueue, the device will be stopped.This patch only considers the case for vhost-kernel, when NetClientDriver is
NET_CLIENT_DRIVER_TAP.
Quoted 8 lines
Quoted 2 lines
Signed-off-by Kangjie Xu <[email protected]>Signed-off-by Xuan Zhuo <[email protected]>I would explain why current vhost_net_start_one()/vhost_net_stop_one() can't work. Is it because it works at queue pair level? If yes can we restructure the code and try to reuse ?
Thanks
Because vhost_net_start_one()/vhost_net_stop_one() works at device level.
The queue pair level start/stop are vhost_virtqueue_start() and vhost_virtqueue_stop().
What we can reuse is the vhost_virtqueue_start(). vhost_virtqueue_stop() cannot be reused because it will destroy device.
I think we do not need to restructure because we've already had an abstraction vhost_virtqueue_start().
Thanks.
Quoted 73 lines
Quoted 73 lines
---
hw/net/vhost_net.c | 48 +++++++++++++++++++++++++++++++++++++++++include/net/vhost_net.h| 2 ++2 files changed, 50 insertions(+)hw/net/vhost_net.c +48
-0@@ -535,3 +535,51 @@ void vhost_net_virtqueue_stop(VirtIODevice *vdev, NetClientState *nc, vhost_dev_virtqueue_stop(&net->dev, vdev, idx); } + +int vhost_net_virtqueue_restart (VirtIODevice *vdev,NetClientState *nc, + int vq_index) +{ + VHostNetState *net = get_vhost_net(nc->peer); + const VhostOps *vhost_ops = net->dev.vhost_ops; + struct vhost_vring_file file = { }; + int idx, r; + + if (!net->dev.started) { + return 0; + } + + assert(vhost_ops); + + idx = vhost_ops->vhost_get_vq_index(&net->dev, vq_index); + + r = vhost_dev_virtqueue_restart(&net->dev, vdev, idx); + if (r < 0) { + goto err_start; + } + + if (net->nc->info->type == NET_CLIENT_DRIVER_TAP) { + file.index = idx; + file.fd = net->backend; + r = vhost_net_set_backend(&net->dev, &file); + if (r < 0) { + r = -errno; + goto err_start; + } + } + + return 0; + +err_start: + error_report("Error when restarting the queue."); + + if (net->nc->info->type == NET_CLIENT_DRIVER_TAP) { + file.fd = -1; + file.index = idx; + int r = vhost_net_set_backend(&net->dev, &file); + assert(r >= 0); + } + + vhost_dev_stop(&net->dev, vdev); + + return r; +}include/net/vhost_net.h +2
-0@@ -50,4 +50,6 @@ int vhost_net_set_mtu(struct vhost_net *net, uint16_t mtu); void vhost_net_virtqueue_stop(VirtIODevice *vdev, NetClientState *nc, int vq_index); +int vhost_net_virtqueue_restart (VirtIODevice *vdev,NetClientState *nc, + int vq_index); #endif
Re: [PATCH v2 12/24] vhost-net: vhost-kernel: introduce vhost_net_virtqueue_restart()
Quoted 17 lines · Kangjie Xu
Quoted 15 lines · Jason Wang
Quoted 9 lines · Kangjie Xu
Introduce
vhost_net_virtqueue_restart(), which can restart the virtqueue when the vhost net started running before. If it fails to restart the virtqueue, the device will be stopped.This patch only considers the case for vhost-kernel, when NetClientDriver is
NET_CLIENT_DRIVER_TAP.Signed-off-by Kangjie Xu <[email protected]>Signed-off-by Xuan Zhuo <[email protected]>I would explain why current vhost_net_start_one()/vhost_net_stop_one() can't work. Is it because it works at queue pair level? If yes can we restructure the code and try to reuse ?
Quoted 7 lines
Quoted 2 lines
Thanks
Because vhost_net_start_one()/vhost_net_stop_one() works at device level.
The queue pair level start/stop are
vhost_virtqueue_start()andvhost_virtqueue_stop().What we can reuse is the
vhost_virtqueue_start().vhost_virtqueue_stop()cannot be reused because it will destroy device.
Let's add this in the changelog or a comment in the code.
Thanks
Quoted 82 lines
I think we do not need to restructure because we've already had an abstraction
vhost_virtqueue_start().Thanks.
Quoted 76 lines
Quoted 76 lines
---
hw/net/vhost_net.c | 48
+++++++++++++++++++++++++++++++++++++++++
include/net/vhost_net.h| 2 ++2 files changed, 50 insertions(+)hw/net/vhost_net.c +2
-0@@ -535,3 +535,51 @@ void vhost_net_virtqueue_stop(VirtIODevice *vdev, NetClientState *nc, vhost_dev_virtqueue_stop(&net->dev, vdev, idx); } + +int vhost_net_virtqueue_restart(VirtIODevice *vdev, NetClientState*nc,
+ intvq_index)hw/net/vhost_net.c +45
-0+{ + VHostNetState *net = get_vhost_net(nc->peer); + const VhostOps *vhost_ops = net->dev.vhost_ops; + struct vhost_vring_file file = { }; + int idx, r; + + if (!net->dev.started) { + return 0; + } + + assert(vhost_ops); + + idx = vhost_ops->vhost_get_vq_index(&net->dev, vq_index); + + r = vhost_dev_virtqueue_restart(&net->dev, vdev, idx); + if (r < 0) { + goto err_start; + } + + if (net->nc->info->type == NET_CLIENT_DRIVER_TAP) { + file.index = idx; + file.fd = net->backend; + r = vhost_net_set_backend(&net->dev, &file); + if (r < 0) { + r = -errno; + goto err_start; + } + } + + return 0; + +err_start: + error_report("Error when restarting the queue."); + + if (net->nc->info->type == NET_CLIENT_DRIVER_TAP) { + file.fd = -1; + file.index = idx; + int r = vhost_net_set_backend(&net->dev, &file); + assert(r >= 0); + } + + vhost_dev_stop(&net->dev, vdev); + + return r; +}include/net/vhost_net.h +2
-0@@ -50,4 +50,6 @@ int vhost_net_set_mtu(struct vhost_net *net, uint16_t mtu); void vhost_net_virtqueue_stop(VirtIODevice *vdev, NetClientState *nc, int vq_index); +int vhost_net_virtqueue_restart (VirtIODevice *vdev,NetClientState *nc, + int vq_index);#endif
Re: [PATCH v2 12/24] vhost-net: vhost-kernel: introduce vhost_net_virtqueue_restart()
Quoted 23 lines · Jason Wang
Quoted 21 lines · Kangjie Xu
Quoted 9 lines · Jason Wang
Quoted 7 lines · Kangjie Xu
Introduce
vhost_net_virtqueue_restart(), which can restart the virtqueue when the vhost net started running before. If it fails to restart the virtqueue, the device will be stopped.This patch only considers the case for vhost-kernel, when NetClientDriver is
NET_CLIENT_DRIVER_TAP.Quoted 8 lines
Quoted 2 lines
Signed-off-by Kangjie Xu <[email protected]>Signed-off-by Xuan Zhuo <[email protected]>I would explain why current vhost_net_start_one()/vhost_net_stop_one() can't work. Is it because it works at queue pair level? If yes can we restructure the code and try to reuse ?
Thanks
Because vhost_net_start_one()/vhost_net_stop_one() works at device level.
Quoted 10 lines
Quoted 3 lines
The queue pair level start/stop are
vhost_virtqueue_start()andvhost_virtqueue_stop().What we can reuse is the
vhost_virtqueue_start().vhost_virtqueue_stop()cannot be reused because it will destroy device.Let's add this in the changelog or a comment in the code.
Thanks
Will fix.
Thanks.
Quoted 82 lines
Quoted 82 lines
I think we do not need to restructure because we've already had an abstraction
vhost_virtqueue_start().Thanks.
Quoted 76 lines
Quoted 76 lines
---
hw/net/vhost_net.c | 48
+++++++++++++++++++++++++++++++++++++++++
include/net/vhost_net.h| 2 ++2 files changed, 50 insertions(+)hw/net/vhost_net.c +2
-0@@ -535,3 +535,51 @@ void vhost_net_virtqueue_stop(VirtIODevice *vdev, NetClientState *nc, vhost_dev_virtqueue_stop(&net->dev, vdev, idx); } + +int vhost_net_virtqueue_restart(VirtIODevice *vdev, NetClientState*nc,
+ intvq_index)hw/net/vhost_net.c +45
-0+{ + VHostNetState *net = get_vhost_net(nc->peer); + const VhostOps *vhost_ops = net->dev.vhost_ops; + struct vhost_vring_file file = { }; + int idx, r; + + if (!net->dev.started) { + return 0; + } + + assert(vhost_ops); + + idx = vhost_ops->vhost_get_vq_index(&net->dev, vq_index); + + r = vhost_dev_virtqueue_restart(&net->dev, vdev, idx); + if (r < 0) { + goto err_start; + } + + if (net->nc->info->type == NET_CLIENT_DRIVER_TAP) { + file.index = idx; + file.fd = net->backend; + r = vhost_net_set_backend(&net->dev, &file); + if (r < 0) { + r = -errno; + goto err_start; + } + } + + return 0; + +err_start: + error_report("Error when restarting the queue."); + + if (net->nc->info->type == NET_CLIENT_DRIVER_TAP) { + file.fd = -1; + file.index = idx; + int r = vhost_net_set_backend(&net->dev, &file); + assert(r >= 0); + } + + vhost_dev_stop(&net->dev, vdev); + + return r; +}include/net/vhost_net.h +2
-0@@ -50,4 +50,6 @@ int vhost_net_set_mtu(struct vhost_net *net, uint16_t mtu); void vhost_net_virtqueue_stop(VirtIODevice *vdev, NetClientState *nc, int vq_index); +int vhost_net_virtqueue_restart (VirtIODevice *vdev,NetClientState *nc, + int vq_index);#endif
PATCH v2 13/24 docs: vhost-user: add VHOST_USER_RESET_VRING message
In reply to Kangjie XuTo support the reset operation for an individual virtqueue, we introduce a new message VHOST_USER_RESET_VRING. This message is submitted by the front-end to reset an individual virtqueue to initial states in the back-end. The reply is needed to ensure that the reset operation is complete.
--- docs/interop/vhost-user.rst | 10 ++++++++++ 1 file changed, 10 insertions(+)
docs/interop/vhost-user.rst +10 -0
@@ -1422,6 +1422,16 @@ Front-end message types query the back-end for its device status as defined in the Virtio specification. +``VHOST_USER_RESET_VRING`` + :id: 41 + :equivalent ioctl: N/A + :request payload: vring state description + :reply payload: ``u64`` + + When the feature ``VIRTIO_F_RING_RESET`` feature has been successfully + negotiated, this message is submitted by the front-end to reset an + individual virtqueue to initial states in the back-end. It will ask + for a reply to ensure the virtqueue is successfully reset in the back-end. Back-end message types ----------------------
Signature · 1 line
2.32.0
Re: [PATCH v2 13/24] docs: vhost-user: add VHOST_USER_RESET_VRING message
Quoted 8 lines · Kangjie Xu
To support the reset operation for an individual virtqueue, we introduce a new message
VHOST_USER_RESET_VRING. This message is submitted by the front-end to reset an individual virtqueue to initial states in the back-end. The reply is needed to ensure that the reset operation is complete.Signed-off-by Kangjie Xu <[email protected]>Signed-off-by Xuan Zhuo <[email protected]>
Quoted 25 lines
--- docs/interop/vhost-user.rst | 10 ++++++++++ 1 file changed, 10 insertions(+)docs/interop/vhost-user.rst +10
-0@@ -1422,6 +1422,16 @@ Front-end message types query the back-end for its device status as defined in the Virtio specification. +``VHOST_USER_RESET_VRING`` + :id: 41 + :equivalent ioctl: N/A + :request payload: vring state description + :reply payload: ``u64`` + + When the feature ``VIRTIO_F_RING_RESET`` feature has been successfully + negotiated, this message is submitted by the front-end to reset an + individual virtqueue to initial states in the back-end. It will ask + for a reply to ensure the virtqueue is successfully reset in the back-end. Back-end message types ----------------------
PATCH v2 14/24 vhost-user: introduce vhost_reset_vring() interface
In reply to Kangjie XuIntroduce the interface vhost_reset_vring(). The interface is a wrapper to send a VHOST_USER_RESET_VRING message to the back-end. It will reset an individual vring in the back-end. Meanwhile, it will wait for a reply to ensure the reset has been completed.
--- hw/virtio/vhost-user.c | 41 +++++++++++++++++++++++++++++++ include/hw/virtio/vhost-backend.h | 3 +++ 2 files changed, 44 insertions(+)
hw/virtio/vhost-user.c +41 -0
@@ -126,6 +126,7 @@ typedef enum VhostUserRequest {VHOST_USER_GET_MAX_MEM_SLOTS =36 ,VHOST_USER_ADD_MEM_REG =37 ,VHOST_USER_REM_MEM_REG =38 , +VHOST_USER_RESET_VRING =41 , VHOST_USER_MAX } VhostUserRequest;
@@ -1498,6 +1499,45 @@ static int vhost_user_get_max_memslots(struct vhost_dev *dev,return 0 ; } +static int vhost_user_reset_vring (struct vhost_dev *dev, +struct vhost_vring_state *ring) +{ +int ret; +VhostUserMsg msg = { + .hdr.request =VHOST_USER_RESET_VRING , + .hdr.flags =VHOST_USER_VERSION , + .payload.state = *ring, + .hdr.size =sizeof (msg.payload.state), + }; + +if (!virtio_has_feature (dev->acked_features,VIRTIO_F_RING_RESET )) { +return -ENOTSUP ; + } + + ret =vhost_user_write (dev, &msg,NULL ,0 ); +if (ret <0 ) { +return ret; + } + + ret =vhost_user_read (dev, &msg); +if (ret <0 ) { +return ret; + } + +if (msg.hdr.request !=VHOST_USER_RESET_VRING ) { +error_report ("Received unexpected msg type. Expected %d received %d" , +VHOST_USER_RESET_VRING , msg.hdr.request); +return -EPROTO ; + } + +if (msg.hdr.size !=sizeof (msg.payload.state)) { +error_report ("Received bad msg size." ); +return -EPROTO ; + } + +return 0 ; +} +static int vhost_user_reset_device (struct vhost_dev *dev) {VhostUserMsg msg = {
@@ -2625,6 +2665,7 @@ const VhostOps user_ops = { .vhost_set_features = vhost_user_set_features, .vhost_get_features = vhost_user_get_features, .vhost_set_owner = vhost_user_set_owner, + .vhost_reset_vring = vhost_user_reset_vring, .vhost_reset_device = vhost_user_reset_device, .vhost_get_vq_index = vhost_user_get_vq_index, .vhost_set_vring_enable = vhost_user_set_vring_enable,
include/hw/virtio/vhost-backend.h +3 -0
@@ -79,6 +79,8 @@ typedef int (*vhost_get_features_op)(struct vhost_dev *dev, uint64_t *features);typedef int (*vhost_set_backend_cap_op )(struct vhost_dev *dev);typedef int (*vhost_set_owner_op )(struct vhost_dev *dev); +typedef int (*vhost_reset_vring_op )(struct vhost_dev *dev, +struct vhost_vring_state *ring);typedef int (*vhost_reset_device_op )(struct vhost_dev *dev);typedef int (*vhost_get_vq_index_op )(struct vhost_dev *dev,int idx); typedef int (*vhost_set_vring_enable_op)(struct vhost_dev *dev,
@@ -154,6 +156,7 @@ typedef struct VhostOps {vhost_set_backend_cap_op vhost_set_backend_cap;vhost_set_owner_op vhost_set_owner;vhost_reset_device_op vhost_reset_device; +vhost_reset_vring_op vhost_reset_vring;vhost_get_vq_index_op vhost_get_vq_index;vhost_set_vring_enable_op vhost_set_vring_enable;vhost_requires_shm_log_op vhost_requires_shm_log;
Signature · 1 line
2.32.0
Re: [PATCH v2 14/24] vhost-user: introduce vhost_reset_vring() interface
Quoted 25 lines · Kangjie Xu
Introduce the interface
vhost_reset_vring(). The interface is a wrapper to send aVHOST_USER_RESET_VRINGmessage to the back-end. It will reset an individual vring in the back-end. Meanwhile, it will wait for a reply to ensure the reset has been completed.Signed-off-by Kangjie Xu <[email protected]>Signed-off-by Xuan Zhuo <[email protected]>--- hw/virtio/vhost-user.c | 41 +++++++++++++++++++++++++++++++ include/hw/virtio/vhost-backend.h | 3 +++ 2 files changed, 44 insertions(+)hw/virtio/vhost-user.c +1
-0@@ -126,6 +126,7 @@ typedef enum VhostUserRequest {VHOST_USER_GET_MAX_MEM_SLOTS =36 ,VHOST_USER_ADD_MEM_REG =37 ,VHOST_USER_REM_MEM_REG =38 , +VHOST_USER_RESET_VRING =41 , VHOST_USER_MAX } VhostUserRequest;
hw/virtio/vhost-user.c static int vhost_user_get_max_memslots(struct vhost_dev *dev, · line 1508
Quoted 10 lines
hw/virtio/vhost-user.c +7
-0@@ -1498,6 +1499,45 @@ static int vhost_user_get_max_memslots(struct vhost_dev *dev,return 0 ; } +static int vhost_user_reset_vring (struct vhost_dev *dev, +struct vhost_vring_state *ring) +{ +int ret; +VhostUserMsg msg = { + .hdr.request =VHOST_USER_RESET_VRING , + .hdr.flags =VHOST_USER_VERSION ,
Do we need VHOST_USER_NEED_REPLY_MASK here?
Other looks good.
Thanks
Quoted 64 lines
hw/virtio/vhost-user.c +33
-0+ .payload.state = *ring, + .hdr.size = sizeof(msg.payload.state), + }; + +if (!virtio_has_feature (dev->acked_features,VIRTIO_F_RING_RESET )) { +return -ENOTSUP ; + } + + ret =vhost_user_write (dev, &msg,NULL ,0 ); +if (ret <0 ) { +return ret; + } + + ret =vhost_user_read (dev, &msg); +if (ret <0 ) { +return ret; + } + +if (msg.hdr.request !=VHOST_USER_RESET_VRING ) { +error_report ("Received unexpected msg type. Expected %d received %d" , +VHOST_USER_RESET_VRING , msg.hdr.request); +return -EPROTO ; + } + +if (msg.hdr.size !=sizeof (msg.payload.state)) { +error_report ("Received bad msg size." ); +return -EPROTO ; + } + +return 0 ; +} +static int vhost_user_reset_device (struct vhost_dev *dev) {VhostUserMsg msg = {@@ -2625,6 +2665,7 @@ const VhostOps user_ops = { .vhost_set_features = vhost_user_set_features, .vhost_get_features = vhost_user_get_features, .vhost_set_owner = vhost_user_set_owner, + .vhost_reset_vring = vhost_user_reset_vring, .vhost_reset_device = vhost_user_reset_device, .vhost_get_vq_index = vhost_user_get_vq_index, .vhost_set_vring_enable = vhost_user_set_vring_enable,include/hw/virtio/vhost-backend.h +3
-0@@ -79,6 +79,8 @@ typedef int (*vhost_get_features_op)(struct vhost_dev *dev, uint64_t *features);typedef int (*vhost_set_backend_cap_op )(struct vhost_dev *dev);typedef int (*vhost_set_owner_op )(struct vhost_dev *dev); +typedef int (*vhost_reset_vring_op )(struct vhost_dev *dev, +struct vhost_vring_state *ring);typedef int (*vhost_reset_device_op )(struct vhost_dev *dev);typedef int (*vhost_get_vq_index_op )(struct vhost_dev *dev,int idx); typedef int (*vhost_set_vring_enable_op)(struct vhost_dev *dev,@@ -154,6 +156,7 @@ typedef struct VhostOps {vhost_set_backend_cap_op vhost_set_backend_cap;vhost_set_owner_op vhost_set_owner;vhost_reset_device_op vhost_reset_device; +vhost_reset_vring_op vhost_reset_vring;vhost_get_vq_index_op vhost_get_vq_index;vhost_set_vring_enable_op vhost_set_vring_enable;vhost_requires_shm_log_op vhost_requires_shm_log;
Re: [PATCH v2 14/24] vhost-user: introduce vhost_reset_vring() interface
Quoted 39 lines · Jason Wang
Quoted 25 lines · Kangjie Xu
Introduce the interface
vhost_reset_vring(). The interface is a wrapper to send aVHOST_USER_RESET_VRINGmessage to the back-end. It will reset an individual vring in the back-end. Meanwhile, it will wait for a reply to ensure the reset has been completed.Signed-off-by Kangjie Xu <[email protected]>Signed-off-by Xuan Zhuo <[email protected]>---
hw/virtio/vhost-user.c | 41 +++++++++++++++++++++++++++++++include/hw/virtio/vhost-backend.h| 3 +++2 files changed, 44 insertions(+)hw/virtio/vhost-user.c +1
-0@@ -126,6 +126,7 @@ typedef enum VhostUserRequest { VHOST_USER_GET_MAX_MEM_SLOTS = 36, VHOST_USER_ADD_MEM_REG = 37, VHOST_USER_REM_MEM_REG = 38, + VHOST_USER_RESET_VRING = 41,
VHOST_USER_MAX
} VhostUserRequest;
@@ -1498,6 +1499,45 @@ static int vhost_user_get_max_memslots(structvhost_dev*dev, return 0;Quoted 8 lines
}
+static int vhost_user_reset_vring(structvhost_dev*dev, + structvhost_vring_state*ring) +{
+ int ret;
+ VhostUserMsg msg = {
+ .hdr.request =VHOST_USER_RESET_VRING,
+ .hdr.flags =VHOST_USER_VERSION,Do we need
VHOST_USER_NEED_REPLY_MASKhere?
Quoted 4 lines
Other looks good.
Thanks
No, it works the same as vhost_user_get_features() or vhost_user_get_vring_base().
Thanks
Quoted 68 lines
Quoted 68 lines
+ .payload.state = *ring,
+ .hdr.size = sizeof(msg.payload.state),
+ };
+
+ if (!virtio_has_feature(dev->acked_features,VIRTIO_F_RING_RESET)) {
+ return-ENOTSUP;
+ }hw/virtio/vhost-user.c +13
-0+ + ret = vhost_user_write(dev, &msg, NULL, 0); + if (ret < 0) { + return ret; + } + + ret = vhost_user_read(dev, &msg); + if (ret < 0) { + return ret; + } + + if (msg.hdr.request != VHOST_USER_RESET_VRING) { + error_report("Received unexpected msg type. Expected %dreceived %d",
+VHOST_USER_RESET_VRING, msg.hdr.request); + return-EPROTO;
+ }hw/virtio/vhost-user.c +9
-0+ + if (msg.hdr.size != sizeof(msg.payload.state)) { + error_report("Received bad msg size."); + return -EPROTO; + } + + return 0; +} +static int
vhost_user_reset_device(struct vhost_dev *dev){
VhostUserMsg msg = {hw/virtio/vhost-user.c +1
-0@@ -2625,6 +2665,7 @@ const VhostOps user_ops = { .vhost_set_features = vhost_user_set_features, .vhost_get_features = vhost_user_get_features, .vhost_set_owner = vhost_user_set_owner, + .vhost_reset_vring = vhost_user_reset_vring, .vhost_reset_device = vhost_user_reset_device, .vhost_get_vq_index = vhost_user_get_vq_index, .vhost_set_vring_enable = vhost_user_set_vring_enable,include/hw/virtio/vhost-backend.h +2
-0@@ -79,6 +79,8 @@ typedef int (*vhost_get_features_op)(structvhost_dev *dev, uint64_t *features); typedef int (*vhost_set_backend_cap_op)(struct vhost_dev *dev); typedef int (*vhost_set_owner_op)(struct vhost_dev *dev); +typedef int (*vhost_reset_vring_op )(struct vhost_dev *dev, + struct vhost_vring_state *ring); typedef int (*vhost_reset_device_op)(struct vhost_dev *dev); typedef int (*vhost_get_vq_index_op)(struct vhost_dev *dev, int idx);typedef int (*vhost_set_vring_enable_op)(struct
vhost_dev*dev,include/hw/virtio/vhost-backend.h +1
-0@@ -154,6 +156,7 @@ typedef struct VhostOps { vhost_set_backend_cap_op vhost_set_backend_cap; vhost_set_owner_op vhost_set_owner; vhost_reset_device_op vhost_reset_device; + vhost_reset_vring_op vhost_reset_vring; vhost_get_vq_index_op vhost_get_vq_index; vhost_set_vring_enable_op vhost_set_vring_enable; vhost_requires_shm_log_op vhost_requires_shm_log;
PATCH v2 15/24 vhost-user: add op to enable or disable a single vring
In reply to Kangjie XuThe interface to set enable status for a single vring is lacked in VhostOps, since the vhost_set_vring_enable_op will manipulate all virtqueues in a device.
Resetting a single vq will rely on this interface.
--- hw/virtio/vhost-user.c | 26 +++++++++++++++++++------- include/hw/virtio/vhost-backend.h | 3 +++ 2 files changed, 22 insertions(+), 7 deletions(-)
hw/virtio/vhost-user.c +19 -7
@@ -1199,6 +1199,22 @@ static int vhost_user_set_vring_base(struct vhost_dev *dev,return vhost_set_vring (dev,VHOST_USER_SET_VRING_BASE , ring); } +static int vhost_user_set_single_vring_enable (struct vhost_dev *dev, +int index, +int enable) +{ +if (index < dev->vq_index || index >= dev->vq_index + dev->nvqs) { +return -EINVAL ; + } + +struct vhost_vring_state state = { + .index = index, + .num = enable, + }; + +return vhost_set_vring (dev,VHOST_USER_SET_VRING_ENABLE , &state); +} +static int vhost_user_set_vring_enable (struct vhost_dev *dev,int enable) {int i;
@@ -1208,13 +1224,8 @@ static int vhost_user_set_vring_enable(struct vhost_dev *dev, int enable) }for (i =0 ; i < dev->nvqs; ++i) { -int ret; -struct vhost_vring_state state = { - .index = dev->vq_index + i, - .num = enable, - }; - - ret =vhost_set_vring (dev,VHOST_USER_SET_VRING_ENABLE , &state); +int ret =vhost_user_set_single_vring_enable (dev, dev->vq_index + i, + enable);if (ret <0 ) {/* * Restoring the previous state is likely infeasible, as well as
@@ -2668,6 +2679,7 @@ const VhostOps user_ops = { .vhost_reset_vring = vhost_user_reset_vring, .vhost_reset_device = vhost_user_reset_device, .vhost_get_vq_index = vhost_user_get_vq_index, + .vhost_set_single_vring_enable = vhost_user_set_single_vring_enable, .vhost_set_vring_enable = vhost_user_set_vring_enable, .vhost_requires_shm_log = vhost_user_requires_shm_log, .vhost_migration_done = vhost_user_migration_done,
include/hw/virtio/vhost-backend.h +3 -0
@@ -83,6 +83,8 @@ typedef int (*vhost_reset_vring_op)(struct vhost_dev *dev, struct vhost_vring_state *ring);typedef int (*vhost_reset_device_op )(struct vhost_dev *dev);typedef int (*vhost_get_vq_index_op )(struct vhost_dev *dev,int idx); +typedef int (*vhost_set_single_vring_enable_op )(struct vhost_dev *dev, +int index,int enable);typedef int (*vhost_set_vring_enable_op )(struct vhost_dev *dev,int enable);typedef bool (*vhost_requires_shm_log_op )(struct vhost_dev *dev);
@@ -158,6 +160,7 @@ typedef struct VhostOps {vhost_reset_device_op vhost_reset_device;vhost_reset_vring_op vhost_reset_vring;vhost_get_vq_index_op vhost_get_vq_index; +vhost_set_single_vring_enable_op vhost_set_single_vring_enable;vhost_set_vring_enable_op vhost_set_vring_enable;vhost_requires_shm_log_op vhost_requires_shm_log;vhost_migration_done_op vhost_migration_done;
Signature · 1 line
2.32.0
Re: [PATCH v2 15/24] vhost-user: add op to enable or disable a single vring
Quoted 41 lines · Kangjie Xu
The interface to set enable status for a single vring is lacked in VhostOps, since the
vhost_set_vring_enable_opwill manipulate all virtqueues in a device.Resetting a single vq will rely on this interface.
Signed-off-by Kangjie Xu <[email protected]>Signed-off-by Xuan Zhuo <[email protected]>--- hw/virtio/vhost-user.c | 26 +++++++++++++++++++------- include/hw/virtio/vhost-backend.h | 3 +++ 2 files changed, 22 insertions(+), 7 deletions(-)hw/virtio/vhost-user.c +16
-0@@ -1199,6 +1199,22 @@ static int vhost_user_set_vring_base(struct vhost_dev *dev,return vhost_set_vring (dev,VHOST_USER_SET_VRING_BASE , ring); } +static int vhost_user_set_single_vring_enable (struct vhost_dev *dev, +int index, +int enable) +{ +if (index < dev->vq_index || index >= dev->vq_index + dev->nvqs) { +return -EINVAL ; + } + +struct vhost_vring_state state = { + .index = index, + .num = enable, + }; + +return vhost_set_vring (dev,VHOST_USER_SET_VRING_ENABLE , &state); +} +static int vhost_user_set_vring_enable (struct vhost_dev *dev,int enable) {int i;
hw/virtio/vhost-user.c static int vhost_user_set_vring_enable(struct vhost_dev *dev, int enable) · line 1217
Quoted 10 lines
hw/virtio/vhost-user.c +0
-7@@ -1208,13 +1224,8 @@ static int vhost_user_set_vring_enable(struct vhost_dev *dev, int enable) }for (i =0 ; i < dev->nvqs; ++i) { -int ret; -struct vhost_vring_state state = { - .index = dev->vq_index + i, - .num = enable, - }; - - ret =vhost_set_vring (dev,VHOST_USER_SET_VRING_ENABLE , &state);
Then I'd squash this into previous patch or re-roder to let this patch (vhost_user_set_single_vring_enable()) to be first.
Thanks
Quoted 34 lines
hw/virtio/vhost-user.c +3
-0+int ret =vhost_user_set_single_vring_enable (dev, dev->vq_index + i, + enable);if (ret <0 ) {/* * Restoring the previous state is likely infeasible, as well as @@ -2668,6 +2679,7 @@ const VhostOps user_ops = { .vhost_reset_vring = vhost_user_reset_vring, .vhost_reset_device = vhost_user_reset_device, .vhost_get_vq_index = vhost_user_get_vq_index, + .vhost_set_single_vring_enable = vhost_user_set_single_vring_enable, .vhost_set_vring_enable = vhost_user_set_vring_enable, .vhost_requires_shm_log = vhost_user_requires_shm_log, .vhost_migration_done = vhost_user_migration_done,include/hw/virtio/vhost-backend.h +3
-0@@ -83,6 +83,8 @@ typedef int (*vhost_reset_vring_op)(struct vhost_dev *dev, struct vhost_vring_state *ring);typedef int (*vhost_reset_device_op )(struct vhost_dev *dev);typedef int (*vhost_get_vq_index_op )(struct vhost_dev *dev,int idx); +typedef int (*vhost_set_single_vring_enable_op )(struct vhost_dev *dev, +int index,int enable);typedef int (*vhost_set_vring_enable_op )(struct vhost_dev *dev,int enable);typedef bool (*vhost_requires_shm_log_op )(struct vhost_dev *dev);@@ -158,6 +160,7 @@ typedef struct VhostOps {vhost_reset_device_op vhost_reset_device;vhost_reset_vring_op vhost_reset_vring;vhost_get_vq_index_op vhost_get_vq_index; +vhost_set_single_vring_enable_op vhost_set_single_vring_enable;vhost_set_vring_enable_op vhost_set_vring_enable;vhost_requires_shm_log_op vhost_requires_shm_log;vhost_migration_done_op vhost_migration_done;
Re: [PATCH v2 15/24] vhost-user: add op to enable or disable a single vring
Quoted 58 lines · Jason Wang
Quoted 42 lines · Kangjie Xu
The interface to set enable status for a single vring is lacked in VhostOps, since the
vhost_set_vring_enable_opwill manipulate all virtqueues in a device.Resetting a single vq will rely on this interface.
Signed-off-by Kangjie Xu <[email protected]>Signed-off-by Xuan Zhuo <[email protected]>---
hw/virtio/vhost-user.c | 26 +++++++++++++++++++-------include/hw/virtio/vhost-backend.h| 3 +++2 files changed, 22 insertions(+), 7 deletions(-)hw/virtio/vhost-user.c +15
-0@@ -1199,6 +1199,22 @@ static int vhost_user_set_vring_base(structvhost_dev *dev, return vhost_set_vring(dev, VHOST_USER_SET_VRING_BASE, ring); } +static int vhost_user_set_single_vring_enable(struct vhost_dev *dev, + int index, + int enable) +{ + if (index < dev->vq_index || index >= dev->vq_index + dev->nvqs) { + return -EINVAL; + } + + struct vhost_vring_state state = { + .index = index, + .num = enable, + }; + + return vhost_set_vring(dev, VHOST_USER_SET_VRING_ENABLE, &state); +} + static int vhost_user_set_vring_enable(struct vhost_dev *dev, int enable) {int i;
hw/virtio/vhost-user.c +0
-0@@ -1208,13 +1224,8 @@ static int vhost_user_set_vring_enable(struct vhost_dev *dev, int enable)hw/virtio/vhost-user.c static int vhost_user_set_vring_enable(struct · line 1227
Quoted 10 lines
hw/virtio/vhost-user.c +0
-7} for (i = 0; i < dev->nvqs; ++i) { - int ret; - struct vhost_vring_state state = { - .index = dev->vq_index + i, - .num = enable, - }; - - ret = vhost_set_vring(dev, VHOST_USER_SET_VRING_ENABLE, &state);Then I'd squash this into previous patch or re-roder to let this patch (
vhost_user_set_single_vring_enable()) to be first.
Quoted 3 lines
Thanks
Sorry, I don't get why we should re-order them, since these two patches are independent.
Thanks
Quoted 37 lines
Quoted 37 lines
hw/virtio/vhost-user.c +2
-0+ int ret = vhost_user_set_single_vring_enable(dev, dev->vq_index + i, + enable); if (ret < 0) {/*
* Restoring the previous state is likely infeasible, as well ashw/virtio/vhost-user.c +1
-0@@ -2668,6 +2679,7 @@ const VhostOps user_ops = { .vhost_reset_vring = vhost_user_reset_vring, .vhost_reset_device = vhost_user_reset_device, .vhost_get_vq_index = vhost_user_get_vq_index, + .vhost_set_single_vring_enable = vhost_user_set_single_vring_enable, .vhost_set_vring_enable = vhost_user_set_vring_enable, .vhost_requires_shm_log = vhost_user_requires_shm_log,.vhost_migration_done =
vhost_user_migration_done,include/hw/virtio/vhost-backend.h +2
-0@@ -83,6 +83,8 @@ typedef int (*vhost_reset_vring_op)(struct vhost_dev *dev, struct vhost_vring_state *ring); typedef int (*vhost_reset_device_op)(struct vhost_dev *dev); typedef int (*vhost_get_vq_index_op)(struct vhost_dev *dev, int idx); +typedef int (*vhost_set_single_vring_enable_op )(struct vhost_dev *dev, + int index, int enable); typedef int (*vhost_set_vring_enable_op)(struct vhost_dev *dev, int enable);typedef bool (*vhost_requires_shm_log_op)(struct
vhost_dev*dev);include/hw/virtio/vhost-backend.h +1
-0@@ -158,6 +160,7 @@ typedef struct VhostOps { vhost_reset_device_op vhost_reset_device; vhost_reset_vring_op vhost_reset_vring; vhost_get_vq_index_op vhost_get_vq_index; + vhost_set_single_vring_enable_op vhost_set_single_vring_enable; vhost_set_vring_enable_op vhost_set_vring_enable; vhost_requires_shm_log_op vhost_requires_shm_log; vhost_migration_done_op vhost_migration_done;
Re: [PATCH v2 15/24] vhost-user: add op to enable or disable a single vring
Quoted 60 lines · Kangjie Xu
Quoted 58 lines · Jason Wang
Quoted 42 lines · Kangjie Xu
The interface to set enable status for a single vring is lacked in VhostOps, since the
vhost_set_vring_enable_opwill manipulate all virtqueues in a device.Resetting a single vq will rely on this interface.
Signed-off-by Kangjie Xu <[email protected]>Signed-off-by Xuan Zhuo <[email protected]>---
hw/virtio/vhost-user.c | 26 +++++++++++++++++++-------include/hw/virtio/vhost-backend.h| 3 +++2 files changed, 22 insertions(+), 7 deletions(-)hw/virtio/vhost-user.c +15
-0@@ -1199,6 +1199,22 @@ static int vhost_user_set_vring_base(structvhost_dev *dev, return vhost_set_vring(dev, VHOST_USER_SET_VRING_BASE, ring); } +static int vhost_user_set_single_vring_enable(struct vhost_dev *dev, + int index, + int enable) +{ + if (index < dev->vq_index || index >= dev->vq_index + dev->nvqs) { + return -EINVAL; + } + + struct vhost_vring_state state = { + .index = index, + .num = enable, + }; + + return vhost_set_vring(dev, VHOST_USER_SET_VRING_ENABLE, &state); +} + static int vhost_user_set_vring_enable(struct vhost_dev *dev, int enable) {int i;
hw/virtio/vhost-user.c +0
-0@@ -1208,13 +1224,8 @@ static int vhost_user_set_vring_enable(struct vhost_dev *dev, int enable)hw/virtio/vhost-user.c static int vhost_user_set_vring_enable(struct · line 1227
Quoted 10 lines
hw/virtio/vhost-user.c +0
-7} for (i = 0; i < dev->nvqs; ++i) { - int ret; - struct vhost_vring_state state = { - .index = dev->vq_index + i, - .num = enable, - }; - - ret = vhost_set_vring(dev, VHOST_USER_SET_VRING_ENABLE, &state);Then I'd squash this into previous patch or re-roder to let this patch (
vhost_user_set_single_vring_enable()) to be first.
Quoted 4 lines
Quoted 3 lines
Thanks
Sorry, I don't get why we should re-order them, since these two patches are independent.
I meant it's not good to introduce some codes in patch 14 but delete them in patch 15 (the above part for example).
Thanks
Quoted 42 lines
Thanks
Quoted 38 lines
Quoted 38 lines
hw/virtio/vhost-user.c +2
-0+ int ret = vhost_user_set_single_vring_enable(dev, dev->vq_index + i, + enable); if (ret < 0) {/*
* Restoring the previous state is likely infeasible, as well ashw/virtio/vhost-user.c +1
-0@@ -2668,6 +2679,7 @@ const VhostOps user_ops = { .vhost_reset_vring = vhost_user_reset_vring, .vhost_reset_device = vhost_user_reset_device, .vhost_get_vq_index = vhost_user_get_vq_index, + .vhost_set_single_vring_enable = vhost_user_set_single_vring_enable, .vhost_set_vring_enable = vhost_user_set_vring_enable, .vhost_requires_shm_log = vhost_user_requires_shm_log,.vhost_migration_done =
vhost_user_migration_done,include/hw/virtio/vhost-backend.h +2
-0@@ -83,6 +83,8 @@ typedef int (*vhost_reset_vring_op)(struct vhost_dev *dev, struct vhost_vring_state *ring); typedef int (*vhost_reset_device_op)(struct vhost_dev *dev); typedef int (*vhost_get_vq_index_op)(struct vhost_dev *dev, int idx); +typedef int (*vhost_set_single_vring_enable_op )(struct vhost_dev *dev, + int index, int enable); typedef int (*vhost_set_vring_enable_op)(struct vhost_dev *dev,int enable); typedef bool (*vhost_requires_shm_log_op)(struct
vhost_dev*dev);include/hw/virtio/vhost-backend.h +1
-0@@ -158,6 +160,7 @@ typedef struct VhostOps { vhost_reset_device_op vhost_reset_device; vhost_reset_vring_op vhost_reset_vring; vhost_get_vq_index_op vhost_get_vq_index; + vhost_set_single_vring_enable_op vhost_set_single_vring_enable; vhost_set_vring_enable_op vhost_set_vring_enable; vhost_requires_shm_log_op vhost_requires_shm_log; vhost_migration_done_op vhost_migration_done;
Re: [PATCH v2 15/24] vhost-user: add op to enable or disable a single vring
Quoted 66 lines · Jason Wang
Quoted 64 lines · Kangjie Xu
Quoted 62 lines · Jason Wang
Quoted 43 lines · Kangjie Xu
The interface to set enable status for a single vring is lacked in VhostOps, since the
vhost_set_vring_enable_opwill manipulate all virtqueues in a device.Resetting a single vq will rely on this interface.
Signed-off-by Kangjie Xu <[email protected]>Signed-off-by Xuan Zhuo <[email protected]>---
hw/virtio/vhost-user.c | 26 +++++++++++++++++++-------include/hw/virtio/vhost-backend.h| 3 +++2 files changed, 22 insertions(+), 7 deletions(-)hw/virtio/vhost-user.c +15
-0@@ -1199,6 +1199,22 @@ static int vhost_user_set_vring_base(structvhost_dev *dev, return vhost_set_vring(dev, VHOST_USER_SET_VRING_BASE, ring); } +static int vhost_user_set_single_vring_enable(struct vhost_dev *dev, + int index, + int enable) +{ + if (index < dev->vq_index || index >= dev->vq_index + dev->nvqs) { + return -EINVAL; + } + + struct vhost_vring_state state = { + .index = index, + .num = enable, + }; + + return vhost_set_vring(dev, VHOST_USER_SET_VRING_ENABLE, &state); +} + static int vhost_user_set_vring_enable(struct vhost_dev *dev, int enable){
int i;hw/virtio/vhost-user.c +0
-0@@ -1208,13 +1224,8 @@ static int vhost_user_set_vring_enable(struct vhost_dev *dev, int enable)hw/virtio/vhost-user.c static int · line 1227
Quoted 10 lines
hw/virtio/vhost-user.c +0
-7} for (i = 0; i < dev->nvqs; ++i) { - int ret; - struct vhost_vring_state state = { - .index = dev->vq_index + i, - .num = enable, - }; - - ret = vhost_set_vring(dev, VHOST_USER_SET_VRING_ENABLE, &state);Then I'd squash this into previous patch or re-roder to let this patch (
vhost_user_set_single_vring_enable()) to be first.Thanks
Quoted 6 lines
Quoted 1 line
Sorry, I don't get why we should re-order them, since these two patches are independent.
I meant it's not good to introduce some codes in patch 14 but delete them in patch 15 (the above part for example).
Thanks
I get your point, but in fact, it seems that the deleded codes here in patch 15 do not appear in patch 14....
Patch 14 is about vhost_user_reset_vring(), patch 15 is about vhost_user_set_vring_enable().
Thanks.
Quoted 43 lines
Quoted 43 lines
Thanks
Quoted 39 lines
Quoted 39 lines
hw/virtio/vhost-user.c +2
-0+ int ret = vhost_user_set_single_vring_enable(dev, dev->vq_index + i, + enable); if (ret < 0) {/*
* Restoring the previous state is likely infeasible, as well ashw/virtio/vhost-user.c +1
-0@@ -2668,6 +2679,7 @@ const VhostOps user_ops = { .vhost_reset_vring = vhost_user_reset_vring, .vhost_reset_device = vhost_user_reset_device, .vhost_get_vq_index = vhost_user_get_vq_index, + .vhost_set_single_vring_enable = vhost_user_set_single_vring_enable, .vhost_set_vring_enable = vhost_user_set_vring_enable, .vhost_requires_shm_log = vhost_user_requires_shm_log,.vhost_migration_done =
vhost_user_migration_done,include/hw/virtio/vhost-backend.h +2
-0@@ -83,6 +83,8 @@ typedef int (*vhost_reset_vring_op)(struct vhost_dev *dev, struct vhost_vring_state *ring); typedef int (*vhost_reset_device_op)(struct vhost_dev *dev); typedef int (*vhost_get_vq_index_op)(struct vhost_dev *dev, int idx); +typedef int (*vhost_set_single_vring_enable_op )(struct vhost_dev *dev, + int index, int enable);typedef int (*vhost_set_vring_enable_op)(struct
vhost_dev*dev, int enable); typedef bool (*vhost_requires_shm_log_op)(structvhost_dev*dev);include/hw/virtio/vhost-backend.h +1
-0@@ -158,6 +160,7 @@ typedef struct VhostOps { vhost_reset_device_op vhost_reset_device; vhost_reset_vring_op vhost_reset_vring; vhost_get_vq_index_op vhost_get_vq_index; + vhost_set_single_vring_enable_op vhost_set_single_vring_enable; vhost_set_vring_enable_op vhost_set_vring_enable; vhost_requires_shm_log_op vhost_requires_shm_log; vhost_migration_done_op vhost_migration_done;
PATCH v2 16/24 vhost: vhost-user: update vhost_dev_virtqueue_stop()
In reply to Kangjie XuUpdate vhost_dev_virtqueue_stop() for vhost-user scenario.
--- hw/virtio/vhost.c | 19 +++++++++++++++++++ 1 file changed, 19 insertions(+)
hw/virtio/vhost.c +19 -0
@@ -1908,10 +1908,29 @@ int vhost_net_set_backend(struct vhost_dev *hdev,void vhost_dev_virtqueue_stop (struct vhost_dev *hdev,VirtIODevice *vdev,int idx) { +const VhostOps *vhost_ops = hdev->vhost_ops; +struct vhost_vring_state state; +int r; + +assert (vhost_ops); + +if (vhost_ops->vhost_reset_vring) { + state.index = hdev->vq_index + idx; + r = vhost_ops->vhost_reset_vring (hdev, &state); +if (r <0 ) { + goto err_queue_reset; + } + } +vhost_virtqueue_unmap (hdev, vdev, hdev->vqs + idx, idx); + +return ; + +err_queue_reset: +error_report ("Error when stopping the qeuue." ); } int vhost_dev_virtqueue_restart(struct vhost_dev *hdev, VirtIODevice *vdev,
Signature · 1 line
2.32.0
Re: [PATCH v2 16/24] vhost: vhost-user: update vhost_dev_virtqueue_stop()
Quoted 20 lines · Kangjie Xu
Update
vhost_dev_virtqueue_stop()for vhost-user scenario.Signed-off-by Kangjie Xu <[email protected]>Signed-off-by Xuan Zhuo <[email protected]>--- hw/virtio/vhost.c | 19 +++++++++++++++++++ 1 file changed, 19 insertions(+)hw/virtio/vhost.c +4
-0@@ -1908,10 +1908,29 @@ int vhost_net_set_backend(struct vhost_dev *hdev,void vhost_dev_virtqueue_stop (struct vhost_dev *hdev,VirtIODevice *vdev,int idx) { +const VhostOps *vhost_ops = hdev->vhost_ops; +struct vhost_vring_state state; +int r; +
hw/virtio/vhost.c int vhost_net_set_backend(struct vhost_dev *hdev, · line 1924
Quoted 10 lines
hw/virtio/vhost.c +10
-0+assert (vhost_ops); + +if (vhost_ops->vhost_reset_vring) { + state.index = hdev->vq_index + idx; + r = vhost_ops->vhost_reset_vring (hdev, &state); +if (r <0 ) { + goto err_queue_reset; + } + } +
So this worries me:
1) having two similar functions
vhost_virtqueue_stop()
and
vhost_dev_virtqueue_stop()
It can easily confuse the people who want stop the device.
I think we need rename vhost_dev_virtqueue_stop() to vhost_virqtueue_reset() since it has different semantic compared to stop:
1) stop means the virtqueue state is reserved, e.g the index could be synced via get_vring_base()
2) reset means the virqtueue state is lost
Thanks
Quoted 12 lines
hw/virtio/vhost.c +5
-0vhost_virtqueue_unmap (hdev, vdev, hdev->vqs + idx, idx); + +return ; + +err_queue_reset: +error_report ("Error when stopping the qeuue." ); } int vhost_dev_virtqueue_restart(struct vhost_dev *hdev, VirtIODevice *vdev,
Re: [PATCH v2 16/24] vhost: vhost-user: update vhost_dev_virtqueue_stop()
Quoted 45 lines · Jason Wang
Quoted 21 lines · Kangjie Xu
Update
vhost_dev_virtqueue_stop()for vhost-user scenario.Signed-off-by Kangjie Xu <[email protected]>Signed-off-by Xuan Zhuo <[email protected]>---
hw/virtio/vhost.c| 19 +++++++++++++++++++1 file changed, 19 insertions(+)hw/virtio/vhost.c +4
-0@@ -1908,10 +1908,29 @@ int vhost_net_set_backend(struct vhost_dev *hdev, void vhost_dev_virtqueue_stop(struct vhost_dev *hdev, VirtIODevice *vdev, int idx) { + const VhostOps *vhost_ops = hdev->vhost_ops; + struct vhost_vring_state state; + int r; +hw/virtio/vhost.c int vhost_net_set_backend(struct vhost_dev · line 1925
Quoted 10 lines
hw/virtio/vhost.c +10
-0+ assert(vhost_ops); + + if (vhost_ops->vhost_reset_vring) { + state.index = hdev->vq_index + idx; + r = vhost_ops->vhost_reset_vring(hdev, &state); + if (r < 0) { + goto err_queue_reset; + } + } +So this worries me:
1) having two similar functions
vhost_virtqueue_stop()and
vhost_dev_virtqueue_stop()
Quoted 10 lines
It can easily confuse the people who want stop the device.
I think we need rename
vhost_dev_virtqueue_stop()tovhost_virqtueue_reset()since it has different semantic compared to stop:1) stop means the virtqueue state is reserved, e.g the index could be synced via
get_vring_base()
2) reset means the virqtueue state is lostThanks
Totally agree, will fix.
Thanks
Quoted 11 lines
Quoted 11 lines
hw/virtio/vhost.c +5
-0vhost_virtqueue_unmap(hdev, vdev, hdev->vqs + idx, idx); + + return; + +err_queue_reset: + error_report("Error when stopping the qeuue."); } int vhost_dev_virtqueue_restart(struct vhost_dev *hdev, VirtIODevice *vdev,
PATCH v2 17/24 vhost: vhost-user: update vhost_dev_virtqueue_restart()
In reply to Kangjie XuUpdate vhost_dev_virtqueue_restart() for vhost-user scenario.
--- hw/virtio/vhost.c | 26 ++++++++++++++++++++++---- 1 file changed, 22 insertions(+), 4 deletions(-)
hw/virtio/vhost.c +22 -4
@@ -1937,11 +1937,29 @@ int vhost_dev_virtqueue_restart(struct vhost_dev *hdev, VirtIODevice *vdev, int idx) {const VhostOps *vhost_ops = hdev->vhost_ops; +int r;assert (vhost_ops); -return vhost_virtqueue_start (hdev, - vdev, - hdev->vqs + idx, - hdev->vq_index + idx); + r =vhost_virtqueue_start (hdev, + vdev, + hdev->vqs + idx, + hdev->vq_index + idx); +if (r <0 ) { + goto err_start; + } + +if (vhost_ops->vhost_set_single_vring_enable) { + r = vhost_ops->vhost_set_single_vring_enable (hdev, + hdev->vq_index + idx, +1 ); +if (r <0 ) { + goto err_start; + } + } + +return 0 ; + +err_start: +return r; }
Signature · 1 line
2.32.0
Re: [PATCH v2 17/24] vhost: vhost-user: update vhost_dev_virtqueue_restart()
Quoted 26 lines · Kangjie Xu
Update
vhost_dev_virtqueue_restart()for vhost-user scenario.Signed-off-by Kangjie Xu <[email protected]>Signed-off-by Xuan Zhuo <[email protected]>--- hw/virtio/vhost.c | 26 ++++++++++++++++++++++---- 1 file changed, 22 insertions(+), 4 deletions(-)hw/virtio/vhost.c +3
-4@@ -1937,11 +1937,29 @@ int vhost_dev_virtqueue_restart(struct vhost_dev *hdev, VirtIODevice *vdev, int idx) {const VhostOps *vhost_ops = hdev->vhost_ops; +int r;assert (vhost_ops); -return vhost_virtqueue_start (hdev, - vdev, - hdev->vqs + idx, - hdev->vq_index + idx); + r = vhost_virtqueue_start(hdev, + vdev,
hw/virtio/vhost.c int vhost_dev_virtqueue_restart(struct vhost_dev *hdev, VirtIODevice *vdev, · line 1955
Quoted 10 lines
hw/virtio/vhost.c +10
-0+ hdev->vqs + idx, + hdev->vq_index + idx); +if (r <0 ) { + goto err_start; + } + +if (vhost_ops->vhost_set_single_vring_enable) { + r = vhost_ops->vhost_set_single_vring_enable (hdev, + hdev->vq_index + idx, +1 );
Rethink about the name, I think the current names is kind of confusing:
.vhost_set_single_vring_enable = vhost_user_set_single_vring_enable, .vhost_set_vring_enable = vhost_user_set_vring_enable,
I wonder if it's better:
1) rename vhost_set_vring_enable to vhost_set_dev_enable()
then we can
2) use vhost_set_vring_enable() for per vq enabling?
Thanks
Quoted 10 lines
hw/virtio/vhost.c +9
-0+if (r <0 ) { + goto err_start; + } + } + +return 0 ; + +err_start: +return r; }
Re: [PATCH v2 17/24] vhost: vhost-user: update vhost_dev_virtqueue_restart()
Quoted 46 lines · Jason Wang
Quoted 25 lines · Kangjie Xu
Update
vhost_dev_virtqueue_restart()for vhost-user scenario.Signed-off-by Kangjie Xu <[email protected]>Signed-off-by Xuan Zhuo <[email protected]>---
hw/virtio/vhost.c| 26 ++++++++++++++++++++++----1 file changed, 22 insertions(+), 4 deletions(-)hw/virtio/vhost.c +3
-3@@ -1937,11 +1937,29 @@ int vhost_dev_virtqueue_restart(struct vhost_dev *hdev, VirtIODevice *vdev, int idx) { const VhostOps *vhost_ops = hdev->vhost_ops; + int r; assert(vhost_ops); - return vhost_virtqueue_start(hdev, - vdev, - hdev->vqs + idx, - hdev->vq_index + idx); + r = vhost_virtqueue_start(hdev, + vdev,hw/virtio/vhost.c int vhost_dev_virtqueue_restart(struct · line 1955
Quoted 10 lines
hw/virtio/vhost.c +10
-0+ hdev->vqs + idx, + hdev->vq_index + idx); + if (r < 0) { + goto err_start; + } + + if (vhost_ops->vhost_set_single_vring_enable) { + r = vhost_ops->vhost_set_single_vring_enable(hdev, + hdev->vq_index + idx, + 1);Rethink about the name, I think the current names is kind of confusing:
.vhost_set_single_vring_enable =
vhost_user_set_single_vring_enable, .vhost_set_vring_enable =vhost_user_set_vring_enable,I wonder if it's better:
Quoted 9 lines
1) rename
vhost_set_vring_enabletovhost_set_dev_enable()then we can
2) use
vhost_set_vring_enable()for per vq enabling?Thanks
I agree. It looks better, I will refactor this part.
Thanks.
Quoted 10 lines
Quoted 10 lines
hw/virtio/vhost.c +9
-0+ if (r < 0) { + goto err_start; + } + } + + return 0; + +err_start: + return r; }
PATCH v2 18/24 vhost-net: vhost-user: update vhost_net_virtqueue_stop()
In reply to Kangjie XuUpdate vhost_net_virtqueue_stop() for vhost-user scenario.
--- hw/net/vhost_net.c | 4 ++++ 1 file changed, 4 insertions(+)
hw/net/vhost_net.c +4 -0
@@ -533,6 +533,10 @@ void vhost_net_virtqueue_stop(VirtIODevice *vdev, NetClientState *nc,assert (r >=0 ); } +if (net->nc->info->type ==NET_CLIENT_DRIVER_VHOST_USER ) { + idx = idx - net->dev.vq_index; + } +vhost_dev_virtqueue_stop (&net->dev, vdev, idx); }
Signature · 1 line
2.32.0
Re: [PATCH v2 18/24] vhost-net: vhost-user: update vhost_net_virtqueue_stop()
Quoted 1 line · Kangjie Xu
Update
vhost_net_virtqueue_stop()for vhost-user scenario.
Let's explain why it is needed now or why it doesn't cause any issue or it's a bug fix or not.
Thanks
Quoted 22 lines
Signed-off-by Kangjie Xu <[email protected]>Signed-off-by Xuan Zhuo <[email protected]>--- hw/net/vhost_net.c | 4 ++++ 1 file changed, 4 insertions(+)hw/net/vhost_net.c +4
-0@@ -533,6 +533,10 @@ void vhost_net_virtqueue_stop(VirtIODevice *vdev, NetClientState *nc,assert (r >=0 ); } +if (net->nc->info->type ==NET_CLIENT_DRIVER_VHOST_USER ) { + idx = idx - net->dev.vq_index; + } +vhost_dev_virtqueue_stop (&net->dev, vdev, idx); }
Re: [PATCH v2 18/24] vhost-net: vhost-user: update vhost_net_virtqueue_stop()
Quoted 9 lines · Jason Wang
Quoted 1 line · Kangjie Xu
Update
vhost_net_virtqueue_stop()for vhost-user scenario.Let's explain why it is needed now or why it doesn't cause any issue or it's a bug fix or not.
Thanks
This patch is to suppport vq reset for vhost-user.
We need this simply because the behavior of vhost_ops->get_vq_index() is different in vhost-user and vhost-kernel.
vhost_user_get_vq_index(dev, idx) simply returns "idx".
vhost_kernel_get_vq_index(dev, idx) returns "idx - dev->vq_index".
Thanks
Quoted 22 lines
Quoted 22 lines
Signed-off-by Kangjie Xu <[email protected]>Signed-off-by Xuan Zhuo <[email protected]>---
hw/net/vhost_net.c| 4 ++++1 file changed, 4 insertions(+)hw/net/vhost_net.c +3
-0@@ -533,6 +533,10 @@ void vhost_net_virtqueue_stop(VirtIODevice *vdev, NetClientState *nc, assert(r >= 0); } + if (net->nc->info->type == NET_CLIENT_DRIVER_VHOST_USER) { + idx = idx - net->dev.vq_index; + } + vhost_dev_virtqueue_stop(&net->dev, vdev, idx); }
Re: [PATCH v2 18/24] vhost-net: vhost-user: update vhost_net_virtqueue_stop()
Quoted 13 lines · Kangjie Xu
Quoted 9 lines · Jason Wang
Quoted 1 line · Kangjie Xu
Update
vhost_net_virtqueue_stop()for vhost-user scenario.Let's explain why it is needed now or why it doesn't cause any issue or it's a bug fix or not.
Thanks
This patch is to suppport vq reset for vhost-user.
Quoted 7 lines
We need this simply because the behavior of
vhost_ops->get_vq_index()is different in vhost-user and vhost-kernel.
vhost_user_get_vq_index(dev, idx)simply returns "idx".
vhost_kernel_get_vq_index(dev, idx)returns "idx -dev->vq_index".Thanks
Let's add them in the change-log in the next version.
But the question still, is this a bug fix (requires a Fixes tag)? If not why do we need this now?
Thanks
Quoted 24 lines
Quoted 23 lines
Quoted 23 lines
Signed-off-by Kangjie Xu <[email protected]>Signed-off-by Xuan Zhuo <[email protected]>---
hw/net/vhost_net.c| 4 ++++1 file changed, 4 insertions(+)hw/net/vhost_net.c +3
-0@@ -533,6 +533,10 @@ void vhost_net_virtqueue_stop(VirtIODevice *vdev, NetClientState *nc, assert(r >= 0); } + if (net->nc->info->type == NET_CLIENT_DRIVER_VHOST_USER) { + idx = idx - net->dev.vq_index; + } + vhost_dev_virtqueue_stop(&net->dev, vdev, idx); }
Re: [PATCH v2 18/24] vhost-net: vhost-user: update vhost_net_virtqueue_stop()
Quoted 17 lines · Jason Wang
Quoted 15 lines · Kangjie Xu
Quoted 9 lines · Jason Wang
Quoted 1 line · Kangjie Xu
Update
vhost_net_virtqueue_stop()for vhost-user scenario.Let's explain why it is needed now or why it doesn't cause any issue or it's a bug fix or not.
Thanks
This patch is to suppport vq reset for vhost-user.
We need this simply because the behavior of
vhost_ops->get_vq_index()is different in vhost-user and vhost-kernel.
Quoted 8 lines
Quoted 5 lines
vhost_user_get_vq_index(dev, idx)simply returns "idx".
vhost_kernel_get_vq_index(dev, idx)returns "idx -dev->vq_index".Thanks
Let's add them in the change-log in the next version.
Sorry, i don't get what to be changed here, could you explain it?
Quoted 5 lines
But the question still, is this a bug fix (requires a Fixes tag)? If not why do we need this now?
Thanks
Actually, it is not a bugfix, it is simply intended to support vhost-user.
Because vhost_ops->get_vq_index returns different values for vhost-kernel and vhost-user.
To align vhost-kernel and vhost-user and reuse the following code,
vhost_dev_virtqueue_stop(&net->dev, vdev, idx);
we process the 'idx' here for vhost-user specifically.
Thanks.
Quoted 25 lines
Quoted 25 lines
Quoted 24 lines
Quoted 24 lines
Signed-off-by Kangjie Xu <[email protected]>Signed-off-by Xuan Zhuo <[email protected]>---
hw/net/vhost_net.c| 4 ++++1 file changed, 4 insertions(+)hw/net/vhost_net.c +3
-0@@ -533,6 +533,10 @@ void vhost_net_virtqueue_stop(VirtIODevice *vdev, NetClientState *nc, assert(r >= 0); } + if (net->nc->info->type == NET_CLIENT_DRIVER_VHOST_USER) { + idx = idx - net->dev.vq_index; + } + vhost_dev_virtqueue_stop(&net->dev, vdev, idx); }
PATCH v2 19/24 vhost-net: vhost-user: update vhost_net_virtqueue_restart()
In reply to Kangjie XuUpdate vhost_net_virtqueue_restart() for vhost-user scenario.
--- hw/net/vhost_net.c | 4 ++++ 1 file changed, 4 insertions(+)
hw/net/vhost_net.c +4 -0
@@ -556,6 +556,10 @@ int vhost_net_virtqueue_restart(VirtIODevice *vdev, NetClientState *nc, idx = vhost_ops->vhost_get_vq_index (&net->dev, vq_index); +if (net->nc->info->type ==NET_CLIENT_DRIVER_VHOST_USER ) { + idx = idx - net->dev.vq_index; + } + r =vhost_dev_virtqueue_restart (&net->dev, vdev, idx);if (r <0 ) { goto err_start;
Signature · 1 line
2.32.0
Re: [PATCH v2 19/24] vhost-net: vhost-user: update vhost_net_virtqueue_restart()
Quoted 1 line · Kangjie Xu
Update
vhost_net_virtqueue_restart()for vhost-user scenario.
Similar to previous patch, let's explain why it it needed.
Thanks
Quoted 22 lines
Signed-off-by Kangjie Xu <[email protected]>Signed-off-by Xuan Zhuo <[email protected]>--- hw/net/vhost_net.c | 4 ++++ 1 file changed, 4 insertions(+)hw/net/vhost_net.c +4
-0@@ -556,6 +556,10 @@ int vhost_net_virtqueue_restart(VirtIODevice *vdev, NetClientState *nc, idx = vhost_ops->vhost_get_vq_index (&net->dev, vq_index); +if (net->nc->info->type ==NET_CLIENT_DRIVER_VHOST_USER ) { + idx = idx - net->dev.vq_index; + } + r =vhost_dev_virtqueue_restart (&net->dev, vdev, idx);if (r <0 ) { goto err_start;
PATCH v2 20/24 virtio-net: introduce flush_or_purge_queued_packets()
In reply to Kangjie XuIntroduce the fucntion flush_or_purge_queued_packets(), it will be used in device reset and virtqueue reset. Therefore, we extract the common logic as a new function.
--- hw/net/virtio-net.c | 17 +++++++++++------ 1 file changed, 11 insertions(+), 6 deletions(-)
hw/net/virtio-net.c +11 -6
@@ -118,6 +118,16 @@ static int vq2q(int queue_index)return queue_index /2 ; } +static void flush_or_purge_queued_packets (NetClientState *nc) +{ +if (!nc->peer) { +return ; + } + +qemu_flush_or_purge_queued_packets (nc->peer, true); +assert (!virtio_net_get_subqueue (nc)->async_tx.elem); +} +/* TODO * - we could suppress RX interrupt if we were so inclined. */
@@ -560,12 +570,7 @@ static void virtio_net_reset(VirtIODevice *vdev)/* Flush any async TX */ for (i =0 ; i < n->max_queue_pairs; i++) { -NetClientState *nc =qemu_get_subqueue (n->nic, i); - -if (nc->peer) { -qemu_flush_or_purge_queued_packets (nc->peer, true); -assert (!virtio_net_get_subqueue (nc)->async_tx.elem); - } +flush_or_purge_queued_packets (qemu_get_subqueue (n->nic, i)); } }
Signature · 1 line
2.32.0
PATCH v2 21/24 virtio-net: support queue reset
In reply to Kangjie XuFrom: Xuan Zhuo <[email protected]>
virtio-net, vhost-kernel, vhost-user implement queue reset. Queued packets in the corresponding queue pair are flushed or purged.
--- hw/net/virtio-net.c | 19 +++++++++++++++++++ 1 file changed, 19 insertions(+)
hw/net/virtio-net.c +19 -0
@@ -540,6 +540,24 @@ static RxFilterInfo *virtio_net_query_rxfilter(NetClientState *nc)return info; } +static void virtio_net_queue_reset (VirtIODevice *vdev,uint32_t queue_index) +{ +VirtIONet *n =VIRTIO_NET (vdev); +NetClientState *nc =qemu_get_subqueue (n->nic,vq2q (queue_index)); + +if (!nc->peer) { +return ; + } + +if (get_vhost_net (nc->peer) && + ((nc->peer->info->type ==NET_CLIENT_DRIVER_TAP ) || + (nc->peer->info->type ==NET_CLIENT_DRIVER_VHOST_USER ))) { +vhost_net_virtqueue_stop (vdev, nc, queue_index); + } + +flush_or_purge_queued_packets (nc); +} +static void virtio_net_reset (VirtIODevice *vdev) {VirtIONet *n =VIRTIO_NET (vdev);
@@ -3784,6 +3802,7 @@ static void virtio_net_class_init(ObjectClass *klass, void *data) vdc->set_features = virtio_net_set_features; vdc->bad_features = virtio_net_bad_features; vdc->reset = virtio_net_reset; + vdc->queue_reset = virtio_net_queue_reset; vdc->set_status = virtio_net_set_status; vdc->guest_notifier_mask = virtio_net_guest_notifier_mask; vdc->guest_notifier_pending = virtio_net_guest_notifier_pending;
Signature · 1 line
2.32.0
PATCH v2 22/24 virtio-net: support queue_enable
In reply to Kangjie Xuvirtio-net: support queue_enable for vhost
Support queue_enable in vhost scenario. It can be called when a vq reset operation is performed and the vq will be restared.
It should be noted that we can restart the vq when the vhost has already started. When launching a new vhost-user device, the vhost is not started and all vqs are not initalized until VIRTIO_PCI_COMMON_STATUS is written. Thus, we should use vhost_started to differentiate the two cases: vq reset and device start.
Currently it only supports vhost-user and vhost-kernel.
--- hw/net/virtio-net.c | 22 ++++++++++++++++++++++ 1 file changed, 22 insertions(+)
hw/net/virtio-net.c +22 -0
@@ -558,6 +558,27 @@ static void virtio_net_queue_reset(VirtIODevice *vdev, uint32_t queue_index)flush_or_purge_queued_packets (nc); } +static void virtio_net_queue_enable (VirtIODevice *vdev,uint32_t queue_index) +{ +VirtIONet *n =VIRTIO_NET (vdev); +NetClientState *nc =qemu_get_subqueue (n->nic,vq2q (queue_index)); +int r; + +if (!nc->peer || !vdev->vhost_started) { +return ; + } + +if (get_vhost_net (nc->peer) && + ((nc->peer->info->type ==NET_CLIENT_DRIVER_TAP ) || + (nc->peer->info->type ==NET_CLIENT_DRIVER_VHOST_USER ))) { + r =vhost_net_virtqueue_restart (vdev, nc, queue_index); +if (r <0 ) { +error_report ("unable to restart vhost net virtqueue: %d, " +"when resetting the queue" , queue_index); + } + } +} +static void virtio_net_reset (VirtIODevice *vdev) {VirtIONet *n =VIRTIO_NET (vdev);
@@ -3803,6 +3824,7 @@ static void virtio_net_class_init(ObjectClass *klass, void *data) vdc->bad_features = virtio_net_bad_features; vdc->reset = virtio_net_reset; vdc->queue_reset = virtio_net_queue_reset; + vdc->queue_enable = virtio_net_queue_enable; vdc->set_status = virtio_net_set_status; vdc->guest_notifier_mask = virtio_net_guest_notifier_mask; vdc->guest_notifier_pending = virtio_net_guest_notifier_pending;
Signature · 1 line
2.32.0
PATCH v2 23/24 vhost: vhost-kernel: enable vq reset feature
In reply to Kangjie XuAdd virtqueue reset feature for vhost-kernel.
--- hw/net/vhost_net.c | 1 + 1 file changed, 1 insertion(+)
hw/net/vhost_net.c +1 -0
@@ -46,6 +46,7 @@ static const int kernel_feature_bits[] = {VIRTIO_NET_F_MTU ,VIRTIO_F_IOMMU_PLATFORM ,VIRTIO_F_RING_PACKED , +VIRTIO_F_RING_RESET ,VIRTIO_NET_F_HASH_REPORT , VHOST_INVALID_FEATURE_BIT };
Signature · 1 line
2.32.0
PATCH v2 24/24 vhost: vhost-user: enable vq reset feature
In reply to Kangjie XuAdd virtqueue reset feature for vhost-user.
--- hw/net/vhost_net.c | 1 + 1 file changed, 1 insertion(+)
hw/net/vhost_net.c +1 -0
@@ -74,6 +74,7 @@ static const int user_feature_bits[] = {VIRTIO_NET_F_MTU ,VIRTIO_F_IOMMU_PLATFORM ,VIRTIO_F_RING_PACKED , +VIRTIO_F_RING_RESET ,VIRTIO_NET_F_RSS , VIRTIO_NET_F_HASH_REPORT,
Signature · 1 line
2.32.0
Quoted 73 lines · Kangjie Xu
The virtio queue reset function has already been defined in the virtio spec 1.2. The relevant virtio spec information is here:
https://github.com/oasis-tcs/virtio-spec/issues/124 https://github.com/oasis-tcs/virtio-spec/issues/139This patch set is to support this function in QEMU. It consists of several parts:
1. Patches 1-7 are the basic interfaces for vq reset in virtio and virtio-pci.
2. Patches 8-12 support vq stop and vq restart for vhost-kernel.
3. Patches 13-19 support vq stop and vq restart for vhost-user.
4. Patches 20-22 support vq reset and re-enable for virtio-net.
5. Patches 23-24 enable the vq reset feature for vhost-kernel and vhost-user.The process of virtqueue reset can be concluded as:
1. The virtqueue is disabled whenVIRTIO_PCI_COMMON_Q_RESETis written.
2. Then the virtqueue can be optionally restarted(re-enabled).Since this patch set involves multiple modules and seems a bit messy, we briefly describe the calling process for different modes below.
virtio-net:
1.VIRTIO_PCI_COMMON_Q_RESETis written [virtio-pci]-> virtio_queue_reset() [virtio] -> virtio_net_queue_reset() [virtio-net] -> __virtio_queue_reset() 2. VIRTIO_PCI_COMMON_Q_ENABLE is written [virtio-pci] -> set enabled, reset status of vq.vhost-kernel:
1.VIRTIO_PCI_COMMON_Q_RESETis written [virtio-pci]-> virtio_queue_reset() [virtio] -> virtio_net_queue_reset() [virtio-net] -> vhost_net_virtqueue_stop() [vhost-net] -> vhost_net_set_backend() [vhost] -> vhost_dev_virtqueue_stop() -> vhost_virtqueue_unmap() -> __virtio_queue_reset() 2. VIRTIO_PCI_COMMON_Q_ENABLE is written [virtio-pci] -> virtio_queue_enable() [virtio] -> virtio_net_queue_enable() [virtio-net] -> vhost_net_virtqueue_restart() [vhost-net] -> vhost_dev_virtqueue_restart() [vhost] -> vhost_virtqueue_start() -> vhost_net_set_backend() -> set enabled, reset status of vq.vhost-user:
1.VIRTIO_PCI_COMMON_Q_RESETis written [virtio-pci]-> virtio_queue_reset() [virtio] -> virtio_net_queue_reset() [virtio-net] -> vhost_net_virtqueue_stop() [vhost-net] -> vhost_dev_virtqueue_stop() [vhost] -> vhost_user_reset_vring() [vhost-user] -> send VHOST_USER_RESET_VRING to the device -> vhost_virtqueue_unmap() -> __virtio_queue_reset() 2. VIRTIO_PCI_COMMON_Q_ENABLE is written [virtio-pci] -> virtio_queue_enable() [virtio] -> virtio_net_queue_enable() [virtio-net] -> vhost_net_virtqueue_restart() [vhost-net] -> vhost_dev_virtqueue_restart() [vhost] -> vhost_virtqueue_start() -> vhost_user_set_single_vring_enable [vhost-user] -> send VHOST_USER_SET_VRING_ENABLE to the device -> set enabled, reset status of vq.Test environment:
Host: 5.19.0-rc3 (With vq reset support)
Qemu: QEMU emulator version 7.0.50
Guest: 5.19.0-rc3 (With vq reset support)
DPDK: 22.07-rc1 (With vq reset support)
Test Cmd: ethtool -g eth1; ethtool -G eth1 rx $1 tx $2; ethtool -g eth1;
Quoted 5 lines
The drvier can resize the virtio queue, then virtio queue reset function should be triggered.
The default is split mode, modify Qemu virtio-net to add PACKED feature to test packed mode.
legacy mode testing?
Quoted 58 lines
Guest Kernel Patch:
https://lore.kernel.org/bpf/[email protected]/DPDK Patch:
https://github.com/middaywords/dpdk/compare/72206323a5dd3182b13f61b25a64abdddfee595c...eabadfac7953da66bc10ffb8284b490d09bb7ec7Host Kernel Patch:
https://github.com/middaywords/linux/commit/19a91e0d7167b2031e46078c6215c213b89cb2c3Looking forward to your review and comments. Thanks.
Kangjie Xu (19):
virtio: introducevirtio_queue_enable()
virtio: core: vq reset feature negotation support
virtio-pci: support queue enable
vhost: extract the logic of unmapping the vrings and desc
vhost: introducevhost_dev_virtqueue_stop()
vhost: introducevhost_dev_virtqueue_restart()
vhost-net: vhost-kernel: introducevhost_net_virtqueue_stop()
vhost-net: vhost-kernel: introducevhost_net_virtqueue_restart()
docs: vhost-user: addVHOST_USER_RESET_VRINGmessage
vhost-user: introducevhost_reset_vring()interface
vhost-user: add op to enable or disable a single vring
vhost: vhost-user: updatevhost_dev_virtqueue_stop()
vhost: vhost-user: updatevhost_dev_virtqueue_restart()
vhost-net: vhost-user: updatevhost_net_virtqueue_stop()
vhost-net: vhost-user: updatevhost_net_virtqueue_restart()
virtio-net: introduceflush_or_purge_queued_packets()
virtio-net: supportqueue_enable
vhost: vhost-kernel: enable vq reset feature
vhost: vhost-user: enable vq reset featureXuan Zhuo (5):
virtio: sync relevant definitions with linux
virtio: introduce__virtio_queue_reset()
virtio: introducevirtio_queue_reset()
virtio-pci: support queue reset
virtio-net: support queue resetdocs/interop/vhost-user.rst | 10 +++ hw/core/machine.c | 1 + hw/net/vhost_net.c | 79 +++++++++++++++++++ hw/net/virtio-net.c | 58 ++++++++++++-- hw/virtio/vhost-user.c | 67 ++++++++++++++-- hw/virtio/vhost.c | 79 +++++++++++++++++-- hw/virtio/virtio-pci.c | 20 +++++ hw/virtio/virtio.c | 62 +++++++++++---- include/hw/virtio/vhost-backend.h | 6 ++ include/hw/virtio/vhost.h | 5 ++ include/hw/virtio/virtio-pci.h | 1 + include/hw/virtio/virtio.h | 8 +- include/net/vhost_net.h | 4 + .../standard-headers/linux/virtio_config.h | 5 ++ include/standard-headers/linux/virtio_pci.h | 2 + 15 files changed, 371 insertions(+), 36 deletions(-)Signature · 1 line
2.32.0
Quoted 74 lines · Michael S. Tsirkin
Quoted 73 lines · Kangjie Xu
The virtio queue reset function has already been defined in the virtio spec 1.2. The relevant virtio spec information is here:
https://github.com/oasis-tcs/virtio-spec/issues/124 https://github.com/oasis-tcs/virtio-spec/issues/139This patch set is to support this function in QEMU. It consists of several parts:
1. Patches 1-7 are the basic interfaces for vq reset in virtio and virtio-pci.
2. Patches 8-12 support vq stop and vq restart for vhost-kernel.
3. Patches 13-19 support vq stop and vq restart for vhost-user.
4. Patches 20-22 support vq reset and re-enable for virtio-net.
5. Patches 23-24 enable the vq reset feature for vhost-kernel and vhost-user.The process of virtqueue reset can be concluded as:
1. The virtqueue is disabled whenVIRTIO_PCI_COMMON_Q_RESETis written.
2. Then the virtqueue can be optionally restarted(re-enabled).Since this patch set involves multiple modules and seems a bit messy, we briefly describe the calling process for different modes below.
virtio-net:
1.VIRTIO_PCI_COMMON_Q_RESETis written [virtio-pci]-> virtio_queue_reset() [virtio] -> virtio_net_queue_reset() [virtio-net] -> __virtio_queue_reset() 2. VIRTIO_PCI_COMMON_Q_ENABLE is written [virtio-pci] -> set enabled, reset status of vq.vhost-kernel:
1.VIRTIO_PCI_COMMON_Q_RESETis written [virtio-pci]-> virtio_queue_reset() [virtio] -> virtio_net_queue_reset() [virtio-net] -> vhost_net_virtqueue_stop() [vhost-net] -> vhost_net_set_backend() [vhost] -> vhost_dev_virtqueue_stop() -> vhost_virtqueue_unmap() -> __virtio_queue_reset() 2. VIRTIO_PCI_COMMON_Q_ENABLE is written [virtio-pci] -> virtio_queue_enable() [virtio] -> virtio_net_queue_enable() [virtio-net] -> vhost_net_virtqueue_restart() [vhost-net] -> vhost_dev_virtqueue_restart() [vhost] -> vhost_virtqueue_start() -> vhost_net_set_backend() -> set enabled, reset status of vq.vhost-user:
1.VIRTIO_PCI_COMMON_Q_RESETis written [virtio-pci]-> virtio_queue_reset() [virtio] -> virtio_net_queue_reset() [virtio-net] -> vhost_net_virtqueue_stop() [vhost-net] -> vhost_dev_virtqueue_stop() [vhost] -> vhost_user_reset_vring() [vhost-user] -> send VHOST_USER_RESET_VRING to the device -> vhost_virtqueue_unmap() -> __virtio_queue_reset() 2. VIRTIO_PCI_COMMON_Q_ENABLE is written [virtio-pci] -> virtio_queue_enable() [virtio] -> virtio_net_queue_enable() [virtio-net] -> vhost_net_virtqueue_restart() [vhost-net] -> vhost_dev_virtqueue_restart() [vhost] -> vhost_virtqueue_start() -> vhost_user_set_single_vring_enable [vhost-user] -> send VHOST_USER_SET_VRING_ENABLE to the device -> set enabled, reset status of vq.Test environment:
Host: 5.19.0-rc3 (With vq reset support) Qemu: QEMU emulator version 7.0.50 Guest: 5.19.0-rc3 (With vq reset support) DPDK: 22.07-rc1 (With vq reset support) Test Cmd: ethtool -g eth1; ethtool -G eth1 rx $1 tx $2; ethtool -g eth1;
Quoted 7 lines
Quoted 5 lines
The drvier can resize the virtio queue, then virtio queue reset function should be triggered.
The default is split mode, modify Qemu virtio-net to add PACKED feature to test packed mode.
legacy mode testing?
legacy does not support vq reset.
Thanks.
Quoted 60 lines
Quoted 59 lines
Guest Kernel Patch:
https://lore.kernel.org/bpf/[email protected]/DPDK Patch:
https://github.com/middaywords/dpdk/compare/72206323a5dd3182b13f61b25a64abdddfee595c...eabadfac7953da66bc10ffb8284b490d09bb7ec7Host Kernel Patch:
https://github.com/middaywords/linux/commit/19a91e0d7167b2031e46078c6215c213b89cb2c3Looking forward to your review and comments. Thanks.
Kangjie Xu (19):
virtio: introducevirtio_queue_enable()
virtio: core: vq reset feature negotation support
virtio-pci: support queue enable
vhost: extract the logic of unmapping the vrings and desc
vhost: introducevhost_dev_virtqueue_stop()
vhost: introducevhost_dev_virtqueue_restart()
vhost-net: vhost-kernel: introducevhost_net_virtqueue_stop()
vhost-net: vhost-kernel: introducevhost_net_virtqueue_restart()
docs: vhost-user: addVHOST_USER_RESET_VRINGmessage
vhost-user: introducevhost_reset_vring()interface
vhost-user: add op to enable or disable a single vring
vhost: vhost-user: updatevhost_dev_virtqueue_stop()
vhost: vhost-user: updatevhost_dev_virtqueue_restart()
vhost-net: vhost-user: updatevhost_net_virtqueue_stop()
vhost-net: vhost-user: updatevhost_net_virtqueue_restart()
virtio-net: introduceflush_or_purge_queued_packets()
virtio-net: supportqueue_enable
vhost: vhost-kernel: enable vq reset feature
vhost: vhost-user: enable vq reset featureXuan Zhuo (5):
virtio: sync relevant definitions with linux
virtio: introduce__virtio_queue_reset()
virtio: introducevirtio_queue_reset()
virtio-pci: support queue reset
virtio-net: support queue resetdocs/interop/vhost-user.rst | 10 +++ hw/core/machine.c | 1 + hw/net/vhost_net.c | 79 +++++++++++++++++++ hw/net/virtio-net.c | 58 ++++++++++++-- hw/virtio/vhost-user.c | 67 ++++++++++++++-- hw/virtio/vhost.c | 79 +++++++++++++++++-- hw/virtio/virtio-pci.c | 20 +++++ hw/virtio/virtio.c | 62 +++++++++++---- include/hw/virtio/vhost-backend.h | 6 ++ include/hw/virtio/vhost.h | 5 ++ include/hw/virtio/virtio-pci.h | 1 + include/hw/virtio/virtio.h | 8 +- include/net/vhost_net.h | 4 + .../standard-headers/linux/virtio_config.h | 5 ++ include/standard-headers/linux/virtio_pci.h | 2 + 15 files changed, 371 insertions(+), 36 deletions(-)--
2.32.0
Quoted 81 lines · Xuan Zhuo
Quoted 80 lines · Michael S. Tsirkin
Quoted 78 lines · Kangjie Xu
The virtio queue reset function has already been defined in the virtio spec 1.2. The relevant virtio spec information is here:
https://github.com/oasis-tcs/virtio-spec/issues/124 https://github.com/oasis-tcs/virtio-spec/issues/139This patch set is to support this function in QEMU. It consists of several parts:
1. Patches 1-7 are the basic interfaces for vq reset in virtio and virtio-pci.
2. Patches 8-12 support vq stop and vq restart for vhost-kernel.
3. Patches 13-19 support vq stop and vq restart for vhost-user.
4. Patches 20-22 support vq reset and re-enable for virtio-net.
5. Patches 23-24 enable the vq reset feature for vhost-kernel and vhost-user.The process of virtqueue reset can be concluded as:
1. The virtqueue is disabled whenVIRTIO_PCI_COMMON_Q_RESETis written.
2. Then the virtqueue can be optionally restarted(re-enabled).Since this patch set involves multiple modules and seems a bit messy, we briefly describe the calling process for different modes below.
virtio-net:
1.VIRTIO_PCI_COMMON_Q_RESETis written [virtio-pci]-> virtio_queue_reset() [virtio] -> virtio_net_queue_reset() [virtio-net] -> __virtio_queue_reset() 2. VIRTIO_PCI_COMMON_Q_ENABLE is written [virtio-pci] -> set enabled, reset status of vq.vhost-kernel:
1.VIRTIO_PCI_COMMON_Q_RESETis written [virtio-pci]-> virtio_queue_reset() [virtio] -> virtio_net_queue_reset() [virtio-net] -> vhost_net_virtqueue_stop() [vhost-net] -> vhost_net_set_backend() [vhost] -> vhost_dev_virtqueue_stop() -> vhost_virtqueue_unmap() -> __virtio_queue_reset() 2. VIRTIO_PCI_COMMON_Q_ENABLE is written [virtio-pci] -> virtio_queue_enable() [virtio] -> virtio_net_queue_enable() [virtio-net] -> vhost_net_virtqueue_restart() [vhost-net] -> vhost_dev_virtqueue_restart() [vhost] -> vhost_virtqueue_start() -> vhost_net_set_backend() -> set enabled, reset status of vq.vhost-user:
1.VIRTIO_PCI_COMMON_Q_RESETis written [virtio-pci]-> virtio_queue_reset() [virtio] -> virtio_net_queue_reset() [virtio-net] -> vhost_net_virtqueue_stop() [vhost-net] -> vhost_dev_virtqueue_stop() [vhost] -> vhost_user_reset_vring() [vhost-user] -> send VHOST_USER_RESET_VRING to the device -> vhost_virtqueue_unmap() -> __virtio_queue_reset() 2. VIRTIO_PCI_COMMON_Q_ENABLE is written [virtio-pci] -> virtio_queue_enable() [virtio] -> virtio_net_queue_enable() [virtio-net] -> vhost_net_virtqueue_restart() [vhost-net] -> vhost_dev_virtqueue_restart() [vhost] -> vhost_virtqueue_start() -> vhost_user_set_single_vring_enable [vhost-user] -> send VHOST_USER_SET_VRING_ENABLE to the device -> set enabled, reset status of vq.Test environment:
Host: 5.19.0-rc3 (With vq reset support)
Qemu: QEMU emulator version 7.0.50
Guest: 5.19.0-rc3 (With vq reset support)
DPDK: 22.07-rc1 (With vq reset support)
Test Cmd: ethtool -g eth1; ethtool -G eth1 rx $1 tx $2; ethtool -g eth1;The drvier can resize the virtio queue, then virtio queue reset function should be triggered.
The default is split mode, modify Qemu virtio-net to add PACKED feature to test packed mode.
Quoted 6 lines
Quoted 1 line
legacy mode testing?
legacy does not support vq reset.
Thanks.
yes but did it break with all these code changes?
Quoted 60 lines
Quoted 60 lines
Quoted 59 lines
Guest Kernel Patch:
https://lore.kernel.org/bpf/[email protected]/DPDK Patch:
https://github.com/middaywords/dpdk/compare/72206323a5dd3182b13f61b25a64abdddfee595c...eabadfac7953da66bc10ffb8284b490d09bb7ec7Host Kernel Patch:
https://github.com/middaywords/linux/commit/19a91e0d7167b2031e46078c6215c213b89cb2c3Looking forward to your review and comments. Thanks.
Kangjie Xu (19):
virtio: introducevirtio_queue_enable()
virtio: core: vq reset feature negotation support
virtio-pci: support queue enable
vhost: extract the logic of unmapping the vrings and desc
vhost: introducevhost_dev_virtqueue_stop()
vhost: introducevhost_dev_virtqueue_restart()
vhost-net: vhost-kernel: introducevhost_net_virtqueue_stop()
vhost-net: vhost-kernel: introducevhost_net_virtqueue_restart()
docs: vhost-user: addVHOST_USER_RESET_VRINGmessage
vhost-user: introducevhost_reset_vring()interface
vhost-user: add op to enable or disable a single vring
vhost: vhost-user: updatevhost_dev_virtqueue_stop()
vhost: vhost-user: updatevhost_dev_virtqueue_restart()
vhost-net: vhost-user: updatevhost_net_virtqueue_stop()
vhost-net: vhost-user: updatevhost_net_virtqueue_restart()
virtio-net: introduceflush_or_purge_queued_packets()
virtio-net: supportqueue_enable
vhost: vhost-kernel: enable vq reset feature
vhost: vhost-user: enable vq reset featureXuan Zhuo (5):
virtio: sync relevant definitions with linux
virtio: introduce__virtio_queue_reset()
virtio: introducevirtio_queue_reset()
virtio-pci: support queue reset
virtio-net: support queue resetdocs/interop/vhost-user.rst | 10 +++ hw/core/machine.c | 1 + hw/net/vhost_net.c | 79 +++++++++++++++++++ hw/net/virtio-net.c | 58 ++++++++++++-- hw/virtio/vhost-user.c | 67 ++++++++++++++-- hw/virtio/vhost.c | 79 +++++++++++++++++-- hw/virtio/virtio-pci.c | 20 +++++ hw/virtio/virtio.c | 62 +++++++++++---- include/hw/virtio/vhost-backend.h | 6 ++ include/hw/virtio/vhost.h | 5 ++ include/hw/virtio/virtio-pci.h | 1 + include/hw/virtio/virtio.h | 8 +- include/net/vhost_net.h | 4 + .../standard-headers/linux/virtio_config.h | 5 ++ include/standard-headers/linux/virtio_pci.h | 2 + 15 files changed, 371 insertions(+), 36 deletions(-)--
2.32.0
Quoted 82 lines · Michael S. Tsirkin
Quoted 81 lines · Xuan Zhuo
Quoted 80 lines · Michael S. Tsirkin
Quoted 78 lines · Kangjie Xu
The virtio queue reset function has already been defined in the virtio spec 1.2. The relevant virtio spec information is here:
https://github.com/oasis-tcs/virtio-spec/issues/124 https://github.com/oasis-tcs/virtio-spec/issues/139This patch set is to support this function in QEMU. It consists of several parts:
1. Patches 1-7 are the basic interfaces for vq reset in virtio and virtio-pci.
2. Patches 8-12 support vq stop and vq restart for vhost-kernel.
3. Patches 13-19 support vq stop and vq restart for vhost-user.
4. Patches 20-22 support vq reset and re-enable for virtio-net.
5. Patches 23-24 enable the vq reset feature for vhost-kernel and vhost-user.The process of virtqueue reset can be concluded as:
1. The virtqueue is disabled whenVIRTIO_PCI_COMMON_Q_RESETis written.
2. Then the virtqueue can be optionally restarted(re-enabled).Since this patch set involves multiple modules and seems a bit messy, we briefly describe the calling process for different modes below.
virtio-net:
1.VIRTIO_PCI_COMMON_Q_RESETis written [virtio-pci]-> virtio_queue_reset() [virtio] -> virtio_net_queue_reset() [virtio-net] -> __virtio_queue_reset() 2. VIRTIO_PCI_COMMON_Q_ENABLE is written [virtio-pci] -> set enabled, reset status of vq.vhost-kernel:
1.VIRTIO_PCI_COMMON_Q_RESETis written [virtio-pci]-> virtio_queue_reset() [virtio] -> virtio_net_queue_reset() [virtio-net] -> vhost_net_virtqueue_stop() [vhost-net] -> vhost_net_set_backend() [vhost] -> vhost_dev_virtqueue_stop() -> vhost_virtqueue_unmap() -> __virtio_queue_reset() 2. VIRTIO_PCI_COMMON_Q_ENABLE is written [virtio-pci] -> virtio_queue_enable() [virtio] -> virtio_net_queue_enable() [virtio-net] -> vhost_net_virtqueue_restart() [vhost-net] -> vhost_dev_virtqueue_restart() [vhost] -> vhost_virtqueue_start() -> vhost_net_set_backend() -> set enabled, reset status of vq.vhost-user:
1.VIRTIO_PCI_COMMON_Q_RESETis written [virtio-pci]-> virtio_queue_reset() [virtio] -> virtio_net_queue_reset() [virtio-net] -> vhost_net_virtqueue_stop() [vhost-net] -> vhost_dev_virtqueue_stop() [vhost] -> vhost_user_reset_vring() [vhost-user] -> send VHOST_USER_RESET_VRING to the device -> vhost_virtqueue_unmap() -> __virtio_queue_reset() 2. VIRTIO_PCI_COMMON_Q_ENABLE is written [virtio-pci] -> virtio_queue_enable() [virtio] -> virtio_net_queue_enable() [virtio-net] -> vhost_net_virtqueue_restart() [vhost-net] -> vhost_dev_virtqueue_restart() [vhost] -> vhost_virtqueue_start() -> vhost_user_set_single_vring_enable [vhost-user] -> send VHOST_USER_SET_VRING_ENABLE to the device -> set enabled, reset status of vq.Test environment:
Host: 5.19.0-rc3 (With vq reset support)
Qemu: QEMU emulator version 7.0.50
Guest: 5.19.0-rc3 (With vq reset support)
DPDK: 22.07-rc1 (With vq reset support)
Test Cmd: ethtool -g eth1; ethtool -G eth1 rx $1 tx $2; ethtool -g eth1;The drvier can resize the virtio queue, then virtio queue reset function should be triggered.
The default is split mode, modify Qemu virtio-net to add PACKED feature to test packed mode.
Quoted 8 lines
Quoted 6 lines
Quoted 1 line
legacy mode testing?
legacy does not support vq reset.
Thanks.
yes but did it break with all these code changes?
OK, I see, we'll test this.
Thanks.
Quoted 62 lines
Quoted 61 lines
Quoted 61 lines
Quoted 60 lines
Guest Kernel Patch:
https://lore.kernel.org/bpf/[email protected]/DPDK Patch:
https://github.com/middaywords/dpdk/compare/72206323a5dd3182b13f61b25a64abdddfee595c...eabadfac7953da66bc10ffb8284b490d09bb7ec7Host Kernel Patch:
https://github.com/middaywords/linux/commit/19a91e0d7167b2031e46078c6215c213b89cb2c3Looking forward to your review and comments. Thanks.
Kangjie Xu (19):
virtio: introducevirtio_queue_enable()
virtio: core: vq reset feature negotation support
virtio-pci: support queue enable
vhost: extract the logic of unmapping the vrings and desc
vhost: introducevhost_dev_virtqueue_stop()
vhost: introducevhost_dev_virtqueue_restart()
vhost-net: vhost-kernel: introducevhost_net_virtqueue_stop()
vhost-net: vhost-kernel: introducevhost_net_virtqueue_restart()
docs: vhost-user: addVHOST_USER_RESET_VRINGmessage
vhost-user: introducevhost_reset_vring()interface
vhost-user: add op to enable or disable a single vring
vhost: vhost-user: updatevhost_dev_virtqueue_stop()
vhost: vhost-user: updatevhost_dev_virtqueue_restart()
vhost-net: vhost-user: updatevhost_net_virtqueue_stop()
vhost-net: vhost-user: updatevhost_net_virtqueue_restart()
virtio-net: introduceflush_or_purge_queued_packets()
virtio-net: supportqueue_enable
vhost: vhost-kernel: enable vq reset feature
vhost: vhost-user: enable vq reset featureXuan Zhuo (5):
virtio: sync relevant definitions with linux
virtio: introduce__virtio_queue_reset()
virtio: introducevirtio_queue_reset()
virtio-pci: support queue reset
virtio-net: support queue resetdocs/interop/vhost-user.rst | 10 +++ hw/core/machine.c | 1 + hw/net/vhost_net.c | 79 +++++++++++++++++++ hw/net/virtio-net.c | 58 ++++++++++++-- hw/virtio/vhost-user.c | 67 ++++++++++++++-- hw/virtio/vhost.c | 79 +++++++++++++++++-- hw/virtio/virtio-pci.c | 20 +++++ hw/virtio/virtio.c | 62 +++++++++++---- include/hw/virtio/vhost-backend.h | 6 ++ include/hw/virtio/vhost.h | 5 ++ include/hw/virtio/virtio-pci.h | 1 + include/hw/virtio/virtio.h | 8 +- include/net/vhost_net.h | 4 + .../standard-headers/linux/virtio_config.h | 5 ++ include/standard-headers/linux/virtio_pci.h | 2 + 15 files changed, 371 insertions(+), 36 deletions(-)--
2.32.0
Quoted 79 lines · Michael S. Tsirkin
Quoted 78 lines · Xuan Zhuo
Quoted 77 lines · Michael S. Tsirkin
Quoted 76 lines · Kangjie Xu
The virtio queue reset function has already been defined in the virtio spec 1.2. The relevant virtio spec information is here:
https://github.com/oasis-tcs/virtio-spec/issues/124 https://github.com/oasis-tcs/virtio-spec/issues/139This patch set is to support this function in QEMU. It consists of several parts:
1. Patches 1-7 are the basic interfaces for vq reset in virtio and virtio-pci.
2. Patches 8-12 support vq stop and vq restart for vhost-kernel.
3. Patches 13-19 support vq stop and vq restart for vhost-user.
4. Patches 20-22 support vq reset and re-enable for virtio-net.
5. Patches 23-24 enable the vq reset feature for vhost-kernel and vhost-user.The process of virtqueue reset can be concluded as:
1. The virtqueue is disabled whenVIRTIO_PCI_COMMON_Q_RESETis written.
2. Then the virtqueue can be optionally restarted(re-enabled).Since this patch set involves multiple modules and seems a bit messy, we briefly describe the calling process for different modes below.
virtio-net:
1.VIRTIO_PCI_COMMON_Q_RESETis written [virtio-pci]-> virtio_queue_reset() [virtio] -> virtio_net_queue_reset() [virtio-net] -> __virtio_queue_reset() 2. VIRTIO_PCI_COMMON_Q_ENABLE is written [virtio-pci] -> set enabled, reset status of vq.vhost-kernel:
1.VIRTIO_PCI_COMMON_Q_RESETis written [virtio-pci]-> virtio_queue_reset() [virtio] -> virtio_net_queue_reset() [virtio-net] -> vhost_net_virtqueue_stop() [vhost-net] -> vhost_net_set_backend() [vhost] -> vhost_dev_virtqueue_stop() -> vhost_virtqueue_unmap() -> __virtio_queue_reset() 2. VIRTIO_PCI_COMMON_Q_ENABLE is written [virtio-pci] -> virtio_queue_enable() [virtio] -> virtio_net_queue_enable() [virtio-net] -> vhost_net_virtqueue_restart() [vhost-net] -> vhost_dev_virtqueue_restart() [vhost] -> vhost_virtqueue_start() -> vhost_net_set_backend() -> set enabled, reset status of vq.vhost-user:
1.VIRTIO_PCI_COMMON_Q_RESETis written [virtio-pci]-> virtio_queue_reset() [virtio] -> virtio_net_queue_reset() [virtio-net] -> vhost_net_virtqueue_stop() [vhost-net] -> vhost_dev_virtqueue_stop() [vhost] -> vhost_user_reset_vring() [vhost-user] -> send VHOST_USER_RESET_VRING to the device -> vhost_virtqueue_unmap() -> __virtio_queue_reset() 2. VIRTIO_PCI_COMMON_Q_ENABLE is written [virtio-pci] -> virtio_queue_enable() [virtio] -> virtio_net_queue_enable() [virtio-net] -> vhost_net_virtqueue_restart() [vhost-net] -> vhost_dev_virtqueue_restart() [vhost] -> vhost_virtqueue_start() -> vhost_user_set_single_vring_enable [vhost-user] -> send VHOST_USER_SET_VRING_ENABLE to the device -> set enabled, reset status of vq.Test environment:
Host: 5.19.0-rc3 (With vq reset support) Qemu: QEMU emulator version 7.0.50 Guest: 5.19.0-rc3 (With vq reset support) DPDK: 22.07-rc1 (With vq reset support) Test Cmd: ethtool -g eth1; ethtool -G eth1 rx $1 tx $2; ethtool -g eth1; The drvier can resize the virtio queue, then virtio queue reset function should be triggered.
Quoted 8 lines
Quoted 7 lines
Quoted 3 lines
Quoted 2 lines
The default is split mode, modify Qemu virtio-net to add PACKED feature to test packed mode.
legacy mode testing?
legacy does not support vq reset.
Thanks.
yes but did it break with all these code changes?
For pci legacy mode, we tested resizing the vq using 'ethtool -G', it reported an error code -ENOENT because the transport did not support the vq reset feature.
We also tested the connectivity and performance in pci legacy mode using iperf 3.9. We placed a client in the host and a server in the guest, or swapped the them.
In all the cases, the connectivity did not break down. The performance was about the same with all our code changes.
Thanks.
Quoted 58 lines
Quoted 58 lines
Quoted 58 lines
Quoted 58 lines
Guest Kernel Patch:
https://lore.kernel.org/bpf/[email protected]/DPDK Patch:
https://github.com/middaywords/dpdk/compare/72206323a5dd3182b13f61b25a64abdddfee595c...eabadfac7953da66bc10ffb8284b490d09bb7ec7Host Kernel Patch:
https://github.com/middaywords/linux/commit/19a91e0d7167b2031e46078c6215c213b89cb2c3Looking forward to your review and comments. Thanks.
Kangjie Xu (19):
virtio: introducevirtio_queue_enable()
virtio: core: vq reset feature negotation support
virtio-pci: support queue enable
vhost: extract the logic of unmapping the vrings and desc
vhost: introducevhost_dev_virtqueue_stop()
vhost: introducevhost_dev_virtqueue_restart()
vhost-net: vhost-kernel: introducevhost_net_virtqueue_stop()
vhost-net: vhost-kernel: introducevhost_net_virtqueue_restart()
docs: vhost-user: addVHOST_USER_RESET_VRINGmessage
vhost-user: introducevhost_reset_vring()interface
vhost-user: add op to enable or disable a single vring
vhost: vhost-user: updatevhost_dev_virtqueue_stop()
vhost: vhost-user: updatevhost_dev_virtqueue_restart()
vhost-net: vhost-user: updatevhost_net_virtqueue_stop()
vhost-net: vhost-user: updatevhost_net_virtqueue_restart()
virtio-net: introduceflush_or_purge_queued_packets()
virtio-net: supportqueue_enable
vhost: vhost-kernel: enable vq reset feature
vhost: vhost-user: enable vq reset featureXuan Zhuo (5):
virtio: sync relevant definitions with linux
virtio: introduce__virtio_queue_reset()
virtio: introducevirtio_queue_reset()
virtio-pci: support queue reset
virtio-net: support queue resetdocs/interop/vhost-user.rst | 10 +++ hw/core/machine.c | 1 + hw/net/vhost_net.c | 79 +++++++++++++++++++ hw/net/virtio-net.c | 58 ++++++++++++-- hw/virtio/vhost-user.c | 67 ++++++++++++++-- hw/virtio/vhost.c | 79 +++++++++++++++++-- hw/virtio/virtio-pci.c | 20 +++++ hw/virtio/virtio.c | 62 +++++++++++---- include/hw/virtio/vhost-backend.h | 6 ++ include/hw/virtio/vhost.h | 5 ++ include/hw/virtio/virtio-pci.h | 1 + include/hw/virtio/virtio.h | 8 +- include/net/vhost_net.h | 4 + .../standard-headers/linux/virtio_config.h | 5 ++ include/standard-headers/linux/virtio_pci.h | 2 + 15 files changed, 371 insertions(+), 36 deletions(-)--
2.32.0
Forgot to append changelog for v2 in the cover letter, so I add it in this email.
changelog:
1. Add support for vhost-net scenario
2. Add a new vhost-user message VHOST_USER_RESET_VRING
3. Add migration compatibility for virtqueue reset
Looking forward to your review and comments to this patch set, thanks. :)
Best regards,
Kangjie
Quoted 135 lines · Kangjie Xu
The virtio queue reset function has already been defined in the virtio spec 1.2. The relevant virtio spec information is here:
https://github.com/oasis-tcs/virtio-spec/issues/124 https://github.com/oasis-tcs/virtio-spec/issues/139This patch set is to support this function in QEMU. It consists of several parts:
1. Patches 1-7 are the basic interfaces for vq reset in virtio and virtio-pci.
2. Patches 8-12 support vq stop and vq restart for vhost-kernel.
3. Patches 13-19 support vq stop and vq restart for vhost-user.
4. Patches 20-22 support vq reset and re-enable for virtio-net.
5. Patches 23-24 enable the vq reset feature for vhost-kernel and vhost-user.The process of virtqueue reset can be concluded as:
1. The virtqueue is disabled whenVIRTIO_PCI_COMMON_Q_RESETis written.
2. Then the virtqueue can be optionally restarted(re-enabled).Since this patch set involves multiple modules and seems a bit messy, we briefly describe the calling process for different modes below.
virtio-net:
1.VIRTIO_PCI_COMMON_Q_RESETis written [virtio-pci]-> virtio_queue_reset() [virtio] -> virtio_net_queue_reset() [virtio-net] -> __virtio_queue_reset() 2. VIRTIO_PCI_COMMON_Q_ENABLE is written [virtio-pci] -> set enabled, reset status of vq.vhost-kernel:
1.VIRTIO_PCI_COMMON_Q_RESETis written [virtio-pci]-> virtio_queue_reset() [virtio] -> virtio_net_queue_reset() [virtio-net] -> vhost_net_virtqueue_stop() [vhost-net] -> vhost_net_set_backend() [vhost] -> vhost_dev_virtqueue_stop() -> vhost_virtqueue_unmap() -> __virtio_queue_reset() 2. VIRTIO_PCI_COMMON_Q_ENABLE is written [virtio-pci] -> virtio_queue_enable() [virtio] -> virtio_net_queue_enable() [virtio-net] -> vhost_net_virtqueue_restart() [vhost-net] -> vhost_dev_virtqueue_restart() [vhost] -> vhost_virtqueue_start() -> vhost_net_set_backend() -> set enabled, reset status of vq.vhost-user:
1.VIRTIO_PCI_COMMON_Q_RESETis written [virtio-pci]-> virtio_queue_reset() [virtio] -> virtio_net_queue_reset() [virtio-net] -> vhost_net_virtqueue_stop() [vhost-net] -> vhost_dev_virtqueue_stop() [vhost] -> vhost_user_reset_vring() [vhost-user] -> send VHOST_USER_RESET_VRING to the device -> vhost_virtqueue_unmap() -> __virtio_queue_reset() 2. VIRTIO_PCI_COMMON_Q_ENABLE is written [virtio-pci] -> virtio_queue_enable() [virtio] -> virtio_net_queue_enable() [virtio-net] -> vhost_net_virtqueue_restart() [vhost-net] -> vhost_dev_virtqueue_restart() [vhost] -> vhost_virtqueue_start() -> vhost_user_set_single_vring_enable [vhost-user] -> send VHOST_USER_SET_VRING_ENABLE to the device -> set enabled, reset status of vq.Test environment:
Host: 5.19.0-rc3 (With vq reset support)
Qemu: QEMU emulator version 7.0.50
Guest: 5.19.0-rc3 (With vq reset support)
DPDK: 22.07-rc1 (With vq reset support)
Test Cmd: ethtool -g eth1; ethtool -G eth1 rx $1 tx $2; ethtool -g eth1;The drvier can resize the virtio queue, then virtio queue reset function should be triggered.
The default is split mode, modify Qemu virtio-net to add PACKED feature to test packed mode.
Guest Kernel Patch:
https://lore.kernel.org/bpf/[email protected]/DPDK Patch:
https://github.com/middaywords/dpdk/compare/72206323a5dd3182b13f61b25a64abdddfee595c...eabadfac7953da66bc10ffb8284b490d09bb7ec7Host Kernel Patch:
https://github.com/middaywords/linux/commit/19a91e0d7167b2031e46078c6215c213b89cb2c3Looking forward to your review and comments. Thanks.
Kangjie Xu (19):
virtio: introducevirtio_queue_enable()
virtio: core: vq reset feature negotation support
virtio-pci: support queue enable
vhost: extract the logic of unmapping the vrings and desc
vhost: introducevhost_dev_virtqueue_stop()
vhost: introducevhost_dev_virtqueue_restart()
vhost-net: vhost-kernel: introducevhost_net_virtqueue_stop()
vhost-net: vhost-kernel: introducevhost_net_virtqueue_restart()
docs: vhost-user: addVHOST_USER_RESET_VRINGmessage
vhost-user: introducevhost_reset_vring()interface
vhost-user: add op to enable or disable a single vring
vhost: vhost-user: updatevhost_dev_virtqueue_stop()
vhost: vhost-user: updatevhost_dev_virtqueue_restart()
vhost-net: vhost-user: updatevhost_net_virtqueue_stop()
vhost-net: vhost-user: updatevhost_net_virtqueue_restart()
virtio-net: introduceflush_or_purge_queued_packets()
virtio-net: supportqueue_enable
vhost: vhost-kernel: enable vq reset feature
vhost: vhost-user: enable vq reset featureXuan Zhuo (5):
virtio: sync relevant definitions with linux
virtio: introduce__virtio_queue_reset()
virtio: introducevirtio_queue_reset()
virtio-pci: support queue reset
virtio-net: support queue resetdocs/interop/vhost-user.rst | 10 +++ hw/core/machine.c | 1 + hw/net/vhost_net.c | 79 +++++++++++++++++++ hw/net/virtio-net.c | 58 ++++++++++++-- hw/virtio/vhost-user.c | 67 ++++++++++++++-- hw/virtio/vhost.c | 79 +++++++++++++++++-- hw/virtio/virtio-pci.c | 20 +++++ hw/virtio/virtio.c | 62 +++++++++++---- include/hw/virtio/vhost-backend.h | 6 ++ include/hw/virtio/vhost.h | 5 ++ include/hw/virtio/virtio-pci.h | 1 + include/hw/virtio/virtio.h | 8 +- include/net/vhost_net.h | 4 + .../standard-headers/linux/virtio_config.h | 5 ++ include/standard-headers/linux/virtio_pci.h | 2 + 15 files changed, 371 insertions(+), 36 deletions(-)
Quoted 82 lines · Kangjie Xu
The virtio queue reset function has already been defined in the virtio spec 1.2. The relevant virtio spec information is here:
https://github.com/oasis-tcs/virtio-spec/issues/124 https://github.com/oasis-tcs/virtio-spec/issues/139This patch set is to support this function in QEMU. It consists of several parts:
1. Patches 1-7 are the basic interfaces for vq reset in virtio and virtio-pci.
2. Patches 8-12 support vq stop and vq restart for vhost-kernel.
3. Patches 13-19 support vq stop and vq restart for vhost-user.
4. Patches 20-22 support vq reset and re-enable for virtio-net.
5. Patches 23-24 enable the vq reset feature for vhost-kernel and vhost-user.The process of virtqueue reset can be concluded as:
1. The virtqueue is disabled whenVIRTIO_PCI_COMMON_Q_RESETis written.
2. Then the virtqueue can be optionally restarted(re-enabled).Since this patch set involves multiple modules and seems a bit messy, we briefly describe the calling process for different modes below.
virtio-net:
1.VIRTIO_PCI_COMMON_Q_RESETis written [virtio-pci]-> virtio_queue_reset() [virtio] -> virtio_net_queue_reset() [virtio-net] -> __virtio_queue_reset() 2. VIRTIO_PCI_COMMON_Q_ENABLE is written [virtio-pci] -> set enabled, reset status of vq.vhost-kernel:
1.VIRTIO_PCI_COMMON_Q_RESETis written [virtio-pci]-> virtio_queue_reset() [virtio] -> virtio_net_queue_reset() [virtio-net] -> vhost_net_virtqueue_stop() [vhost-net] -> vhost_net_set_backend() [vhost] -> vhost_dev_virtqueue_stop() -> vhost_virtqueue_unmap() -> __virtio_queue_reset() 2. VIRTIO_PCI_COMMON_Q_ENABLE is written [virtio-pci] -> virtio_queue_enable() [virtio] -> virtio_net_queue_enable() [virtio-net] -> vhost_net_virtqueue_restart() [vhost-net] -> vhost_dev_virtqueue_restart() [vhost] -> vhost_virtqueue_start() -> vhost_net_set_backend() -> set enabled, reset status of vq.vhost-user:
1.VIRTIO_PCI_COMMON_Q_RESETis written [virtio-pci]-> virtio_queue_reset() [virtio] -> virtio_net_queue_reset() [virtio-net] -> vhost_net_virtqueue_stop() [vhost-net] -> vhost_dev_virtqueue_stop() [vhost] -> vhost_user_reset_vring() [vhost-user] -> send VHOST_USER_RESET_VRING to the device -> vhost_virtqueue_unmap() -> __virtio_queue_reset() 2. VIRTIO_PCI_COMMON_Q_ENABLE is written [virtio-pci] -> virtio_queue_enable() [virtio] -> virtio_net_queue_enable() [virtio-net] -> vhost_net_virtqueue_restart() [vhost-net] -> vhost_dev_virtqueue_restart() [vhost] -> vhost_virtqueue_start() -> vhost_user_set_single_vring_enable [vhost-user] -> send VHOST_USER_SET_VRING_ENABLE to the device -> set enabled, reset status of vq.Test environment:
Host: 5.19.0-rc3 (With vq reset support)
Qemu: QEMU emulator version 7.0.50
Guest: 5.19.0-rc3 (With vq reset support)
DPDK: 22.07-rc1 (With vq reset support)
Test Cmd: ethtool -g eth1; ethtool -G eth1 rx $1 tx $2; ethtool -g eth1;The drvier can resize the virtio queue, then virtio queue reset function should be triggered.
The default is split mode, modify Qemu virtio-net to add PACKED feature to test packed mode.
Guest Kernel Patch:
https://lore.kernel.org/bpf/[email protected]/
Quoted 7 lines
DPDK Patch:
https://github.com/middaywords/dpdk/compare/72206323a5dd3182b13f61b25a64abdddfee595c...eabadfac7953da66bc10ffb8284b490d09bb7ec7Host Kernel Patch:
https://github.com/middaywords/linux/commit/19a91e0d7167b2031e46078c6215c213b89cb2c3Looking forward to your review and comments. Thanks.
I suggest to split vhost-user part into another series since:
1) vhost-net kernel is ready for this future
2) vhost-user probably requires some extensions of the current protocol
Then it can speed up the merging and reduce the rebasing of a huge series.
Thanks
Quoted 46 lines
Kangjie Xu (19):
virtio: introducevirtio_queue_enable()
virtio: core: vq reset feature negotation support
virtio-pci: support queue enable
vhost: extract the logic of unmapping the vrings and desc
vhost: introducevhost_dev_virtqueue_stop()
vhost: introducevhost_dev_virtqueue_restart()
vhost-net: vhost-kernel: introducevhost_net_virtqueue_stop()
vhost-net: vhost-kernel: introducevhost_net_virtqueue_restart()
docs: vhost-user: addVHOST_USER_RESET_VRINGmessage
vhost-user: introducevhost_reset_vring()interface
vhost-user: add op to enable or disable a single vring
vhost: vhost-user: updatevhost_dev_virtqueue_stop()
vhost: vhost-user: updatevhost_dev_virtqueue_restart()
vhost-net: vhost-user: updatevhost_net_virtqueue_stop()
vhost-net: vhost-user: updatevhost_net_virtqueue_restart()
virtio-net: introduceflush_or_purge_queued_packets()
virtio-net: supportqueue_enable
vhost: vhost-kernel: enable vq reset feature
vhost: vhost-user: enable vq reset featureXuan Zhuo (5):
virtio: sync relevant definitions with linux
virtio: introduce__virtio_queue_reset()
virtio: introducevirtio_queue_reset()
virtio-pci: support queue reset
virtio-net: support queue resetdocs/interop/vhost-user.rst | 10 +++ hw/core/machine.c | 1 + hw/net/vhost_net.c | 79 +++++++++++++++++++ hw/net/virtio-net.c | 58 ++++++++++++-- hw/virtio/vhost-user.c | 67 ++++++++++++++-- hw/virtio/vhost.c | 79 +++++++++++++++++-- hw/virtio/virtio-pci.c | 20 +++++ hw/virtio/virtio.c | 62 +++++++++++---- include/hw/virtio/vhost-backend.h | 6 ++ include/hw/virtio/vhost.h | 5 ++ include/hw/virtio/virtio-pci.h | 1 + include/hw/virtio/virtio.h | 8 +- include/net/vhost_net.h | 4 + .../standard-headers/linux/virtio_config.h | 5 ++ include/standard-headers/linux/virtio_pci.h | 2 + 15 files changed, 371 insertions(+), 36 deletions(-)
Quoted 2 lines · Kangjie Xu
Host Kernel Patch:
https://github.com/middaywords/linux/commit/19a91e0d7167b2031e46078c6215c213b89cb2c3
Btw, this patch could be posted right now. Then it would be more easier for people to try you series.
Thanks