Support VIRTIO_F_RING_RESET for virtio-net and vhost-user in virtio pci
cover Support VIRTIO_F_RING_RESET for virtio-net and vhost-user in virtio pci
The virtio queue reset function has already been defined in the virtio spec 1.2. The relevant virtio spec information is here:
https://github.com/oasis-tcs/virtio-spec/issues/124 https://github.com/oasis-tcs/virtio-spec/issues/139
This patch set is to support this function in QEMU. It consists of two parts: virtio-net and vhost-user. The patches 1-7 are the implementation for virtio-net and the patches 8-16 are for vhost-user.
The process of virt queue reset can be concluded as:
1. The virtqueue is disabled when VIRTIO_PCI_COMMON_Q_RESET is written.
2. Then the virtqueue is restarted after the information of vrings is passed to QEMU and VIRTIO_PCI_COMMON_Q_ENABLE is written.
Test environment:
Host: 5.4.189
Qemu: QEMU emulator version 7.0.50
Guest: 5.19.0-rc3 (With vq reset support)
DPDK: 22.07-rc1 (With vq reset support)
Test Cmd: ethtool -g eth1; ethtool -G eth1 rx $1 tx $2; ethtool -g eth1;
The frontend can resize the virtio queue, then virtio queue reset function should be triggered.
The default is split mode, modify Qemu virtio-net to add PACKED feature to test packed mode.
Guest Kernel Patch:
https://lore.kernel.org/bpf/[email protected]/
DPDK Code:
https://github.com/middaywords/dpdk/commit/098c8e1dfae10b747da8dd8950a82890efca7bee
Please review. Thanks.
Kangjie Xu (9):
vhost: add op to enable or disable a single vring
vhost-user: enable/disable a single vring
vhost: extract the logic of unmapping the vrings and desc
vhost: introduce restart and release for vhost_dev's vqs
vhost-net: introduce restart and stop for vhost_net's vqs
virtio: introduce queue_enable in virtio
virtio-net: support queue_enable for vhost-user
virtio-net: support queue_reset for vhost-user
vhost-net: vq reset feature bit support
Xuan Zhuo (7):
virtio-pci: virtio_pci_common_cfg add queue_notify_data
virtio: add VIRTIO_F_RING_RESET
virtio: pci: virtio_pci_common_cfg add queue_reset
virtio: introduce __virtio_queue_reset()
virtio: introduce virtio_queue_reset()
virtio-pci: support queue reset
virtio-net: support queue reset
hw/net/vhost_net.c | 56 ++++++++++++++++++ hw/net/virtio-net.c | 40 +++++++++++++ hw/virtio/vhost-user.c | 55 +++++++++++++++--- hw/virtio/vhost.c | 49 ++++++++++++++-- hw/virtio/virtio-pci.c | 17 ++++++ hw/virtio/virtio.c | 57 +++++++++++++------ include/hw/virtio/vhost-backend.h | 4 ++ include/hw/virtio/vhost.h | 6 ++ include/hw/virtio/virtio-pci.h | 1 + include/hw/virtio/virtio.h | 4 ++ include/net/vhost_net.h | 5 ++ .../standard-headers/linux/virtio_config.h | 5 ++ include/standard-headers/linux/virtio_pci.h | 4 ++ 13 files changed, 274 insertions(+), 29 deletions(-)
Signature · 1 line
2.32.0
PATCH 1/16 virtio-pci: virtio_pci_common_cfg add queue_notify_data
From: Xuan Zhuo <[email protected]>
Add queue_notify_data in struct virtio_pci_common_cfg, which comes from here https://github.com/oasis-tcs/virtio-spec/issues/89
Since I want to add queue_reset after queue_notify_data, I submitted this patch first.
--- include/standard-headers/linux/virtio_pci.h | 2 ++ 1 file changed, 2 insertions(+)
include/standard-headers/linux/virtio_pci.h +2 -0
@@ -164,6 +164,7 @@ struct virtio_pci_common_cfg {uint32_t queue_avail_hi;/* read-write */ uint32_t queue_used_lo;/* read-write */ uint32_t queue_used_hi;/* read-write */ +uint16_t queue_notify_data;/* read-write */ };/* Fields in VIRTIO_PCI_CAP_PCI_CFG: */
@@ -202,6 +203,7 @@ struct virtio_pci_cfg_cap {#define VIRTIO_PCI_COMMON_Q_AVAILHI 44#define VIRTIO_PCI_COMMON_Q_USEDLO 48#define VIRTIO_PCI_COMMON_Q_USEDHI 52 +#define VIRTIO_PCI_COMMON_Q_NOTIFY_DATA 56#endif /* VIRTIO_PCI_NO_MODERN */
Signature · 1 line
2.32.0
Re: [PATCH 01/16] virtio-pci: virtio_pci_common_cfg add queue_notify_data
Quoted 9 lines · Kangjie Xu
From: Xuan Zhuo <[email protected]>
Add
queue_notify_datain structvirtio_pci_common_cfg, which comes from here https://github.com/oasis-tcs/virtio-spec/issues/89Since I want to add
queue_resetafterqueue_notify_data, I submitted this patch first.Signed-off-by Xuan Zhuo <[email protected]>
This should be done by script/update-linux-headers.sh.
Thanks
Quoted 24 lines
--- include/standard-headers/linux/virtio_pci.h | 2 ++ 1 file changed, 2 insertions(+)include/standard-headers/linux/virtio_pci.h +2
-0@@ -164,6 +164,7 @@ struct virtio_pci_common_cfg {uint32_t queue_avail_hi;/* read-write */ uint32_t queue_used_lo;/* read-write */ uint32_t queue_used_hi;/* read-write */ +uint16_t queue_notify_data;/* read-write */ };/* Fields in VIRTIO_PCI_CAP_PCI_CFG: */ @@ -202,6 +203,7 @@ struct virtio_pci_cfg_cap {#define VIRTIO_PCI_COMMON_Q_AVAILHI 44#define VIRTIO_PCI_COMMON_Q_USEDLO 48#define VIRTIO_PCI_COMMON_Q_USEDHI 52 +#define VIRTIO_PCI_COMMON_Q_NOTIFY_DATA 56#endif /* VIRTIO_PCI_NO_MODERN */
Re: [PATCH 01/16] virtio-pci: virtio_pci_common_cfg add queue_notify_data
Quoted 10 lines · Jason Wang
Quoted 8 lines · Kangjie Xu
From: Xuan Zhuo <[email protected]>
Add
queue_notify_datain structvirtio_pci_common_cfg, which comes from here https://github.com/oasis-tcs/virtio-spec/issues/89Since I want to add
queue_resetafterqueue_notify_data, I submitted this patch first.
Quoted 7 lines
Quoted 1 line
Signed-off-by Xuan Zhuo <[email protected]>This should be done by
script/update-linux-headers.sh.Thanks
Will fix.
Thanks
Quoted 22 lines
Quoted 22 lines
---
include/standard-headers/linux/virtio_pci.h| 2 ++1 file changed, 2 insertions(+)include/standard-headers/linux/virtio_pci.h +0
-0@@ -164,6 +164,7 @@ struct virtio_pci_common_cfg {
uint32_tqueue_avail_hi; /* read-write */uint32_tqueue_used_lo; /* read-write */uint32_tqueue_used_hi; /* read-write */ +uint16_tqueue_notify_data; /* read-write */ };
/* Fields inVIRTIO_PCI_CAP_PCI_CFG: */include/standard-headers/linux/virtio_pci.h +1
-0@@ -202,6 +203,7 @@ struct virtio_pci_cfg_cap { #define VIRTIO_PCI_COMMON_Q_AVAILHI 44 #define VIRTIO_PCI_COMMON_Q_USEDLO 48 #define VIRTIO_PCI_COMMON_Q_USEDHI 52 +#define VIRTIO_PCI_COMMON_Q_NOTIFY_DATA 56 #endif /* VIRTIO_PCI_NO_MODERN */
PATCH 2/16 virtio: add VIRTIO_F_RING_RESET
In reply to Kangjie XuFrom: Xuan Zhuo <[email protected]>
Added VIRTIO_F_RING_RESET, it came from here
https://github.com/oasis-tcs/virtio-spec/issues/124
This feature indicates that the driver can reset a queue individually.
--- include/standard-headers/linux/virtio_config.h | 5 +++++ 1 file changed, 5 insertions(+)
include/standard-headers/linux/virtio_config.h +5 -0
@@ -96,4 +96,9 @@* Does the device support Single Root I/O Virtualization? */ #define VIRTIO_F_SR_IOV 37 + +/* +* This feature indicates that the driver can reset a queue individually. +*/ +#define VIRTIO_F_RING_RESET 40#endif /* _LINUX_VIRTIO_CONFIG_H */
Signature · 1 line
2.32.0
PATCH 3/16 virtio: pci: virtio_pci_common_cfg add queue_reset
In reply to Kangjie XuFrom: Xuan Zhuo <[email protected]>
Add queue_reset in virtio_pci_common_cfg.
--- include/standard-headers/linux/virtio_pci.h | 2 ++ 1 file changed, 2 insertions(+)
include/standard-headers/linux/virtio_pci.h +2 -0
@@ -165,6 +165,7 @@ struct virtio_pci_common_cfg {uint32_t queue_used_lo;/* read-write */ uint32_t queue_used_hi;/* read-write */ uint16_t queue_notify_data;/* read-write */ +uint16_t queue_reset;/* read-write */ };/* Fields in VIRTIO_PCI_CAP_PCI_CFG: */
@@ -204,6 +205,7 @@ struct virtio_pci_cfg_cap {#define VIRTIO_PCI_COMMON_Q_USEDLO 48#define VIRTIO_PCI_COMMON_Q_USEDHI 52#define VIRTIO_PCI_COMMON_Q_NOTIFY_DATA 56 +#define VIRTIO_PCI_COMMON_Q_RESET 58#endif /* VIRTIO_PCI_NO_MODERN */
Signature · 1 line
2.32.0
PATCH 4/16 virtio: introduce __virtio_queue_reset()
In reply to Kangjie XuFrom: Xuan Zhuo <[email protected]>
Separate the logic of vq reset. This logic will be called directly later.
--- hw/virtio/virtio.c | 37 +++++++++++++++++++++---------------- 1 file changed, 21 insertions(+), 16 deletions(-)
hw/virtio/virtio.c +21 -16
@@ -2019,6 +2019,26 @@ static enum virtio_device_endian virtio_current_cpu_endian(void) } } +static void __virtio_queue_reset (VirtIODevice *vdev,uint32_t i) +{ + vdev->vq[i].vring.desc =0 ; + vdev->vq[i].vring.avail =0 ; + vdev->vq[i].vring.used =0 ; + vdev->vq[i].last_avail_idx =0 ; + vdev->vq[i].shadow_avail_idx =0 ; + vdev->vq[i].used_idx =0 ; + vdev->vq[i].last_avail_wrap_counter = true; + vdev->vq[i].shadow_avail_wrap_counter = true; + vdev->vq[i].used_wrap_counter = true; +virtio_queue_set_vector (vdev, i,VIRTIO_NO_VECTOR ); + vdev->vq[i].signalled_used =0 ; + vdev->vq[i].signalled_used_valid = false; + vdev->vq[i].notification = true; + vdev->vq[i].vring.num = vdev->vq[i].vring.num_default; + vdev->vq[i].inuse =0 ; +virtio_virtqueue_reset_region_cache (&vdev->vq[i]); +} +void virtio_reset (void *opaque) {VirtIODevice *vdev = opaque;
@@ -2050,22 +2070,7 @@ void virtio_reset(void *opaque)virtio_notify_vector (vdev, vdev->config_vector);for (i =0 ; i <VIRTIO_QUEUE_MAX ; i++) { - vdev->vq[i].vring.desc =0 ; - vdev->vq[i].vring.avail =0 ; - vdev->vq[i].vring.used =0 ; - vdev->vq[i].last_avail_idx =0 ; - vdev->vq[i].shadow_avail_idx =0 ; - vdev->vq[i].used_idx =0 ; - vdev->vq[i].last_avail_wrap_counter = true; - vdev->vq[i].shadow_avail_wrap_counter = true; - vdev->vq[i].used_wrap_counter = true; -virtio_queue_set_vector (vdev, i,VIRTIO_NO_VECTOR ); - vdev->vq[i].signalled_used =0 ; - vdev->vq[i].signalled_used_valid = false; - vdev->vq[i].notification = true; - vdev->vq[i].vring.num = vdev->vq[i].vring.num_default; - vdev->vq[i].inuse =0 ; -virtio_virtqueue_reset_region_cache (&vdev->vq[i]); +__virtio_queue_reset (vdev, i); } }
Signature · 1 line
2.32.0
Re: [PATCH 04/16] virtio: introduce __virtio_queue_reset()
Quoted 6 lines · Kangjie Xu
From: Xuan Zhuo <[email protected]>
Separate the logic of vq reset. This logic will be called directly later.
Signed-off-by Xuan Zhuo <[email protected]>
Quoted 59 lines
--- hw/virtio/virtio.c | 37 +++++++++++++++++++++---------------- 1 file changed, 21 insertions(+), 16 deletions(-)hw/virtio/virtio.c +21
-16@@ -2019,6 +2019,26 @@ static enum virtio_device_endian virtio_current_cpu_endian(void) } } +static void __virtio_queue_reset (VirtIODevice *vdev,uint32_t i) +{ + vdev->vq[i].vring.desc =0 ; + vdev->vq[i].vring.avail =0 ; + vdev->vq[i].vring.used =0 ; + vdev->vq[i].last_avail_idx =0 ; + vdev->vq[i].shadow_avail_idx =0 ; + vdev->vq[i].used_idx =0 ; + vdev->vq[i].last_avail_wrap_counter = true; + vdev->vq[i].shadow_avail_wrap_counter = true; + vdev->vq[i].used_wrap_counter = true; +virtio_queue_set_vector (vdev, i,VIRTIO_NO_VECTOR ); + vdev->vq[i].signalled_used =0 ; + vdev->vq[i].signalled_used_valid = false; + vdev->vq[i].notification = true; + vdev->vq[i].vring.num = vdev->vq[i].vring.num_default; + vdev->vq[i].inuse =0 ; +virtio_virtqueue_reset_region_cache (&vdev->vq[i]); +} +void virtio_reset (void *opaque) {VirtIODevice *vdev = opaque;@@ -2050,22 +2070,7 @@ void virtio_reset(void *opaque)virtio_notify_vector (vdev, vdev->config_vector);for (i =0 ; i <VIRTIO_QUEUE_MAX ; i++) { - vdev->vq[i].vring.desc =0 ; - vdev->vq[i].vring.avail =0 ; - vdev->vq[i].vring.used =0 ; - vdev->vq[i].last_avail_idx =0 ; - vdev->vq[i].shadow_avail_idx =0 ; - vdev->vq[i].used_idx =0 ; - vdev->vq[i].last_avail_wrap_counter = true; - vdev->vq[i].shadow_avail_wrap_counter = true; - vdev->vq[i].used_wrap_counter = true; -virtio_queue_set_vector (vdev, i,VIRTIO_NO_VECTOR ); - vdev->vq[i].signalled_used =0 ; - vdev->vq[i].signalled_used_valid = false; - vdev->vq[i].notification = true; - vdev->vq[i].vring.num = vdev->vq[i].vring.num_default; - vdev->vq[i].inuse =0 ; -virtio_virtqueue_reset_region_cache (&vdev->vq[i]); +__virtio_queue_reset (vdev, i); } }
PATCH 5/16 virtio: introduce virtio_queue_reset()
In reply to Kangjie XuFrom: Xuan Zhuo <[email protected]>
Introduce a new interface function virtio_queue_reset() to implement reset for vq.
Add a new callback to VirtioDeviceClass for queue reset operation for each child device.
--- hw/virtio/virtio.c | 11 +++++++++++ include/hw/virtio/virtio.h | 2 ++ 2 files changed, 13 insertions(+)
hw/virtio/virtio.c +11 -0
@@ -2039,6 +2039,17 @@ static void __virtio_queue_reset(VirtIODevice *vdev, uint32_t i)virtio_virtqueue_reset_region_cache (&vdev->vq[i]); } +void virtio_queue_reset (VirtIODevice *vdev,uint32_t queue_index) +{ +VirtioDeviceClass *k =VIRTIO_DEVICE_GET_CLASS (vdev); + +if (k->queue_reset) { + k->queue_reset (vdev, queue_index); + } + +__virtio_queue_reset (vdev, queue_index); +} +void virtio_reset (void *opaque) {VirtIODevice *vdev = opaque;
include/hw/virtio/virtio.h +2 -0
@@ -130,6 +130,7 @@ struct VirtioDeviceClass {void (*set_config)(VirtIODevice *vdev,const uint8_t *config);void (*reset)(VirtIODevice *vdev);void (*set_status)(VirtIODevice *vdev,uint8_t val); +void (*queue_reset)(VirtIODevice *vdev,uint32_t queue_index);/* For transitional devices, this is a bitmap of features * that are only exposed on the legacy interface but not * the modern one.
@@ -268,6 +269,7 @@ int virtio_queue_set_host_notifier_mr(VirtIODevice *vdev, int n, MemoryRegion *mr, bool assign);int virtio_set_status (VirtIODevice *vdev,uint8_t val);void virtio_reset (void *opaque); +void virtio_queue_reset (VirtIODevice *vdev,uint32_t queue_index);void virtio_update_irq (VirtIODevice *vdev);int virtio_set_features (VirtIODevice *vdev,uint64_t val);
Signature · 1 line
2.32.0
Re: [PATCH 05/16] virtio: introduce virtio_queue_reset()
Quoted 9 lines · Kangjie Xu
From: Xuan Zhuo <[email protected]>
Introduce a new interface function
virtio_queue_reset()to implement reset for vq.Add a new callback to VirtioDeviceClass for queue reset operation for each child device.
Signed-off-by Xuan Zhuo <[email protected]>
Quoted 47 lines
--- hw/virtio/virtio.c | 11 +++++++++++ include/hw/virtio/virtio.h | 2 ++ 2 files changed, 13 insertions(+)hw/virtio/virtio.c +11
-0@@ -2039,6 +2039,17 @@ static void __virtio_queue_reset(VirtIODevice *vdev, uint32_t i)virtio_virtqueue_reset_region_cache (&vdev->vq[i]); } +void virtio_queue_reset (VirtIODevice *vdev,uint32_t queue_index) +{ +VirtioDeviceClass *k =VIRTIO_DEVICE_GET_CLASS (vdev); + +if (k->queue_reset) { + k->queue_reset (vdev, queue_index); + } + +__virtio_queue_reset (vdev, queue_index); +} +void virtio_reset (void *opaque) {VirtIODevice *vdev = opaque;include/hw/virtio/virtio.h +2
-0@@ -130,6 +130,7 @@ struct VirtioDeviceClass {void (*set_config)(VirtIODevice *vdev,const uint8_t *config);void (*reset)(VirtIODevice *vdev);void (*set_status)(VirtIODevice *vdev,uint8_t val); +void (*queue_reset)(VirtIODevice *vdev,uint32_t queue_index);/* For transitional devices, this is a bitmap of features * that are only exposed on the legacy interface but not * the modern one. @@ -268,6 +269,7 @@ int virtio_queue_set_host_notifier_mr(VirtIODevice *vdev, int n, MemoryRegion *mr, bool assign);int virtio_set_status (VirtIODevice *vdev,uint8_t val);void virtio_reset (void *opaque); +void virtio_queue_reset (VirtIODevice *vdev,uint32_t queue_index);void virtio_update_irq (VirtIODevice *vdev);int virtio_set_features (VirtIODevice *vdev,uint64_t val);
PATCH 6/16 virtio-pci: support queue reset
In reply to Kangjie XuFrom: Xuan Zhuo <[email protected]>
PCI devices support vq reset.
Based on this function, the driver can adjust the size of the ring, and quickly recycle the buffer in the ring.
--- hw/virtio/virtio-pci.c | 16 ++++++++++++++++ include/hw/virtio/virtio-pci.h | 1 + 2 files changed, 17 insertions(+)
hw/virtio/virtio-pci.c +16 -0
@@ -1246,6 +1246,9 @@ static uint64_t virtio_pci_common_read(void *opaque, hwaddr addr,case VIRTIO_PCI_COMMON_Q_USEDHI : val = proxy->vqs[vdev->queue_sel].used[1 ];break ; +case VIRTIO_PCI_COMMON_Q_RESET : + val = proxy->vqs[vdev->queue_sel].reset; +break ;default : val =0 ; }
@@ -1333,6 +1336,7 @@ static void virtio_pci_common_write(void *opaque, hwaddr addr, ((uint64_t )proxy->vqs[vdev->queue_sel].used[1 ]) <<32 | proxy->vqs[vdev->queue_sel].used[0]); proxy->vqs[vdev->queue_sel].enabled =1 ; + proxy->vqs[vdev->queue_sel].reset =0 ; }else {virtio_error (vdev,"wrong value for queue_enable %" PRIx64, val); }
@@ -1355,6 +1359,17 @@ static void virtio_pci_common_write(void *opaque, hwaddr addr,case VIRTIO_PCI_COMMON_Q_USEDHI : proxy->vqs[vdev->queue_sel].used[1 ] = val;break ; +case VIRTIO_PCI_COMMON_Q_RESET : +if (val ==1 ) { + proxy->vqs[vdev->queue_sel].reset =1 ; + +virtio_queue_reset (vdev, vdev->queue_sel); + +/* mark reset complete */ + proxy->vqs[vdev->queue_sel].reset =0 ; + proxy->vqs[vdev->queue_sel].enabled =0 ; + } +break ;default :break ; }
@@ -1950,6 +1965,7 @@ static void virtio_pci_reset(DeviceState *qdev)for (i =0 ; i <VIRTIO_QUEUE_MAX ; i++) { proxy->vqs[i].enabled =0 ; + proxy->vqs[i].reset =0 ; proxy->vqs[i].num =0 ; proxy->vqs[i].desc[0 ] = proxy->vqs[i].desc[1 ] =0 ; proxy->vqs[i].avail[0 ] = proxy->vqs[i].avail[1 ] =0 ;
include/hw/virtio/virtio-pci.h +1 -0
@@ -117,6 +117,7 @@ typedef struct VirtIOPCIRegion { typedef struct VirtIOPCIQueue { uint16_t num; bool enabled; + bool reset; uint32_t desc[2]; uint32_t avail[2]; uint32_t used[2];
Signature · 1 line
2.32.0
Re: [PATCH 06/16] virtio-pci: support queue reset
Quoted 36 lines · Kangjie Xu
From: Xuan Zhuo <[email protected]>
PCI devices support vq reset.
Based on this function, the driver can adjust the size of the ring, and quickly recycle the buffer in the ring.
Signed-off-by Xuan Zhuo <[email protected]>--- hw/virtio/virtio-pci.c | 16 ++++++++++++++++ include/hw/virtio/virtio-pci.h | 1 + 2 files changed, 17 insertions(+)hw/virtio/virtio-pci.c +4
-0@@ -1246,6 +1246,9 @@ static uint64_t virtio_pci_common_read(void *opaque, hwaddr addr,case VIRTIO_PCI_COMMON_Q_USEDHI : val = proxy->vqs[vdev->queue_sel].used[1 ];break ; +case VIRTIO_PCI_COMMON_Q_RESET : + val = proxy->vqs[vdev->queue_sel].reset; +break ;default : val =0 ; }@@ -1333,6 +1336,7 @@ static void virtio_pci_common_write(void *opaque, hwaddr addr, ((uint64_t )proxy->vqs[vdev->queue_sel].used[1 ]) <<32 | proxy->vqs[vdev->queue_sel].used[0]); proxy->vqs[vdev->queue_sel].enabled =1 ; + proxy->vqs[vdev->queue_sel].reset =0 ; }else {virtio_error (vdev,"wrong value for queue_enable %" PRIx64, val); }
hw/virtio/virtio-pci.c static void virtio_pci_common_write(void *opaque, hwaddr addr, · line 1368
Quoted 10 lines
hw/virtio/virtio-pci.c +7
-0@@ -1355,6 +1359,17 @@ static void virtio_pci_common_write(void *opaque, hwaddr addr,case VIRTIO_PCI_COMMON_Q_USEDHI : proxy->vqs[vdev->queue_sel].used[1 ] = val;break ; +case VIRTIO_PCI_COMMON_Q_RESET : +if (val ==1 ) { + proxy->vqs[vdev->queue_sel].reset =1 ; + +virtio_queue_reset (vdev, vdev->queue_sel); + +/* mark reset complete */
The code explain itself, so the comment could be removed.
Quoted 20 lines
hw/virtio/virtio-pci.c +5
-0+ proxy->vqs[vdev->queue_sel].reset =0 ; + proxy->vqs[vdev->queue_sel].enabled =0 ; + } +break ;default :break ; }@@ -1950,6 +1965,7 @@ static void virtio_pci_reset(DeviceState *qdev)for (i =0 ; i <VIRTIO_QUEUE_MAX ; i++) { proxy->vqs[i].enabled =0 ; + proxy->vqs[i].reset =0 ; proxy->vqs[i].num =0 ; proxy->vqs[i].desc[0 ] = proxy->vqs[i].desc[1 ] =0 ; proxy->vqs[i].avail[0 ] = proxy->vqs[i].avail[1 ] =0 ;include/hw/virtio/virtio-pci.h +0
-0
include/hw/virtio/virtio-pci.h typedef struct VirtIOPCIRegion { · line 120
Quoted 4 lines
include/hw/virtio/virtio-pci.h +1
-0@@ -117,6 +117,7 @@ typedef struct VirtIOPCIRegion { typedef struct VirtIOPCIQueue { uint16_t num; bool enabled; + bool reset;
How is this migrated?
Thanks
Quoted 3 lines
include/hw/virtio/virtio-pci.h +0
-0uint32_t desc[2 ];uint32_t avail[2 ];uint32_t used[2 ];
PATCH 7/16 virtio-net: support queue reset
In reply to Kangjie XuFrom: Xuan Zhuo <[email protected]>
virtio-net implements queue reset. Queued packets in the corresponding queue pair are flushed or purged.
Queue reset is currently only implemented for non-vhosts.
--- hw/net/virtio-net.c | 15 +++++++++++++++ 1 file changed, 15 insertions(+)
hw/net/virtio-net.c +15 -0
@@ -531,6 +531,19 @@ static RxFilterInfo *virtio_net_query_rxfilter(NetClientState *nc)return info; } +static void virtio_net_queue_reset (VirtIODevice *vdev,uint32_t queue_index) +{ +VirtIONet *n =VIRTIO_NET (vdev); +NetClientState *nc =qemu_get_subqueue (n->nic,vq2q (queue_index)); + +if (!nc->peer) { +return ; + } + +qemu_flush_or_purge_queued_packets (nc->peer, true); +assert (!virtio_net_get_subqueue (nc)->async_tx.elem); +} +static void virtio_net_reset (VirtIODevice *vdev) {VirtIONet *n =VIRTIO_NET (vdev);
@@ -741,6 +754,7 @@ static uint64_t virtio_net_get_features(VirtIODevice *vdev, uint64_t features, }if (!get_vhost_net (nc->peer)) { +virtio_add_feature (&features,VIRTIO_F_RING_RESET );return features; }
@@ -3766,6 +3780,7 @@ static void virtio_net_class_init(ObjectClass *klass, void *data) vdc->set_features = virtio_net_set_features; vdc->bad_features = virtio_net_bad_features; vdc->reset = virtio_net_reset; + vdc->queue_reset = virtio_net_queue_reset; vdc->set_status = virtio_net_set_status; vdc->guest_notifier_mask = virtio_net_guest_notifier_mask; vdc->guest_notifier_pending = virtio_net_guest_notifier_pending;
Signature · 1 line
2.32.0
Re: [PATCH 07/16] virtio-net: support queue reset
Quoted 21 lines · Kangjie Xu
From: Xuan Zhuo <[email protected]>
virtio-net implements queue reset. Queued packets in the corresponding queue pair are flushed or purged.
Queue reset is currently only implemented for non-vhosts.
Signed-off-by Xuan Zhuo <[email protected]>--- hw/net/virtio-net.c | 15 +++++++++++++++ 1 file changed, 15 insertions(+)hw/net/virtio-net.c +1
-0@@ -531,6 +531,19 @@ static RxFilterInfo *virtio_net_query_rxfilter(NetClientState *nc)return info; } +static void virtio_net_queue_reset (VirtIODevice *vdev,uint32_t queue_index)
hw/net/virtio-net.c static RxFilterInfo *virtio_net_query_rxfilter(NetClientState *nc) · line 544
Quoted 10 lines
hw/net/virtio-net.c +10
-0+{ +VirtIONet *n =VIRTIO_NET (vdev); +NetClientState *nc =qemu_get_subqueue (n->nic,vq2q (queue_index)); + +if (!nc->peer) { +return ; + } + +qemu_flush_or_purge_queued_packets (nc->peer, true); +assert (!virtio_net_get_subqueue (nc)->async_tx.elem);
Let's try to reuse this function in virtio_net_reset().
hw/net/virtio-net.c static uint64_t virtio_net_get_features(VirtIODevice *vdev, uint64_t features, · line 757
Quoted 10 lines
hw/net/virtio-net.c +3
-0+} +static void virtio_net_reset (VirtIODevice *vdev) {VirtIONet *n =VIRTIO_NET (vdev);@@ -741,6 +754,7 @@ static uint64_t virtio_net_get_features(VirtIODevice *vdev, uint64_t features, }if (!get_vhost_net (nc->peer)) { +virtio_add_feature (&features,VIRTIO_F_RING_RESET );
This breaks migration compatibility.
We probably need:
1) a new command line parameter
2) make it disabled for pre-7.2 machine
Thanks
Quoted 11 lines
hw/net/virtio-net.c +1
-0return features; }@@ -3766,6 +3780,7 @@ static void virtio_net_class_init(ObjectClass *klass, void *data) vdc->set_features = virtio_net_set_features; vdc->bad_features = virtio_net_bad_features; vdc->reset = virtio_net_reset; + vdc->queue_reset = virtio_net_queue_reset; vdc->set_status = virtio_net_set_status; vdc->guest_notifier_mask = virtio_net_guest_notifier_mask; vdc->guest_notifier_pending = virtio_net_guest_notifier_pending;
Re: [PATCH 07/16] virtio-net: support queue reset
Quoted 28 lines · Jason Wang
Quoted 26 lines · Kangjie Xu
From: Xuan Zhuo <[email protected]>
virtio-net implements queue reset. Queued packets in the corresponding queue pair are flushed or purged.
Queue reset is currently only implemented for non-vhosts.
Signed-off-by Xuan Zhuo <[email protected]>---
hw/net/virtio-net.c| 15 +++++++++++++++1 file changed, 15 insertions(+)hw/net/virtio-net.c +5
-0@@ -531,6 +531,19 @@ static RxFilterInfo *virtio_net_query_rxfilter(NetClientState *nc) return info; } +static void virtio_net_queue_reset(VirtIODevice *vdev, uint32_t queue_index) +{ + VirtIONet *n = VIRTIO_NET(vdev); + NetClientState *nc = qemu_get_subqueue(n->nic, vq2q(queue_index)); + + if (!nc->peer) {
Quoted 9 lines
hw/net/virtio-net.c static RxFilterInfo *virtio_net_query_rxfilter(NetClientState *nc) · line 544
Quoted 5 lines
hw/net/virtio-net.c +5
-0+ return; + } + + qemu_flush_or_purge_queued_packets(nc->peer, true); + assert(!virtio_net_get_subqueue(nc)->async_tx.elem);Let's try to reuse this function in
virtio_net_reset().
Yeah, I'll fix it.
Thanks.
Quoted 15 lines
hw/net/virtio-net.c static uint64_t · line 757
Quoted 11 lines
hw/net/virtio-net.c +2
-0+} + static void virtio_net_reset(VirtIODevice *vdev) {VirtIONet *n =
VIRTIO_NET(vdev);hw/net/virtio-net.c +1
-0@@ -741,6 +754,7 @@ static uint64_t virtio_net_get_features(VirtIODevice *vdev, uint64_t features, } if (!get_vhost_net(nc->peer)) { + virtio_add_feature(&features, VIRTIO_F_RING_RESET);This breaks migration compatibility.
Quoted 8 lines
We probably need:
1) a new command line parameter
2) make it disabled for pre-7.2 machineThanks
Sorry, I don't get what is the meaning of "pre-7.2 machine". Could you please explain it?
Thanks
Quoted 10 lines
Quoted 10 lines
hw/net/virtio-net.c +0
-0return features; } @@ -3766,6 +3780,7 @@ static void virtio_net_class_init(ObjectClass *klass, void *data)
vdc->set_features=virtio_net_set_features;vdc->bad_features=virtio_net_bad_features;vdc->reset=virtio_net_reset;
+vdc->queue_reset=virtio_net_queue_reset;vdc->set_status=virtio_net_set_status;vdc->guest_notifier_mask=virtio_net_guest_notifier_mask;vdc->guest_notifier_pending=virtio_net_guest_notifier_pending;
Re: [PATCH 07/16] virtio-net: support queue reset
Quoted 62 lines · Kangjie Xu
Quoted 29 lines · Jason Wang
Quoted 27 lines · Kangjie Xu
From: Xuan Zhuo <[email protected]>
virtio-net implements queue reset. Queued packets in the corresponding queue pair are flushed or purged.
Queue reset is currently only implemented for non-vhosts.
Signed-off-by Xuan Zhuo <[email protected]>--- hw/net/virtio-net.c | 15 +++++++++++++++ 1 file changed, 15 insertions(+)hw/net/virtio-net.c +5
-0@@ -531,6 +531,19 @@ static RxFilterInfo *virtio_net_query_rxfilter(NetClientState *nc)return info; } +static void virtio_net_queue_reset(VirtIODevice *vdev, uint32_t queue_index) +{ +VirtIONet *n =VIRTIO_NET (vdev); +NetClientState *nc =qemu_get_subqueue (n->nic,vq2q (queue_index)); + +if (!nc->peer) {Quoted 9 lines
hw/net/virtio-net.c static RxFilterInfo *virtio_net_query_rxfilter(NetClientState *nc) · line 544
Quoted 5 lines
hw/net/virtio-net.c +5
-0+return ; + } + +qemu_flush_or_purge_queued_packets (nc->peer, true); +assert (!virtio_net_get_subqueue (nc)->async_tx.elem);Let's try to reuse this function in
virtio_net_reset().Yeah, I'll fix it.
Thanks.
Quoted 17 lines
hw/net/virtio-net.c static uint64_t · line 757
Quoted 11 lines
hw/net/virtio-net.c +3
-0+} +static void virtio_net_reset (VirtIODevice *vdev) {VirtIONet *n =VIRTIO_NET (vdev);@@ -741,6 +754,7 @@ static uint64_t virtio_net_get_features(VirtIODevice *vdev, uint64_t features, } if (!get_vhost_net(nc->peer)) { + virtio_add_feature(&features, VIRTIO_F_RING_RESET);This breaks migration compatibility.
We probably need:
Quoted 8 lines
Quoted 6 lines
1) a new command line parameter
2) make it disabled for pre-7.2 machineThanks
Sorry, I don't get what is the meaning of "pre-7.2 machine". Could you please explain it?
I meant for pre 7.2 machine type, we should make reset fault off by default.
Otherwise we break migration compatibility.
Thanks
Quoted 15 lines
Thanks
Quoted 11 lines
Quoted 11 lines
hw/net/virtio-net.c +0
-0return features; } @@ -3766,6 +3780,7 @@ static void virtio_net_class_init(ObjectClass*klass, void *data)
vdc->set_features = virtio_net_set_features; vdc->bad_features = virtio_net_bad_features; vdc->reset = virtio_net_reset; + vdc->queue_reset = virtio_net_queue_reset; vdc->set_status = virtio_net_set_status; vdc->guest_notifier_mask = virtio_net_guest_notifier_mask; vdc->guest_notifier_pending = virtio_net_guest_notifier_pending;
Re: [PATCH 07/16] virtio-net: support queue reset
Quoted 64 lines · Jason Wang
Quoted 63 lines · Kangjie Xu
Quoted 27 lines · Jason Wang
Quoted 26 lines · Kangjie Xu
From: Xuan Zhuo <[email protected]>
virtio-net implements queue reset. Queued packets in the corresponding queue pair are flushed or purged.
Queue reset is currently only implemented for non-vhosts.
Signed-off-by Xuan Zhuo <[email protected]>--- hw/net/virtio-net.c | 15 +++++++++++++++ 1 file changed, 15 insertions(+)hw/net/virtio-net.c +4
-0@@ -531,6 +531,19 @@ static RxFilterInfo *virtio_net_query_rxfilter(NetClientState *nc)return info; } +static void virtio_net_queue_reset(VirtIODevice *vdev, uint32_t queue_index) +{ +VirtIONet *n =VIRTIO_NET (vdev); +NetClientState *nc =qemu_get_subqueue (n->nic,vq2q (queue_index)); +Quoted 9 lines
hw/net/virtio-net.c static RxFilterInfo *virtio_net_query_rxfilter(NetClientState *nc) · line 544
Quoted 6 lines
hw/net/virtio-net.c +6
-0+if (!nc->peer) { +return ; + } + +qemu_flush_or_purge_queued_packets (nc->peer, true); +assert (!virtio_net_get_subqueue (nc)->async_tx.elem);Let's try to reuse this function in
virtio_net_reset().Yeah, I'll fix it.
Thanks.
Quoted 21 lines
hw/net/virtio-net.c static uint64_t · line 757
Quoted 10 lines
hw/net/virtio-net.c +3
-0+} +static void virtio_net_reset (VirtIODevice *vdev) {VirtIONet *n =VIRTIO_NET (vdev);@@ -741,6 +754,7 @@ static uint64_t virtio_net_get_features(VirtIODevice *vdev, uint64_t features, } if (!get_vhost_net(nc->peer)) { + virtio_add_feature(&features, VIRTIO_F_RING_RESET);This breaks migration compatibility.
We probably need:
1) a new command line parameter
2) make it disabled for pre-7.2 machineThanks
Quoted 7 lines
Quoted 2 lines
Sorry, I don't get what is the meaning of "pre-7.2 machine". Could you please explain it?
I meant for pre 7.2 machine type, we should make reset fault off by default.
Otherwise we break migration compatibility.
Thanks
Sorry, I did not express myself clearly. Is "7.2" the version of a system or a module? If it is a system version, what is the system?
I did not have backgrond knowledge related to this part and will investigate migration issues afterwards.
Thanks.
Quoted 13 lines
Quoted 13 lines
Thanks
Quoted 11 lines
Quoted 11 lines
hw/net/virtio-net.c +0
-0return features; } @@ -3766,6 +3780,7 @@ static void virtio_net_class_init(ObjectClass*klass, void *data)
vdc->set_features = virtio_net_set_features; vdc->bad_features = virtio_net_bad_features; vdc->reset = virtio_net_reset; + vdc->queue_reset = virtio_net_queue_reset; vdc->set_status = virtio_net_set_status; vdc->guest_notifier_mask = virtio_net_guest_notifier_mask; vdc->guest_notifier_pending = virtio_net_guest_notifier_pending;
Re: [PATCH 07/16] virtio-net: support queue reset
Quoted 69 lines · Kangjie Xu
Quoted 66 lines · Jason Wang
Quoted 65 lines · Kangjie Xu
Quoted 27 lines · Jason Wang
Quoted 26 lines · Kangjie Xu
From: Xuan Zhuo <[email protected]>
virtio-net implements queue reset. Queued packets in the corresponding queue pair are flushed or purged.
Queue reset is currently only implemented for non-vhosts.
Signed-off-by Xuan Zhuo <[email protected]>--- hw/net/virtio-net.c | 15 +++++++++++++++ 1 file changed, 15 insertions(+)hw/net/virtio-net.c +4
-0@@ -531,6 +531,19 @@ static RxFilterInfo *virtio_net_query_rxfilter(NetClientState *nc)return info; } +static void virtio_net_queue_reset(VirtIODevice *vdev, uint32_t queue_index) +{ +VirtIONet *n =VIRTIO_NET (vdev); +NetClientState *nc =qemu_get_subqueue (n->nic,vq2q (queue_index)); +Quoted 9 lines
hw/net/virtio-net.c static RxFilterInfo *virtio_net_query_rxfilter(NetClientState *nc) · line 544
Quoted 6 lines
hw/net/virtio-net.c +6
-0+if (!nc->peer) { +return ; + } + +qemu_flush_or_purge_queued_packets (nc->peer, true); +assert (!virtio_net_get_subqueue (nc)->async_tx.elem);Let's try to reuse this function in
virtio_net_reset().Yeah, I'll fix it.
Thanks.
Quoted 13 lines
hw/net/virtio-net.c static uint64_t · line 757
Quoted 10 lines
hw/net/virtio-net.c +3
-0+} +static void virtio_net_reset (VirtIODevice *vdev) {VirtIONet *n =VIRTIO_NET (vdev);@@ -741,6 +754,7 @@ static uint64_t virtio_net_get_features(VirtIODevice *vdev, uint64_t features, } if (!get_vhost_net(nc->peer)) { + virtio_add_feature(&features, VIRTIO_F_RING_RESET);This breaks migration compatibility.
Quoted 8 lines
We probably need:
1) a new command line parameter
2) make it disabled for pre-7.2 machineThanks
Sorry, I don't get what is the meaning of "pre-7.2 machine". Could you please explain it?
Quoted 8 lines
Quoted 5 lines
I meant for pre 7.2 machine type, we should make reset fault off by default.
Otherwise we break migration compatibility.
Thanks
Sorry, I did not express myself clearly. Is "7.2" the version of a system or a module? If it is a system version, what is the system?
It's the machine type to make sure the migration can work. (you can get the list of those types via qemu -machine ?)
E.g you can start a 7.0 machine on Qemu 7.1 and it guarantees to be migrated to 7.0 machine on Qemu 7.0.
Quoted 3 lines
I did not have backgrond knowledge related to this part and will investigate migration issues afterwards.
Git grep hw_compat_7_0 may give you more hints.
Thanks
Quoted 17 lines
Thanks.
Quoted 13 lines
Quoted 13 lines
Thanks
Quoted 11 lines
Quoted 11 lines
hw/net/virtio-net.c +0
-0return features; } @@ -3766,6 +3780,7 @@ static void virtio_net_class_init(ObjectClass*klass, void *data)
vdc->set_features = virtio_net_set_features; vdc->bad_features = virtio_net_bad_features; vdc->reset = virtio_net_reset; + vdc->queue_reset = virtio_net_queue_reset; vdc->set_status = virtio_net_set_status; vdc->guest_notifier_mask = virtio_net_guest_notifier_mask; vdc->guest_notifier_pending = virtio_net_guest_notifier_pending;
Re: [PATCH 07/16] virtio-net: support queue reset
Quoted 76 lines · Jason Wang
Quoted 65 lines · Kangjie Xu
Quoted 63 lines · Jason Wang
Quoted 62 lines · Kangjie Xu
Quoted 26 lines · Jason Wang
Quoted 25 lines · Kangjie Xu
From: Xuan Zhuo <[email protected]>
virtio-net implements queue reset. Queued packets in the corresponding queue pair are flushed or purged.
Queue reset is currently only implemented for non-vhosts.
Signed-off-by Xuan Zhuo <[email protected]>--- hw/net/virtio-net.c | 15 +++++++++++++++ 1 file changed, 15 insertions(+)hw/net/virtio-net.c +3
-0@@ -531,6 +531,19 @@ static RxFilterInfo *virtio_net_query_rxfilter(NetClientState *nc)return info; } +static void virtio_net_queue_reset(VirtIODevice *vdev, uint32_t queue_index) +{ +VirtIONet *n =VIRTIO_NET (vdev); +NetClientState *nc =qemu_get_subqueue (n->nic,vq2q (queue_index));Quoted 9 lines
hw/net/virtio-net.c static RxFilterInfo *virtio_net_query_rxfilter(NetClientState *nc) · line 544
Quoted 7 lines
hw/net/virtio-net.c +7
-0+ +if (!nc->peer) { +return ; + } + +qemu_flush_or_purge_queued_packets (nc->peer, true); +assert (!virtio_net_get_subqueue (nc)->async_tx.elem);Let's try to reuse this function in
virtio_net_reset().Yeah, I'll fix it.
Thanks.
Quoted 12 lines
hw/net/virtio-net.c static uint64_t · line 757
Quoted 10 lines
hw/net/virtio-net.c +3
-0+} +static void virtio_net_reset (VirtIODevice *vdev) {VirtIONet *n =VIRTIO_NET (vdev);@@ -741,6 +754,7 @@ static uint64_t virtio_net_get_features(VirtIODevice *vdev, uint64_t features, } if (!get_vhost_net(nc->peer)) { + virtio_add_feature(&features, VIRTIO_F_RING_RESET);This breaks migration compatibility.
Quoted 8 lines
We probably need:
1) a new command line parameter
2) make it disabled for pre-7.2 machineThanks
Sorry, I don't get what is the meaning of "pre-7.2 machine". Could you please explain it?
Quoted 7 lines
Quoted 5 lines
I meant for pre 7.2 machine type, we should make reset fault off by default.
Otherwise we break migration compatibility.
Thanks
Sorry, I did not express myself clearly. Is "7.2" the version of a system or a module? If it is a system version, what is the system?
It's the machine type to make sure the migration can work. (you can get the list of those types via qemu -machine ?)
Quoted 8 lines
E.g you can start a 7.0 machine on Qemu 7.1 and it guarantees to be migrated to 7.0 machine on Qemu 7.0.
Quoted 2 lines
I did not have backgrond knowledge related to this part and will investigate migration issues afterwards.
Git grep
hw_compat_7_0may give you more hints.Thanks
Okay, I get it.
Thanks for your guidelines.
Quoted 16 lines
Quoted 16 lines
Thanks.
Quoted 13 lines
Quoted 13 lines
Thanks
Quoted 11 lines
Quoted 11 lines
hw/net/virtio-net.c +0
-0return features; } @@ -3766,6 +3780,7 @@ static void virtio_net_class_init(ObjectClass*klass, void *data)
vdc->set_features = virtio_net_set_features; vdc->bad_features = virtio_net_bad_features; vdc->reset = virtio_net_reset; + vdc->queue_reset = virtio_net_queue_reset; vdc->set_status = virtio_net_set_status; vdc->guest_notifier_mask = virtio_net_guest_notifier_mask; vdc->guest_notifier_pending = virtio_net_guest_notifier_pending;
PATCH 8/16 vhost: add op to enable or disable a single vring
In reply to Kangjie XuThe interface to set enable status for a single vring is lacked in VhostOps, since the vhost_set_vring_enable_op will manipulate all virtqueues in a device.
Resetting a single vq will rely on this interface. It requires a reply to indicate that the reset operation is finished, so the parameter, wait_for_reply, is added.
--- include/hw/virtio/vhost-backend.h | 4 ++++ 1 file changed, 4 insertions(+)
include/hw/virtio/vhost-backend.h +4 -0
@@ -81,6 +81,9 @@ typedef int (*vhost_set_backend_cap_op)(struct vhost_dev *dev);typedef int (*vhost_set_owner_op )(struct vhost_dev *dev);typedef int (*vhost_reset_device_op )(struct vhost_dev *dev);typedef int (*vhost_get_vq_index_op )(struct vhost_dev *dev,int idx); +typedef int (*vhost_set_single_vring_enable_op )(struct vhost_dev *dev, +int index,int enable, +bool wait_for_reply);typedef int (*vhost_set_vring_enable_op )(struct vhost_dev *dev,int enable);typedef bool (*vhost_requires_shm_log_op )(struct vhost_dev *dev);
@@ -155,6 +158,7 @@ typedef struct VhostOps {vhost_set_owner_op vhost_set_owner;vhost_reset_device_op vhost_reset_device;vhost_get_vq_index_op vhost_get_vq_index; +vhost_set_single_vring_enable_op vhost_set_single_vring_enable;vhost_set_vring_enable_op vhost_set_vring_enable;vhost_requires_shm_log_op vhost_requires_shm_log;vhost_migration_done_op vhost_migration_done;
Signature · 1 line
2.32.0
Re: [PATCH 08/16] vhost: add op to enable or disable a single vring
Quoted 7 lines · Kangjie Xu
The interface to set enable status for a single vring is lacked in VhostOps, since the
vhost_set_vring_enable_opwill manipulate all virtqueues in a device.Resetting a single vq will rely on this interface. It requires a reply to indicate that the reset operation is finished, so the parameter,
wait_for_reply, is added.
The wait reply seems to be a implementation specific thing. Can we hide it?
Thanks
Quoted 29 lines
Signed-off-by Kangjie Xu <[email protected]>Signed-off-by Xuan Zhuo <[email protected]>--- include/hw/virtio/vhost-backend.h | 4 ++++ 1 file changed, 4 insertions(+)include/hw/virtio/vhost-backend.h +4
-0@@ -81,6 +81,9 @@ typedef int (*vhost_set_backend_cap_op)(struct vhost_dev *dev);typedef int (*vhost_set_owner_op )(struct vhost_dev *dev);typedef int (*vhost_reset_device_op )(struct vhost_dev *dev);typedef int (*vhost_get_vq_index_op )(struct vhost_dev *dev,int idx); +typedef int (*vhost_set_single_vring_enable_op )(struct vhost_dev *dev, +int index,int enable, +bool wait_for_reply);typedef int (*vhost_set_vring_enable_op )(struct vhost_dev *dev,int enable);typedef bool (*vhost_requires_shm_log_op )(struct vhost_dev *dev);@@ -155,6 +158,7 @@ typedef struct VhostOps {vhost_set_owner_op vhost_set_owner;vhost_reset_device_op vhost_reset_device;vhost_get_vq_index_op vhost_get_vq_index; +vhost_set_single_vring_enable_op vhost_set_single_vring_enable;vhost_set_vring_enable_op vhost_set_vring_enable;vhost_requires_shm_log_op vhost_requires_shm_log;vhost_migration_done_op vhost_migration_done;
Re: [PATCH 08/16] vhost: add op to enable or disable a single vring
Quoted 6 lines · Jason Wang
Quoted 4 lines · Kangjie Xu
The interface to set enable status for a single vring is lacked in VhostOps, since the
vhost_set_vring_enable_opwill manipulate all virtqueues in a device.
Quoted 9 lines
Quoted 3 lines
Resetting a single vq will rely on this interface. It requires a reply to indicate that the reset operation is finished, so the parameter,
wait_for_reply, is added.The wait reply seems to be a implementation specific thing. Can we hide it?
Thanks
I do not hide wait_for_reply here because when stopping the queue, a reply is needed to ensure that the message has been processed and queue has been disabled.
When restarting the queue, we do not need a reply, which is the same as what qemu does in vhost_dev_start().
So I add this parameter to distinguish the two cases.
I think one alternative implementation is to add a interface in VhostOps: queue_reset(). In this way details can be hidden. What do you think of this solution? Does it look better?
Thanks
Quoted 31 lines
Quoted 31 lines
Signed-off-by Kangjie Xu <[email protected]>Signed-off-by Xuan Zhuo <[email protected]>---
include/hw/virtio/vhost-backend.h| 4 ++++1 file changed, 4 insertions(+)include/hw/virtio/vhost-backend.h +3
-0@@ -81,6 +81,9 @@ typedef int (*vhost_set_backend_cap_op)(struct vhost_dev *dev); typedef int (*vhost_set_owner_op)(struct vhost_dev *dev); typedef int (*vhost_reset_device_op)(struct vhost_dev *dev); typedef int (*vhost_get_vq_index_op)(struct vhost_dev *dev, int idx); +typedef int (*vhost_set_single_vring_enable_op )(struct vhost_dev *dev, + int index, int enable, + bool wait_for_reply); typedef int (*vhost_set_vring_enable_op)(struct vhost_dev *dev, int enable);typedef bool (*vhost_requires_shm_log_op)(struct
vhost_dev*dev);include/hw/virtio/vhost-backend.h +1
-0@@ -155,6 +158,7 @@ typedef struct VhostOps { vhost_set_owner_op vhost_set_owner; vhost_reset_device_op vhost_reset_device; vhost_get_vq_index_op vhost_get_vq_index; + vhost_set_single_vring_enable_op vhost_set_single_vring_enable; vhost_set_vring_enable_op vhost_set_vring_enable; vhost_requires_shm_log_op vhost_requires_shm_log; vhost_migration_done_op vhost_migration_done;
Re: [PATCH 08/16] vhost: add op to enable or disable a single vring
Quoted 14 lines · Kangjie Xu
Quoted 11 lines · Jason Wang
Quoted 7 lines · Kangjie Xu
The interface to set enable status for a single vring is lacked in VhostOps, since the
vhost_set_vring_enable_opwill manipulate all virtqueues in a device.Resetting a single vq will rely on this interface. It requires a reply to indicate that the reset operation is finished, so the parameter,
wait_for_reply, is added.
Quoted 8 lines
Quoted 5 lines
The wait reply seems to be a implementation specific thing. Can we hide it?
Thanks
I do not hide
wait_for_replyhere because when stopping the queue, a reply is needed to ensure that the message has been processed and queue has been disabled.
This needs to be done at vhost-backend level instead of the general vhost code.
E.g vhost-kernel or vDPA is using ioctl() which is synchronous.
Quoted 9 lines
When restarting the queue, we do not need a reply, which is the same as what qemu does in
vhost_dev_start().So I add this parameter to distinguish the two cases.
I think one alternative implementation is to add a interface in
VhostOps:queue_reset(). In this way details can be hidden. What do you think of this solution? Does it look better?
Let me ask it differently, under which case can we call this function with wait_for_reply = false?
Thanks
Quoted 36 lines
Thanks
Quoted 32 lines
Quoted 32 lines
Signed-off-by Kangjie Xu <[email protected]>Signed-off-by Xuan Zhuo <[email protected]>--- include/hw/virtio/vhost-backend.h | 4 ++++ 1 file changed, 4 insertions(+)include/hw/virtio/vhost-backend.h +3
-0@@ -81,6 +81,9 @@ typedef int (*vhost_set_backend_cap_op)(struct vhost_dev *dev);typedef int (*vhost_set_owner_op )(struct vhost_dev *dev);typedef int (*vhost_reset_device_op )(struct vhost_dev *dev);typedef int (*vhost_get_vq_index_op )(struct vhost_dev *dev,int idx); +typedef int (*vhost_set_single_vring_enable_op )(struct vhost_dev *dev, +int index,int enable, +bool wait_for_reply);typedef int (*vhost_set_vring_enable_op )(struct vhost_dev *dev,int enable);typedef bool (*vhost_requires_shm_log_op)(struct vhost_dev *dev);include/hw/virtio/vhost-backend.h +1
-0@@ -155,6 +158,7 @@ typedef struct VhostOps {vhost_set_owner_op vhost_set_owner;vhost_reset_device_op vhost_reset_device;vhost_get_vq_index_op vhost_get_vq_index; +vhost_set_single_vring_enable_op vhost_set_single_vring_enable;vhost_set_vring_enable_op vhost_set_vring_enable;vhost_requires_shm_log_op vhost_requires_shm_log;vhost_migration_done_op vhost_migration_done;
Re: [PATCH 08/16] vhost: add op to enable or disable a single vring
Quoted 15 lines · Jason Wang
Quoted 14 lines · Kangjie Xu
Quoted 12 lines · Jason Wang
Quoted 7 lines · Kangjie Xu
The interface to set enable status for a single vring is lacked in VhostOps, since the
vhost_set_vring_enable_opwill manipulate all virtqueues in a device.Resetting a single vq will rely on this interface. It requires a reply to indicate that the reset operation is finished, so the parameter,
wait_for_reply, is added.The wait reply seems to be a implementation specific thing. Can we hide it?
Quoted 8 lines
Quoted 5 lines
Quoted 2 lines
Thanks
I do not hide
wait_for_replyhere because when stopping the queue, a reply is needed to ensure that the message has been processed and queue has been disabled.This needs to be done at vhost-backend level instead of the general vhost code.
E.g vhost-kernel or vDPA is using
ioctl()which is synchronous.
Yeah, I understand your concerns, will fix it in the next version.
Quoted 5 lines
Quoted 5 lines
When restarting the queue, we do not need a reply, which is the same as what qemu does in
vhost_dev_start().So I add this parameter to distinguish the two cases.
Quoted 7 lines
Quoted 3 lines
I think one alternative implementation is to add a interface in
VhostOps:queue_reset(). In this way details can be hidden. What do you think of this solution? Does it look better?Let me ask it differently, under which case can we call this function with
wait_for_reply= false?Thanks
Cases when we do not need to wait until that the reply has been processed. Actually in dev_start(), or dev_stop(), they do not wait for replies when enabling/disabling vqs.
Specifically, vhost_set_vring_enable() will call it with wait_for_reply = false.
In our vq reset scenario, it should be synchronous because the driver need to modify queues after the device stop using the vq in the backend. Undefined errors will occur If the device is still using the queue when the driver is making modifications to it,
Back to our implementation, we will not expose this parameter in the next version.
Thanks.
Quoted 32 lines
Quoted 32 lines
Thanks
Quoted 30 lines
Quoted 30 lines
Signed-off-by Kangjie Xu <[email protected]>Signed-off-by Xuan Zhuo <[email protected]>--- include/hw/virtio/vhost-backend.h | 4 ++++ 1 file changed, 4 insertions(+)include/hw/virtio/vhost-backend.h +3
-0@@ -81,6 +81,9 @@ typedef int (*vhost_set_backend_cap_op)(struct vhost_dev *dev);typedef int (*vhost_set_owner_op )(struct vhost_dev *dev);typedef int (*vhost_reset_device_op )(struct vhost_dev *dev);typedef int (*vhost_get_vq_index_op )(struct vhost_dev *dev,int idx); +typedef int (*vhost_set_single_vring_enable_op )(struct vhost_dev *dev, +int index,int enable, +bool wait_for_reply);typedef int (*vhost_set_vring_enable_op )(struct vhost_dev *dev,int enable);typedef bool (*vhost_requires_shm_log_op)(struct vhost_dev *dev);include/hw/virtio/vhost-backend.h +1
-0@@ -155,6 +158,7 @@ typedef struct VhostOps {vhost_set_owner_op vhost_set_owner;vhost_reset_device_op vhost_reset_device;vhost_get_vq_index_op vhost_get_vq_index; +vhost_set_single_vring_enable_op vhost_set_single_vring_enable;vhost_set_vring_enable_op vhost_set_vring_enable;vhost_requires_shm_log_op vhost_requires_shm_log;vhost_migration_done_op vhost_migration_done;
Re: [PATCH 08/16] vhost: add op to enable or disable a single vring
Quoted 35 lines · Kangjie Xu
Quoted 15 lines · Jason Wang
Quoted 14 lines · Kangjie Xu
Quoted 12 lines · Jason Wang
Quoted 7 lines · Kangjie Xu
The interface to set enable status for a single vring is lacked in VhostOps, since the
vhost_set_vring_enable_opwill manipulate all virtqueues in a device.Resetting a single vq will rely on this interface. It requires a reply to indicate that the reset operation is finished, so the parameter,
wait_for_reply, is added.The wait reply seems to be a implementation specific thing. Can we hide it?
Quoted 8 lines
Quoted 5 lines
Quoted 2 lines
Thanks
I do not hide
wait_for_replyhere because when stopping the queue, a reply is needed to ensure that the message has been processed and queue has been disabled.This needs to be done at vhost-backend level instead of the general vhost code.
E.g vhost-kernel or vDPA is using
ioctl()which is synchronous.Yeah, I understand your concerns, will fix it in the next version.
Quoted 8 lines
Quoted 8 lines
When restarting the queue, we do not need a reply, which is the same as what qemu does in
vhost_dev_start().So I add this parameter to distinguish the two cases.
I think one alternative implementation is to add a interface in
VhostOps:queue_reset(). In this way details can be hidden. What do you think of this solution? Does it look better?
Quoted 7 lines
Quoted 4 lines
Let me ask it differently, under which case can we call this function with
wait_for_reply= false?Thanks
Cases when we do not need to wait until that the reply has been processed. Actually in
dev_start(), ordev_stop(),
But we don't need to send virtqueue reset requests in those cases?
Quoted 6 lines
they do not wait for
replies when enabling/disabling vqs.Specifically,
vhost_set_vring_enable()will call it withwait_for_reply= false.
Quoted 7 lines
In our vq reset scenario, it should be synchronous because the driver need to modify queues after the device stop using the vq in the backend. Undefined errors will occur If the device is still using the queue when the driver is making modifications to it,
Back to our implementation, we will not expose this parameter in the next version.
Ok.
Thanks
Quoted 36 lines
Thanks.
Quoted 32 lines
Quoted 32 lines
Thanks
Quoted 30 lines
Quoted 30 lines
Signed-off-by Kangjie Xu <[email protected]>Signed-off-by Xuan Zhuo <[email protected]>--- include/hw/virtio/vhost-backend.h | 4 ++++ 1 file changed, 4 insertions(+)include/hw/virtio/vhost-backend.h +3
-0@@ -81,6 +81,9 @@ typedef int (*vhost_set_backend_cap_op)(struct vhost_dev *dev);typedef int (*vhost_set_owner_op )(struct vhost_dev *dev);typedef int (*vhost_reset_device_op )(struct vhost_dev *dev);typedef int (*vhost_get_vq_index_op )(struct vhost_dev *dev,int idx); +typedef int (*vhost_set_single_vring_enable_op )(struct vhost_dev *dev, +int index,int enable, +bool wait_for_reply);typedef int (*vhost_set_vring_enable_op )(struct vhost_dev *dev,int enable);typedef bool (*vhost_requires_shm_log_op)(struct vhost_dev *dev);include/hw/virtio/vhost-backend.h +1
-0@@ -155,6 +158,7 @@ typedef struct VhostOps {vhost_set_owner_op vhost_set_owner;vhost_reset_device_op vhost_reset_device;vhost_get_vq_index_op vhost_get_vq_index; +vhost_set_single_vring_enable_op vhost_set_single_vring_enable;vhost_set_vring_enable_op vhost_set_vring_enable;vhost_requires_shm_log_op vhost_requires_shm_log;vhost_migration_done_op vhost_migration_done;
Re: [PATCH 08/16] vhost: add op to enable or disable a single vring
Quoted 33 lines · Jason Wang
Quoted 32 lines · Kangjie Xu
Quoted 13 lines · Jason Wang
Quoted 12 lines · Kangjie Xu
Quoted 11 lines · Jason Wang
Quoted 7 lines · Kangjie Xu
The interface to set enable status for a single vring is lacked in VhostOps, since the
vhost_set_vring_enable_opwill manipulate all virtqueues in a device.Resetting a single vq will rely on this interface. It requires a reply to indicate that the reset operation is finished, so the parameter,
wait_for_reply, is added.The wait reply seems to be a implementation specific thing. Can we hide it?
Quoted 8 lines
Quoted 5 lines
Quoted 2 lines
Thanks
I do not hide
wait_for_replyhere because when stopping the queue, a reply is needed to ensure that the message has been processed and queue has been disabled.This needs to be done at vhost-backend level instead of the general vhost code.
E.g vhost-kernel or vDPA is using
ioctl()which is synchronous.Yeah, I understand your concerns, will fix it in the next version.
Quoted 8 lines
Quoted 8 lines
When restarting the queue, we do not need a reply, which is the same as what qemu does in
vhost_dev_start().So I add this parameter to distinguish the two cases.
I think one alternative implementation is to add a interface in
VhostOps:queue_reset(). In this way details can be hidden. What do you think of this solution? Does it look better?
Quoted 7 lines
Quoted 6 lines
Quoted 4 lines
Let me ask it differently, under which case can we call this function with
wait_for_reply= false?Thanks
Cases when we do not need to wait until that the reply has been processed. Actually in
dev_start(), ordev_stop(),But we don't need to send virtqueue reset requests in those cases?
You're right, we don't need to do this in those cases.
Thanks
Quoted 51 lines
Quoted 6 lines
they do not wait for
replies when enabling/disabling vqs.Specifically,
vhost_set_vring_enable()will call it withwait_for_reply= false.Quoted 7 lines
In our vq reset scenario, it should be synchronous because the driver need to modify queues after the device stop using the vq in the backend. Undefined errors will occur If the device is still using the queue when the driver is making modifications to it,
Back to our implementation, we will not expose this parameter in the next version.
Ok.
Thanks
Quoted 34 lines
Thanks.
Quoted 32 lines
Quoted 32 lines
Thanks
Quoted 30 lines
Quoted 30 lines
Signed-off-by Kangjie Xu <[email protected]>Signed-off-by Xuan Zhuo <[email protected]>--- include/hw/virtio/vhost-backend.h | 4 ++++ 1 file changed, 4 insertions(+)include/hw/virtio/vhost-backend.h +3
-0@@ -81,6 +81,9 @@ typedef int (*vhost_set_backend_cap_op)(struct vhost_dev *dev);typedef int (*vhost_set_owner_op )(struct vhost_dev *dev);typedef int (*vhost_reset_device_op )(struct vhost_dev *dev);typedef int (*vhost_get_vq_index_op )(struct vhost_dev *dev,int idx); +typedef int (*vhost_set_single_vring_enable_op )(struct vhost_dev *dev, +int index,int enable, +bool wait_for_reply);typedef int (*vhost_set_vring_enable_op )(struct vhost_dev *dev,int enable);typedef bool (*vhost_requires_shm_log_op)(struct vhost_dev *dev);include/hw/virtio/vhost-backend.h +1
-0@@ -155,6 +158,7 @@ typedef struct VhostOps {vhost_set_owner_op vhost_set_owner;vhost_reset_device_op vhost_reset_device;vhost_get_vq_index_op vhost_get_vq_index; +vhost_set_single_vring_enable_op vhost_set_single_vring_enable;vhost_set_vring_enable_op vhost_set_vring_enable;vhost_requires_shm_log_op vhost_requires_shm_log;vhost_migration_done_op vhost_migration_done;
PATCH 9/16 vhost-user: enable/disable a single vring
In reply to Kangjie XuImplement the vhost_set_single_vring_enable, which is to enable or disable a single vring.
The parameter wait_for_reply is added to help for some cases such as vq reset.
Meanwhile, vhost_user_set_vring_enable() is refactored.
--- hw/virtio/vhost-user.c | 55 ++++++++++++++++++++++++++++++++++++------ 1 file changed, 48 insertions(+), 7 deletions(-)
hw/virtio/vhost-user.c +48 -7
@@ -267,6 +267,8 @@ struct scrub_regions {int fd_idx; }; +static int enforce_reply (struct vhost_dev *dev,const VhostUserMsg *msg); +static bool ioeventfd_enabled (void ) {return !kvm_enabled () ||kvm_eventfds_enabled ();
@@ -1198,6 +1200,49 @@ static int vhost_user_set_vring_base(struct vhost_dev *dev,return vhost_set_vring (dev,VHOST_USER_SET_VRING_BASE , ring); } + +static int vhost_user_set_single_vring_enable (struct vhost_dev *dev, +int index, +int enable, +bool wait_for_reply) +{ +int ret; + +if (index < dev->vq_index || index >= dev->vq_index + dev->nvqs) { +return -EINVAL ; + } + +struct vhost_vring_state state = { + .index = index, + .num = enable, + }; + +VhostUserMsg msg = { + .hdr.request =VHOST_USER_SET_VRING_ENABLE , + .hdr.flags =VHOST_USER_VERSION , + .payload.state = state, + .hdr.size =sizeof (msg.payload.state), + }; + +bool reply_supported =virtio_has_feature (dev->protocol_features, +VHOST_USER_PROTOCOL_F_REPLY_ACK ); + +if (reply_supported && wait_for_reply) { + msg.hdr.flags |=VHOST_USER_NEED_REPLY_MASK ; + } + + ret =vhost_user_write (dev, &msg,NULL ,0 ); +if (ret <0 ) { +return ret; + } + +if (wait_for_reply) { +return enforce_reply (dev, &msg); + } + +return ret; +} +static int vhost_user_set_vring_enable (struct vhost_dev *dev,int enable) {int i;
@@ -1207,13 +1252,8 @@ static int vhost_user_set_vring_enable(struct vhost_dev *dev, int enable) }for (i =0 ; i < dev->nvqs; ++i) { -int ret; -struct vhost_vring_state state = { - .index = dev->vq_index + i, - .num = enable, - }; - - ret =vhost_set_vring (dev,VHOST_USER_SET_VRING_ENABLE , &state); +int ret =vhost_user_set_single_vring_enable (dev, dev->vq_index + i, + enable, false);if (ret <0 ) {/* * Restoring the previous state is likely infeasible, as well as
@@ -2627,6 +2667,7 @@ const VhostOps user_ops = { .vhost_set_owner = vhost_user_set_owner, .vhost_reset_device = vhost_user_reset_device, .vhost_get_vq_index = vhost_user_get_vq_index, + .vhost_set_single_vring_enable = vhost_user_set_single_vring_enable, .vhost_set_vring_enable = vhost_user_set_vring_enable, .vhost_requires_shm_log = vhost_user_requires_shm_log, .vhost_migration_done = vhost_user_migration_done,
Signature · 1 line
2.32.0
Re: [PATCH 09/16] vhost-user: enable/disable a single vring
Quoted 51 lines · Kangjie Xu
Implement the
vhost_set_single_vring_enable, which is to enable or disable a single vring.The parameter
wait_for_replyis added to help for some cases such as vq reset.Meanwhile,
vhost_user_set_vring_enable()is refactored.Signed-off-by Kangjie Xu <[email protected]>Signed-off-by Xuan Zhuo <[email protected]>--- hw/virtio/vhost-user.c | 55 ++++++++++++++++++++++++++++++++++++------ 1 file changed, 48 insertions(+), 7 deletions(-)hw/virtio/vhost-user.c +22
-0@@ -267,6 +267,8 @@ struct scrub_regions {int fd_idx; }; +static int enforce_reply (struct vhost_dev *dev,const VhostUserMsg *msg); +static bool ioeventfd_enabled (void ) {return !kvm_enabled () ||kvm_eventfds_enabled ();@@ -1198,6 +1200,49 @@ static int vhost_user_set_vring_base(struct vhost_dev *dev,return vhost_set_vring (dev,VHOST_USER_SET_VRING_BASE , ring); } + +static int vhost_user_set_single_vring_enable (struct vhost_dev *dev, +int index, +int enable, +bool wait_for_reply) +{ +int ret; + +if (index < dev->vq_index || index >= dev->vq_index + dev->nvqs) { +return -EINVAL ; + } + +struct vhost_vring_state state = { + .index = index, + .num = enable, + }; + +VhostUserMsg msg = { + .hdr.request =VHOST_USER_SET_VRING_ENABLE , + .hdr.flags =VHOST_USER_VERSION ,
hw/virtio/vhost-user.c static int vhost_user_set_vring_base(struct vhost_dev *dev, · line 1232
Quoted 10 lines
hw/virtio/vhost-user.c +10
-0+ .payload.state = state, + .hdr.size = sizeof(msg.payload.state), + }; + +bool reply_supported =virtio_has_feature (dev->protocol_features, +VHOST_USER_PROTOCOL_F_REPLY_ACK ); + +if (reply_supported && wait_for_reply) { + msg.hdr.flags |=VHOST_USER_NEED_REPLY_MASK ; + }
Do we need to fail if !realy_supported && wait_for_reply?
Thanks
Quoted 40 lines
hw/virtio/vhost-user.c +16
-7+ + ret =vhost_user_write (dev, &msg,NULL ,0 ); +if (ret <0 ) { +return ret; + } + +if (wait_for_reply) { +return enforce_reply (dev, &msg); + } + +return ret; +} +static int vhost_user_set_vring_enable (struct vhost_dev *dev,int enable) {int i;@@ -1207,13 +1252,8 @@ static int vhost_user_set_vring_enable(struct vhost_dev *dev, int enable) }for (i =0 ; i < dev->nvqs; ++i) { -int ret; -struct vhost_vring_state state = { - .index = dev->vq_index + i, - .num = enable, - }; - - ret =vhost_set_vring (dev,VHOST_USER_SET_VRING_ENABLE , &state); +int ret =vhost_user_set_single_vring_enable (dev, dev->vq_index + i, + enable, false);if (ret <0 ) {/* * Restoring the previous state is likely infeasible, as well as @@ -2627,6 +2667,7 @@ const VhostOps user_ops = { .vhost_set_owner = vhost_user_set_owner, .vhost_reset_device = vhost_user_reset_device, .vhost_get_vq_index = vhost_user_get_vq_index, + .vhost_set_single_vring_enable = vhost_user_set_single_vring_enable, .vhost_set_vring_enable = vhost_user_set_vring_enable, .vhost_requires_shm_log = vhost_user_requires_shm_log, .vhost_migration_done = vhost_user_migration_done,
Re: [PATCH 09/16] vhost-user: enable/disable a single vring
Quoted 66 lines · Jason Wang
Quoted 50 lines · Kangjie Xu
Implement the
vhost_set_single_vring_enable, which is to enable or disable a single vring.The parameter
wait_for_replyis added to help for some cases such as vq reset.Meanwhile,
vhost_user_set_vring_enable()is refactored.Signed-off-by Kangjie Xu <[email protected]>Signed-off-by Xuan Zhuo <[email protected]>---
hw/virtio/vhost-user.c| 55 ++++++++++++++++++++++++++++++++++++------1 file changed, 48 insertions(+), 7 deletions(-)hw/virtio/vhost-user.c +1
-0@@ -267,6 +267,8 @@ struct scrub_regions { int fd_idx; }; +static int enforce_reply(struct vhost_dev *dev, const VhostUserMsg *msg); + static bool ioeventfd_enabled(void) { return !kvm_enabled() || kvm_eventfds_enabled();@@ -1198,6 +1200,49 @@ static int vhost_user_set_vring_base(structvhost_dev *dev, return vhost_set_vring(dev, VHOST_USER_SET_VRING_BASE, ring); }+
hw/virtio/vhost-user.c +19
-0+static int vhost_user_set_single_vring_enable (struct vhost_dev *dev, + int index, + int enable, + bool wait_for_reply) +{ + int ret; + + if (index < dev->vq_index || index >= dev->vq_index + dev->nvqs) { + return -EINVAL; + } + + struct vhost_vring_state state = { + .index = index, + .num = enable, + }; + + VhostUserMsg msg = { + .hdr.request = VHOST_USER_SET_VRING_ENABLE, + .hdr.flags = VHOST_USER_VERSION,hw/virtio/vhost-user.c static int vhost_user_set_vring_base(struct vhost_dev *dev, · line 1232
Quoted 10 lines
hw/virtio/vhost-user.c +10
-0+ .payload.state = state, + .hdr.size = sizeof(msg.payload.state), + }; + + bool reply_supported = virtio_has_feature(dev->protocol_features, +VHOST_USER_PROTOCOL_F_REPLY_ACK ); + + if (reply_supported && wait_for_reply) { + msg.hdr.flags |= VHOST_USER_NEED_REPLY_MASK; + }Do we need to fail if !realy_supported &&
wait_for_reply?
Quoted 3 lines
Thanks
I guess you mean "should we fail if VHOST_USER_PROTOCOL_F_REPLY_ACK feature is not supported?".
The implementation here is similar to that in vhost_user_set_vring_addr().
If this feature is not supported, it will call enforce_reply(), then call vhost_user_get_features() to get a reply.
Since the messages will be processed sequentailly in DPDK, success of enforce_reply() means the previous message VHOST_USER_SET_VRING_ENABLE has been processed.
Thanks
Quoted 44 lines
Quoted 44 lines
hw/virtio/vhost-user.c +13
-0+ + ret = vhost_user_write(dev, &msg, NULL, 0); + if (ret < 0) { + return ret; + } + + if (wait_for_reply) { + return enforce_reply(dev, &msg); + } + + return ret; +} +static int
vhost_user_set_vring_enable(struct vhost_dev *dev, int enable){
int i;hw/virtio/vhost-user.c +2
-7@@ -1207,13 +1252,8 @@ static int vhost_user_set_vring_enable(struct vhost_dev *dev, int enable) } for (i = 0; i < dev->nvqs; ++i) { - int ret; - struct vhost_vring_state state = { - .index = dev->vq_index + i, - .num = enable, - }; - - ret = vhost_set_vring(dev, VHOST_USER_SET_VRING_ENABLE, &state); + int ret = vhost_user_set_single_vring_enable(dev, dev->vq_index + i, + enable, false); if (ret < 0) {/*
* Restoring the previous state is likely infeasible, as well ashw/virtio/vhost-user.c +1
-0@@ -2627,6 +2667,7 @@ const VhostOps user_ops = { .vhost_set_owner = vhost_user_set_owner, .vhost_reset_device = vhost_user_reset_device, .vhost_get_vq_index = vhost_user_get_vq_index, + .vhost_set_single_vring_enable = vhost_user_set_single_vring_enable, .vhost_set_vring_enable = vhost_user_set_vring_enable, .vhost_requires_shm_log = vhost_user_requires_shm_log,.vhost_migration_done =
vhost_user_migration_done,
Re: [PATCH 09/16] vhost-user: enable/disable a single vring
Quoted 73 lines · Kangjie Xu
Quoted 67 lines · Jason Wang
Quoted 51 lines · Kangjie Xu
Implement the
vhost_set_single_vring_enable, which is to enable or disable a single vring.The parameter
wait_for_replyis added to help for some cases such as vq reset.Meanwhile,
vhost_user_set_vring_enable()is refactored.Signed-off-by Kangjie Xu <[email protected]>Signed-off-by Xuan Zhuo <[email protected]>--- hw/virtio/vhost-user.c | 55 ++++++++++++++++++++++++++++++++++++------ 1 file changed, 48 insertions(+), 7 deletions(-)hw/virtio/vhost-user.c +20
-0@@ -267,6 +267,8 @@ struct scrub_regions {int fd_idx; }; +static int enforce_reply(struct vhost_dev *dev, const VhostUserMsg *msg); +static bool ioeventfd_enabled (void ) {return !kvm_enabled () ||kvm_eventfds_enabled ();@@ -1198,6 +1200,49 @@ static int vhost_user_set_vring_base(structvhost_dev *dev, return vhost_set_vring(dev, VHOST_USER_SET_VRING_BASE, ring); } + +static int vhost_user_set_single_vring_enable(struct vhost_dev *dev, +int index, +int enable, +bool wait_for_reply) +{ +int ret; + +if (index < dev->vq_index || index >= dev->vq_index + dev->nvqs) { +return -EINVAL ; + } + +struct vhost_vring_state state = { + .index = index, + .num = enable, + }; + +VhostUserMsg msg = { + .hdr.request =VHOST_USER_SET_VRING_ENABLE , + .hdr.flags =VHOST_USER_VERSION ,hw/virtio/vhost-user.c static int vhost_user_set_vring_base(struct · line 1232
Quoted 10 lines
hw/virtio/vhost-user.c +10
-0+ .payload.state = state, + .hdr.size = sizeof(msg.payload.state), + }; + +bool reply_supported =virtio_has_feature (dev->protocol_features, +VHOST_USER_PROTOCOL_F_REPLY_ACK ); + +if (reply_supported && wait_for_reply) { + msg.hdr.flags |=VHOST_USER_NEED_REPLY_MASK ; + }Do we need to fail if !realy_supported &&
wait_for_reply?Quoted 3 lines
Thanks
Quoted 7 lines
I guess you mean "should we fail if
VHOST_USER_PROTOCOL_F_REPLY_ACKfeature is not supported?".The implementation here is similar to that in
vhost_user_set_vring_addr().If this feature is not supported, it will call
enforce_reply(), then callvhost_user_get_features()to get a reply.
Ok, so you meant we can then fallback to VHOST_USER_GET_FEATURES? I wonder how robust is this, e.g is the bakcend required to process vhost-user request in order?
Thanks
Quoted 54 lines
Since the messages will be processed sequentailly in DPDK, success of
enforce_reply()means the previous messageVHOST_USER_SET_VRING_ENABLEhas been processed.Thanks
Quoted 46 lines
Quoted 46 lines
hw/virtio/vhost-user.c +15
-7+ + ret =vhost_user_write (dev, &msg,NULL ,0 ); +if (ret <0 ) { +return ret; + } + +if (wait_for_reply) { +return enforce_reply (dev, &msg); + } + +return ret; +} +static int vhost_user_set_vring_enable (struct vhost_dev *dev,int enable) {int i;@@ -1207,13 +1252,8 @@ static int vhost_user_set_vring_enable(struct vhost_dev *dev, int enable) }for (i =0 ; i < dev->nvqs; ++i) { -int ret; -struct vhost_vring_state state = { - .index = dev->vq_index + i, - .num = enable, - }; - - ret =vhost_set_vring (dev,VHOST_USER_SET_VRING_ENABLE , &state); +int ret =vhost_user_set_single_vring_enable (dev, dev->vq_index + i, + enable, false);if (ret <0 ) {/*
* Restoring the previous state is likely infeasible,
as well ashw/virtio/vhost-user.c +1
-0@@ -2627,6 +2667,7 @@ const VhostOps user_ops = { .vhost_set_owner = vhost_user_set_owner, .vhost_reset_device = vhost_user_reset_device, .vhost_get_vq_index = vhost_user_get_vq_index, + .vhost_set_single_vring_enable = vhost_user_set_single_vring_enable, .vhost_set_vring_enable = vhost_user_set_vring_enable, .vhost_requires_shm_log = vhost_user_requires_shm_log,.vhost_migration_done =
vhost_user_migration_done,
Re: [PATCH 09/16] vhost-user: enable/disable a single vring
Quoted 76 lines · Jason Wang
Quoted 75 lines · Kangjie Xu
Quoted 65 lines · Jason Wang
Quoted 51 lines · Kangjie Xu
Implement the
vhost_set_single_vring_enable, which is to enable or disable a single vring.The parameter
wait_for_replyis added to help for some cases such as vq reset.Meanwhile,
vhost_user_set_vring_enable()is refactored.Signed-off-by Kangjie Xu <[email protected]>Signed-off-by Xuan Zhuo <[email protected]>--- hw/virtio/vhost-user.c | 55 ++++++++++++++++++++++++++++++++++++------ 1 file changed, 48 insertions(+), 7 deletions(-)hw/virtio/vhost-user.c +20
-0@@ -267,6 +267,8 @@ struct scrub_regions {int fd_idx; }; +static int enforce_reply(struct vhost_dev *dev, const VhostUserMsg *msg); +static bool ioeventfd_enabled (void ) {return !kvm_enabled () ||kvm_eventfds_enabled ();@@ -1198,6 +1200,49 @@ static int vhost_user_set_vring_base(structvhost_dev *dev, return vhost_set_vring(dev, VHOST_USER_SET_VRING_BASE, ring); } + +static int vhost_user_set_single_vring_enable(struct vhost_dev *dev, +int index, +int enable, +bool wait_for_reply) +{ +int ret; + +if (index < dev->vq_index || index >= dev->vq_index + dev->nvqs) { +return -EINVAL ; + } + +struct vhost_vring_state state = { + .index = index, + .num = enable, + }; + +VhostUserMsg msg = { + .hdr.request =VHOST_USER_SET_VRING_ENABLE , + .hdr.flags =VHOST_USER_VERSION ,hw/virtio/vhost-user.c static int vhost_user_set_vring_base(struct · line 1232
Quoted 10 lines
hw/virtio/vhost-user.c +10
-0+ .payload.state = state, + .hdr.size = sizeof(msg.payload.state), + }; + +bool reply_supported =virtio_has_feature (dev->protocol_features, +VHOST_USER_PROTOCOL_F_REPLY_ACK ); + +if (reply_supported && wait_for_reply) { + msg.hdr.flags |=VHOST_USER_NEED_REPLY_MASK ; + }Do we need to fail if !realy_supported &&
wait_for_reply?Quoted 3 lines
Thanks
I guess you mean "should we fail if
VHOST_USER_PROTOCOL_F_REPLY_ACKfeature is not supported?".The implementation here is similar to that in
vhost_user_set_vring_addr().
Quoted 7 lines
Quoted 2 lines
If this feature is not supported, it will call
enforce_reply(), then callvhost_user_get_features()to get a reply.Ok, so you meant we can then fallback to
VHOST_USER_GET_FEATURES? I wonder how robust is this, e.g is the bakcend required to process vhost-user request in order?Thanks
Yes, we rely on VHOST_USER_GET_FEATURES message to ensure that VHOST_USER_SET_VRING_ENABLE has been processed.
It's not robust. I reviewed the vhost-user protocol in qemu doc, actually it does not specify that the backend should process them in order.
I think adding a new vhost protocol message can fix this issue. The new invented message should reset the queue, and have a blocked read to ensure the message has been processed.
Thanks
Quoted 51 lines
Quoted 51 lines
Since the messages will be processed sequentailly in DPDK, success of
enforce_reply()means the previous messageVHOST_USER_SET_VRING_ENABLEhas been processed.Thanks
Quoted 45 lines
Quoted 45 lines
hw/virtio/vhost-user.c +15
-7+ + ret =vhost_user_write (dev, &msg,NULL ,0 ); +if (ret <0 ) { +return ret; + } + +if (wait_for_reply) { +return enforce_reply (dev, &msg); + } + +return ret; +} +static int vhost_user_set_vring_enable (struct vhost_dev *dev,int enable) {int i;@@ -1207,13 +1252,8 @@ static int vhost_user_set_vring_enable(struct vhost_dev *dev, int enable) }for (i =0 ; i < dev->nvqs; ++i) { -int ret; -struct vhost_vring_state state = { - .index = dev->vq_index + i, - .num = enable, - }; - - ret =vhost_set_vring (dev,VHOST_USER_SET_VRING_ENABLE , &state); +int ret =vhost_user_set_single_vring_enable (dev, dev->vq_index + i, + enable, false);if (ret <0 ) {/*
* Restoring the previous state is likely infeasible,
as well ashw/virtio/vhost-user.c +1
-0@@ -2627,6 +2667,7 @@ const VhostOps user_ops = { .vhost_set_owner = vhost_user_set_owner, .vhost_reset_device = vhost_user_reset_device, .vhost_get_vq_index = vhost_user_get_vq_index, + .vhost_set_single_vring_enable = vhost_user_set_single_vring_enable, .vhost_set_vring_enable = vhost_user_set_vring_enable, .vhost_requires_shm_log = vhost_user_requires_shm_log,.vhost_migration_done =
vhost_user_migration_done,
PATCH 10/16 vhost: extract the logic of unmapping the vrings and desc
In reply to Kangjie XuIntroduce vhost_virtqueue_unmap() to ummap the vrings and desc of a virtqueue.
The function will be used later.
--- hw/virtio/vhost.c | 20 ++++++++++++++------ 1 file changed, 14 insertions(+), 6 deletions(-)
hw/virtio/vhost.c +14 -6
@@ -1197,6 +1197,19 @@ fail_alloc_desc:return r; } +static void vhost_virtqueue_unmap (struct vhost_dev *dev, +struct VirtIODevice *vdev, +struct vhost_virtqueue *vq, +unsigned idx) +{ +vhost_memory_unmap (dev, vq->used,virtio_queue_get_used_size (vdev, idx), +1 ,virtio_queue_get_used_size (vdev, idx)); +vhost_memory_unmap (dev, vq->avail,virtio_queue_get_avail_size (vdev, idx), +0 ,virtio_queue_get_avail_size (vdev, idx)); +vhost_memory_unmap (dev, vq->desc,virtio_queue_get_desc_size (vdev, idx), +0 ,virtio_queue_get_desc_size (vdev, idx)); +} + static void vhost_virtqueue_stop(struct vhost_dev *dev, struct VirtIODevice *vdev, struct vhost_virtqueue *vq,
@@ -1235,12 +1248,7 @@ static void vhost_virtqueue_stop(struct vhost_dev *dev, vhost_vq_index); } -vhost_memory_unmap (dev, vq->used,virtio_queue_get_used_size (vdev, idx), -1 ,virtio_queue_get_used_size (vdev, idx)); -vhost_memory_unmap (dev, vq->avail,virtio_queue_get_avail_size (vdev, idx), -0 ,virtio_queue_get_avail_size (vdev, idx)); -vhost_memory_unmap (dev, vq->desc,virtio_queue_get_desc_size (vdev, idx), -0 ,virtio_queue_get_desc_size (vdev, idx)); +vhost_virtqueue_unmap (dev, vdev, vq, idx); } static void vhost_eventfd_add(MemoryListener *listener,
Signature · 1 line
2.32.0
Re: [PATCH 10/16] vhost: extract the logic of unmapping the vrings and desc
Quoted 7 lines · Kangjie Xu
Introduce
vhost_virtqueue_unmap()to ummap the vrings and desc of a virtqueue.The function will be used later.
Signed-off-by Kangjie Xu <[email protected]>Signed-off-by Xuan Zhuo <[email protected]>
Quoted 42 lines
--- hw/virtio/vhost.c | 20 ++++++++++++++------ 1 file changed, 14 insertions(+), 6 deletions(-)hw/virtio/vhost.c +14
-6@@ -1197,6 +1197,19 @@ fail_alloc_desc:return r; } +static void vhost_virtqueue_unmap (struct vhost_dev *dev, +struct VirtIODevice *vdev, +struct vhost_virtqueue *vq, +unsigned idx) +{ +vhost_memory_unmap (dev, vq->used,virtio_queue_get_used_size (vdev, idx), +1 ,virtio_queue_get_used_size (vdev, idx)); +vhost_memory_unmap (dev, vq->avail,virtio_queue_get_avail_size (vdev, idx), +0 ,virtio_queue_get_avail_size (vdev, idx)); +vhost_memory_unmap (dev, vq->desc,virtio_queue_get_desc_size (vdev, idx), +0 ,virtio_queue_get_desc_size (vdev, idx)); +} + static void vhost_virtqueue_stop(struct vhost_dev *dev, struct VirtIODevice *vdev, struct vhost_virtqueue *vq,@@ -1235,12 +1248,7 @@ static void vhost_virtqueue_stop(struct vhost_dev *dev, vhost_vq_index); } -vhost_memory_unmap (dev, vq->used,virtio_queue_get_used_size (vdev, idx), -1 ,virtio_queue_get_used_size (vdev, idx)); -vhost_memory_unmap (dev, vq->avail,virtio_queue_get_avail_size (vdev, idx), -0 ,virtio_queue_get_avail_size (vdev, idx)); -vhost_memory_unmap (dev, vq->desc,virtio_queue_get_desc_size (vdev, idx), -0 ,virtio_queue_get_desc_size (vdev, idx)); +vhost_virtqueue_unmap (dev, vdev, vq, idx); } static void vhost_eventfd_add(MemoryListener *listener,
PATCH 11/16 vhost: introduce restart and release for vhost_dev's vqs
In reply to Kangjie XuIntroduce vhost_dev_virtqueue_restart(), which can restart the virtqueue when the vhost has already started running.
Meanwhile, vhost_dev_virtqueue_release(), which can ummap the vrings and the desc of a specific vq of a device.
Combining the two functions, we can reset a virtqueue with a started vhost.
--- hw/virtio/vhost.c | 29 +++++++++++++++++++++++++++++ include/hw/virtio/vhost.h | 6 ++++++ 2 files changed, 35 insertions(+)
hw/virtio/vhost.c +29 -0
@@ -1904,3 +1904,32 @@ int vhost_net_set_backend(struct vhost_dev *hdev,return -ENOSYS ; } + +void vhost_dev_virtqueue_release (struct vhost_dev *hdev,VirtIODevice *vdev, +int vq_index) +{ +int idx = vq_index - hdev->vq_index; + + idx = hdev->vhost_ops->vhost_get_vq_index (hdev, idx); + +vhost_virtqueue_unmap (hdev, + vdev, + hdev->vqs + idx, + hdev->vq_index + idx); +} + +int vhost_dev_virtqueue_restart (struct vhost_dev *hdev,VirtIODevice *vdev, +int vq_index) +{ +int idx = vq_index - hdev->vq_index; +int r =0 ; + + idx = hdev->vhost_ops->vhost_get_vq_index (hdev, idx); + + r =vhost_virtqueue_start (hdev, + vdev, + hdev->vqs + idx, + hdev->vq_index + idx); + +return r; +}
include/hw/virtio/vhost.h +6 -0
@@ -277,6 +277,12 @@ bool vhost_has_free_slot(void);int vhost_net_set_backend (struct vhost_dev *hdev,struct vhost_vring_file *file); + +void vhost_dev_virtqueue_release (struct vhost_dev *hdev,VirtIODevice *vdev, +int vq_index); +int vhost_dev_virtqueue_restart (struct vhost_dev *hdev,VirtIODevice *vdev, +int vq_index); +int vhost_device_iotlb_miss (struct vhost_dev *dev,uint64_t iova,int write);void vhost_dev_reset_inflight (struct vhost_inflight *inflight);
Signature · 1 line
2.32.0
Re: [PATCH 11/16] vhost: introduce restart and release for vhost_dev's vqs
Quoted 27 lines · Kangjie Xu
Introduce
vhost_dev_virtqueue_restart(), which can restart the virtqueue when the vhost has already started running.Meanwhile,
vhost_dev_virtqueue_release(), which can ummap the vrings and the desc of a specific vq of a device.Combining the two functions, we can reset a virtqueue with a started vhost.
Signed-off-by Kangjie Xu <[email protected]>Signed-off-by Xuan Zhuo <[email protected]>--- hw/virtio/vhost.c | 29 +++++++++++++++++++++++++++++ include/hw/virtio/vhost.h | 6 ++++++ 2 files changed, 35 insertions(+)hw/virtio/vhost.c +3
-0@@ -1904,3 +1904,32 @@ int vhost_net_set_backend(struct vhost_dev *hdev,return -ENOSYS ; } + +void vhost_dev_virtqueue_release (struct vhost_dev *hdev,VirtIODevice *vdev, +int vq_index)
hw/virtio/vhost.c int vhost_net_set_backend(struct vhost_dev *hdev, · line 1919
Quoted 10 lines
hw/virtio/vhost.c +10
-0+{ +int idx = vq_index - hdev->vq_index; + + idx = hdev->vhost_ops->vhost_get_vq_index (hdev, idx); + +vhost_virtqueue_unmap (hdev, + vdev, + hdev->vqs + idx, + hdev->vq_index + idx); +}
Anything wrong that makes you can't use vhost_virtqueue_stop() here?
Thanks
Quoted 33 lines
hw/virtio/vhost.c +16
-0+ +int vhost_dev_virtqueue_restart (struct vhost_dev *hdev,VirtIODevice *vdev, +int vq_index) +{ +int idx = vq_index - hdev->vq_index; +int r =0 ; + + idx = hdev->vhost_ops->vhost_get_vq_index (hdev, idx); + + r =vhost_virtqueue_start (hdev, + vdev, + hdev->vqs + idx, + hdev->vq_index + idx); + +return r; +}include/hw/virtio/vhost.h +6
-0@@ -277,6 +277,12 @@ bool vhost_has_free_slot(void);int vhost_net_set_backend (struct vhost_dev *hdev,struct vhost_vring_file *file); + +void vhost_dev_virtqueue_release (struct vhost_dev *hdev,VirtIODevice *vdev, +int vq_index); +int vhost_dev_virtqueue_restart (struct vhost_dev *hdev,VirtIODevice *vdev, +int vq_index); +int vhost_device_iotlb_miss (struct vhost_dev *dev,uint64_t iova,int write);void vhost_dev_reset_inflight (struct vhost_inflight *inflight);
Re: [PATCH 11/16] vhost: introduce restart and release for vhost_dev's vqs
Quoted 43 lines · Jason Wang
Quoted 27 lines · Kangjie Xu
Introduce
vhost_dev_virtqueue_restart(), which can restart the virtqueue when the vhost has already started running.Meanwhile,
vhost_dev_virtqueue_release(), which can ummap the vrings and the desc of a specific vq of a device.Combining the two functions, we can reset a virtqueue with a started vhost.
Signed-off-by Kangjie Xu <[email protected]>Signed-off-by Xuan Zhuo <[email protected]>---
hw/virtio/vhost.c | 29 +++++++++++++++++++++++++++++include/hw/virtio/vhost.h| 6 ++++++2 files changed, 35 insertions(+)hw/virtio/vhost.c +3
-0@@ -1904,3 +1904,32 @@ int vhost_net_set_backend(struct vhost_dev *hdev, return -ENOSYS; } + +void vhost_dev_virtqueue_release (struct vhost_dev *hdev,VirtIODevice *vdev, + int vq_index)hw/virtio/vhost.c int vhost_net_set_backend(struct vhost_dev *hdev, · line 1919
Quoted 10 lines
hw/virtio/vhost.c +10
-0+{ + int idx = vq_index - hdev->vq_index; + + idx = hdev->vhost_ops->vhost_get_vq_index(hdev, idx); + + vhost_virtqueue_unmap(hdev, + vdev, + hdev->vqs + idx, + hdev->vq_index + idx); +}Anything wrong that makes you can't use
vhost_virtqueue_stop()here?
Quoted 3 lines
Thanks
Yeah, vhost_virtqueue_stop() will result in all the queues in the device being stopped.
In the scenario of DPDK-OVS, vhost_virtqueue_stop will send VHOST_USER_GET_VRING_BASE to DPDK. This message is meant to destroy the queue.
However, In DPDK, VHOST_USER_GET_VRING_BASE message will lead to vhost_destroy_device_notify(). This function will disable all the queues in the device.
Thus, when restarting the queue, other queues except the restarted one are still disabled.
On the other hand, I think it may be not necessary to destroy the queue. We can simply disable it. When restarting the queue, it will be intialized again.
I'm sorry I forgot to cc in the last email, so I resend this email.
Thanks
Quoted 34 lines
Quoted 34 lines
hw/virtio/vhost.c +2
-0+ +int vhost_dev_virtqueue_restart(struct vhost_dev *hdev, VirtIODevice*vdev,
+ intvq_index)hw/virtio/vhost.c +13
-0+{ + int idx = vq_index - hdev->vq_index; + int r = 0; + + idx = hdev->vhost_ops->vhost_get_vq_index(hdev, idx); + + r = vhost_virtqueue_start(hdev, + vdev, + hdev->vqs + idx, + hdev->vq_index + idx); + + return r; +}include/hw/virtio/vhost.h +0
-0@@ -277,6 +277,12 @@ bool vhost_has_free_slot(void); int vhost_net_set_backend(struct vhost_dev *hdev, struct vhost_vring_file *file);+
+void vhost_dev_virtqueue_release(structvhost_dev*hdev, VirtIODevice *vdev,
+ intvq_index);
+int vhost_dev_virtqueue_restart(structvhost_dev*hdev, VirtIODevice *vdev,
+ intvq_index);
+
intvhost_device_iotlb_miss(struct vhost_dev *dev, uint64_t iova, int write); voidvhost_dev_reset_inflight(struct vhost_inflight *inflight);
A message that is not in the archive (sent off-list or not yet received): [email protected]
A message that is not in the archive (sent off-list or not yet received): CACGkMEtxXSm8Qc1LpKJJYm9cQ-F+eU5Lqecr62maRPxq1tM5rg@mail.gmail.com
Re: [PATCH 11/16] vhost: introduce restart and release for vhost_dev's vqs
Quoted 44 lines · Jason Wang
Quoted 43 lines · Kangjie Xu
Quoted 41 lines · Jason Wang
Quoted 27 lines · Kangjie Xu
Introduce
vhost_dev_virtqueue_restart(), which can restart the virtqueue when the vhost has already started running.Meanwhile,
vhost_dev_virtqueue_release(), which can ummap the vrings and the desc of a specific vq of a device.Combining the two functions, we can reset a virtqueue with a started vhost.
Signed-off-by Kangjie Xu <[email protected]>Signed-off-by Xuan Zhuo <[email protected]>--- hw/virtio/vhost.c | 29 +++++++++++++++++++++++++++++ include/hw/virtio/vhost.h | 6 ++++++ 2 files changed, 35 insertions(+)hw/virtio/vhost.c +3
-0@@ -1904,3 +1904,32 @@ int vhost_net_set_backend(struct vhost_dev *hdev,return -ENOSYS ; } + +void vhost_dev_virtqueue_release (struct vhost_dev *hdev,VirtIODevice *vdev, +int vq_index)hw/virtio/vhost.c int vhost_net_set_backend(struct vhost_dev *hdev, · line 1919
Quoted 10 lines
hw/virtio/vhost.c +10
-0+{ +int idx = vq_index - hdev->vq_index; + + idx = hdev->vhost_ops->vhost_get_vq_index (hdev, idx); + +vhost_virtqueue_unmap (hdev, + vdev, + hdev->vqs + idx, + hdev->vq_index + idx); +}Anything wrong that makes you can't use
vhost_virtqueue_stop()here?
Quoted 6 lines
Quoted 4 lines
Quoted 2 lines
Thanks
Yeah,
vhost_virtqueue_stop()will result in all the queues in the device being stopped.This sounds a defect of the backend. I think we'd better not workaround it in the qemu.
Yeah, I agree.
Quoted 11 lines
Quoted 3 lines
In the scenario of DPDK-OVS,
vhost_virtqueue_stopwill sendVHOST_USER_GET_VRING_BASEto DPDK. This message is meant to destroy the queue.That's tricky.
Quoted 3 lines
However, In DPDK,
VHOST_USER_GET_VRING_BASEmessage will lead tovhost_destroy_device_notify(). This function will disable all the queues in the device.Ok. I wonder how the reset is implemented in DPDK? It looks to me a new type of request is required.
Currently, DPDK does not support reset of queue level, but it supports device-level reset. VHOST_USER_RESET_DEVICE request exists in vhost protocol but it is not used in vhost-net module in QEMU.
It seems like there is no way for the driver to directly reset the device in vhost user net (If we use ethtool -r <eth_name>, we will get operation not supported.)
Actually we can plug the driver in and out to perform a device reset operation, but it will be applied to all the devices. I take it for example here.
If you plug out the driver(modprobe -r virtio-net), vhost_dev_stop() will be called in qemu. DPDK will receive VHOST_USER_GET_VRING_BASE messages for each virtqueue.
Then you can plug in the driver(modprobe virtio-net), vhost_dev_start() will be called in qemu. The process is same as when we boot the machine, DPDK will receive:
VHOST_USER_SET_VRING_NUM, VHOST_USER_SET_VRING_BASE, VHOST_USER_SET_VRING_ADDR, VHOST_USER_SET_VRING_KICK, VHOST_USER_SET_VRING_CALL
in order to start a virtqueue.
Our vq reset implemenation does not need any change to the DPDK except adding the feature bit, since we reuse all the request types in vhost-user protocol.
I agree that we should add a new request in vhost-user protocol to support vq reset, we'll implement it in the next version.
Quoted 11 lines
Quoted 6 lines
Thus, when restarting the queue, other queues except the restarted one are still disabled.
On the other hand, I think it may be not necessary to destroy the queue. We can simply disable it. When restarting the queue, it will be intialized again.
So for vhost-net kernel, we can simply set NULL as the backend for a specific virtqueue. We just need to refactor the stop codes (which did it at vhost device level instead of queue level currently).
Thanks
Yes, I think so.
Thanks
Quoted 37 lines
Quoted 37 lines
Thanks
Quoted 35 lines
Quoted 35 lines
hw/virtio/vhost.c +2
-0+ +int vhost_dev_virtqueue_restart(struct vhost_dev *hdev, VirtIODevice*vdev,
hw/virtio/vhost.c +14
-0+ int vq_index) +{ +int idx = vq_index - hdev->vq_index; +int r =0 ; + + idx = hdev->vhost_ops->vhost_get_vq_index (hdev, idx); + + r =vhost_virtqueue_start (hdev, + vdev, + hdev->vqs + idx, + hdev->vq_index + idx); + +return r; +}include/hw/virtio/vhost.h +5
-0@@ -277,6 +277,12 @@ bool vhost_has_free_slot(void);int vhost_net_set_backend (struct vhost_dev *hdev,struct vhost_vring_file *file); + +void vhost_dev_virtqueue_release(struct vhost_dev *hdev, VirtIODevice *vdev, + int vq_index); +int vhost_dev_virtqueue_restart (struct vhost_dev *hdev,VirtIODevice *vdev, +int vq_index); + int vhost_device_iotlb_miss(struct vhost_dev *dev, uint64_t iova,int write); void vhost_dev_reset_inflight(struct vhost_inflight *inflight);
PATCH 12/16 vhost-net: introduce restart and stop for vhost_net's vqs
In reply to Kangjie XuIntroduce vhost_virtqueue_restart(), which can restart the virtqueue when the vhost net started running before.
Introduce vhost_virtqueue_stop(), which can disable the vq and unmap vrings and the desc of the vq. When disabling the vq, the function is blocked and waits for a reply.
Combining the two functions, we can reset a virtqueue with a started vhost net.
--- hw/net/vhost_net.c | 55 +++++++++++++++++++++++++++++++++++++++++ include/net/vhost_net.h | 5 ++++ 2 files changed, 60 insertions(+)
hw/net/vhost_net.c +55 -0
@@ -514,3 +514,58 @@ int vhost_net_set_mtu(struct vhost_net *net, uint16_t mtu)return vhost_ops->vhost_net_set_mtu (&net->dev, mtu); } + +void vhost_virtqueue_stop (VirtIODevice *vdev,NetClientState *nc, +int vq_index) +{ +VHostNetState *net =get_vhost_net (nc->peer); +const VhostOps *vhost_ops = net->dev.vhost_ops; +int r; + +assert (vhost_ops); + + r = vhost_ops->vhost_set_single_vring_enable (&net->dev, vq_index,0 , true); +if (r <0 ) { + goto err_queue_disable; + } + +vhost_dev_virtqueue_release (&net->dev, vdev, vq_index); + +return ; + +err_queue_disable: +error_report ("Error when releasing the qeuue." ); +} + +int vhost_virtqueue_restart (VirtIODevice *vdev,NetClientState *nc, +int vq_index) +{ +VHostNetState *net =get_vhost_net (nc->peer); +const VhostOps *vhost_ops = net->dev.vhost_ops; +int r; + +if (!net->dev.started) { +return 0 ; + } + +assert (vhost_ops); + + r =vhost_dev_virtqueue_restart (&net->dev, vdev, vq_index); +if (r <0 ) { + goto err_start; + } + + r = vhost_ops->vhost_set_single_vring_enable (&net->dev, vq_index,1 , + false); +if (r <0 ) { + goto err_start; + } + +return 0 ; + +err_start: +error_report ("Error when restarting the queue." ); +vhost_dev_stop (&net->dev, vdev); + +return r; +}
include/net/vhost_net.h +5 -0
@@ -48,4 +48,9 @@ uint64_t vhost_net_get_acked_features(VHostNetState *net);int vhost_net_set_mtu (struct vhost_net *net,uint16_t mtu); +void vhost_virtqueue_stop (VirtIODevice *vdev,NetClientState *nc, +int vq_index); +int vhost_virtqueue_restart (VirtIODevice *vdev,NetClientState *nc, +int vq_index); +#endif
Signature · 1 line
2.32.0
Re: [PATCH 12/16] vhost-net: introduce restart and stop for vhost_net's vqs
Quoted 22 lines · Kangjie Xu
Introduce
vhost_virtqueue_restart(), which can restart the virtqueue when the vhost net started running before.Introduce
vhost_virtqueue_stop(), which can disable the vq and unmap vrings and the desc of the vq. When disabling the vq, the function is blocked and waits for a reply.Combining the two functions, we can reset a virtqueue with a started vhost net.
Signed-off-by Kangjie Xu <[email protected]>Signed-off-by Xuan Zhuo <[email protected]>--- hw/net/vhost_net.c | 55 +++++++++++++++++++++++++++++++++++++++++ include/net/vhost_net.h | 5 ++++ 2 files changed, 60 insertions(+)hw/net/vhost_net.c +0
-0
hw/net/vhost_net.c int vhost_net_set_mtu(struct vhost_net *net, uint16_t mtu) · line 519
Quoted 6 lines
hw/net/vhost_net.c +3
-0@@ -514,3 +514,58 @@ int vhost_net_set_mtu(struct vhost_net *net, uint16_t mtu)return vhost_ops->vhost_net_set_mtu (&net->dev, mtu); } + +void vhost_virtqueue_stop (VirtIODevice *vdev,NetClientState *nc, +int vq_index)
Let's rename this as vhost_net_virtqueue_stop()
Quoted 30 lines
hw/net/vhost_net.c +30
-0+{ +VHostNetState *net =get_vhost_net (nc->peer); +const VhostOps *vhost_ops = net->dev.vhost_ops; +int r; + +assert (vhost_ops); + + r = vhost_ops->vhost_set_single_vring_enable (&net->dev, vq_index,0 , true); +if (r <0 ) { + goto err_queue_disable; + } + +vhost_dev_virtqueue_release (&net->dev, vdev, vq_index); + +return ; + +err_queue_disable: +error_report ("Error when releasing the qeuue." ); +} + +int vhost_virtqueue_restart (VirtIODevice *vdev,NetClientState *nc, +int vq_index) +{ +VHostNetState *net =get_vhost_net (nc->peer); +const VhostOps *vhost_ops = net->dev.vhost_ops; +int r; + +if (!net->dev.started) { +return 0 ; + }
hw/net/vhost_net.c int vhost_net_set_mtu(struct vhost_net *net, uint16_t mtu) · line 559
Quoted 10 lines
hw/net/vhost_net.c +10
-0+ +assert (vhost_ops); + + r =vhost_dev_virtqueue_restart (&net->dev, vdev, vq_index); +if (r <0 ) { + goto err_start; + } + + r = vhost_ops->vhost_set_single_vring_enable (&net->dev, vq_index,1 , + false);
So it looks nothing vhost_net specific but vhost. And why not do set_single_vring_enable in vhost_dev_virtqueue_restart() (btw, the name of this function looks confusing).
Thanks
Quoted 26 lines
hw/net/vhost_net.c +12
-0+if (r <0 ) { + goto err_start; + } + +return 0 ; + +err_start: +error_report ("Error when restarting the queue." ); +vhost_dev_stop (&net->dev, vdev); + +return r; +}include/net/vhost_net.h +5
-0@@ -48,4 +48,9 @@ uint64_t vhost_net_get_acked_features(VHostNetState *net);int vhost_net_set_mtu (struct vhost_net *net,uint16_t mtu); +void vhost_virtqueue_stop (VirtIODevice *vdev,NetClientState *nc, +int vq_index); +int vhost_virtqueue_restart (VirtIODevice *vdev,NetClientState *nc, +int vq_index); +#endif
Re: [PATCH 12/16] vhost-net: introduce restart and stop for vhost_net's vqs
Quoted 19 lines · Jason Wang
Quoted 17 lines · Kangjie Xu
Introduce
vhost_virtqueue_restart(), which can restart the virtqueue when the vhost net started running before.Introduce
vhost_virtqueue_stop(), which can disable the vq and unmap vrings and the desc of the vq. When disabling the vq, the function is blocked and waits for a reply.Combining the two functions, we can reset a virtqueue with a started vhost net.
Signed-off-by Kangjie Xu <[email protected]>Signed-off-by Xuan Zhuo <[email protected]>---
hw/net/vhost_net.c | 55 +++++++++++++++++++++++++++++++++++++++++include/net/vhost_net.h| 5 ++++2 files changed, 60 insertions(+)
Quoted 15 lines
hw/net/vhost_net.c int vhost_net_set_mtu(struct vhost_net *net, · line 519
Quoted 11 lines
hw/net/vhost_net.c +3
-0@@ -514,3 +514,58 @@ int vhost_net_set_mtu(struct vhost_net *net, uint16_t mtu) return vhost_ops->vhost_net_set_mtu(&net->dev, mtu); } + +void vhost_virtqueue_stop (VirtIODevice *vdev,NetClientState *nc, + int vq_index)Let's rename this as
vhost_net_virtqueue_stop()
Yeah, I agree.
Thanks
Quoted 47 lines
Quoted 41 lines
hw/net/vhost_net.c +8
-0+{ + VHostNetState *net = get_vhost_net(nc->peer); + const VhostOps *vhost_ops = net->dev.vhost_ops; + int r; + + assert(vhost_ops); + + r = vhost_ops->vhost_set_single_vring_enable(&net->dev,
vq_index, 0, true);
+ if (r < 0) {
+ gotoerr_queue_disable;
+ }hw/net/vhost_net.c +28
-0+ + vhost_dev_virtqueue_release(&net->dev, vdev, vq_index); + + return; + +err_queue_disable: + error_report("Error when releasing the qeuue."); +} + +int vhost_virtqueue_restart (VirtIODevice *vdev,NetClientState *nc, + int vq_index) +{ + VHostNetState *net = get_vhost_net(nc->peer); + const VhostOps *vhost_ops = net->dev.vhost_ops; + int r; + + if (!net->dev.started) { + return 0; + } + + assert(vhost_ops); + + r = vhost_dev_virtqueue_restart(&net->dev, vdev, vq_index); + if (r < 0) { + goto err_start; + } + + r = vhost_ops->vhost_set_single_vring_enable(&net->dev,Quoted 2 lines
vq_index, 1,
+ false);So it looks nothing
vhost_netspecific but vhost. And why not doset_single_vring_enableinvhost_dev_virtqueue_restart()(btw, the name of this function looks confusing).
Quoted 3 lines
Thanks
I thought it was a convention that virtio-net will call vhost_net first, and cannot call vhost directly. So I implement it in this way.
It's no probem to move it to vhost. I'll fix it.
Thanks
Quoted 25 lines
Quoted 25 lines
+ if (r < 0) {
+ gotoerr_start;
+ }hw/net/vhost_net.c +9
-0+ + return 0; + +err_start: + error_report("Error when restarting the queue."); + vhost_dev_stop(&net->dev, vdev); + + return r; +}include/net/vhost_net.h +4
-0@@ -48,4 +48,9 @@ uint64_t vhost_net_get_acked_features(VHostNetState *net); int vhost_net_set_mtu(struct vhost_net *net, uint16_t mtu); +void vhost_virtqueue_stop(VirtIODevice *vdev, NetClientState *nc, + int vq_index); +int vhost_virtqueue_restart (VirtIODevice *vdev,NetClientState *nc, + int vq_index); + #endif
PATCH 13/16 virtio: introduce queue_enable in virtio
In reply to Kangjie XuIntroduce the interface queue_enable() in VirtioDeviceClass and the fucntion virtio_queue_enable() in virtio, it can be called when VIRTIO_PCI_COMMON_Q_ENABLE is written.
--- hw/virtio/virtio-pci.c | 1 + hw/virtio/virtio.c | 9 +++++++++ include/hw/virtio/virtio.h | 2 ++ 3 files changed, 12 insertions(+)
hw/virtio/virtio-pci.c +1 -0
@@ -1335,6 +1335,7 @@ static void virtio_pci_common_write(void *opaque, hwaddr addr, proxy->vqs[vdev->queue_sel].avail[0 ], ((uint64_t )proxy->vqs[vdev->queue_sel].used[1 ]) <<32 | proxy->vqs[vdev->queue_sel].used[0]); +virtio_queue_enable (vdev, vdev->queue_sel); proxy->vqs[vdev->queue_sel].enabled =1 ; proxy->vqs[vdev->queue_sel].reset =0 ; }else {
hw/virtio/virtio.c +9 -0
@@ -2050,6 +2050,15 @@ void virtio_queue_reset(VirtIODevice *vdev, uint32_t queue_index)__virtio_queue_reset (vdev, queue_index); } +void virtio_queue_enable (VirtIODevice *vdev,uint32_t queue_index) +{ +VirtioDeviceClass *k =VIRTIO_DEVICE_GET_CLASS (vdev); + +if (k->queue_enable) { + k->queue_enable (vdev, queue_index); + } +} +void virtio_reset (void *opaque) {VirtIODevice *vdev = opaque;
include/hw/virtio/virtio.h +2 -0
@@ -131,6 +131,7 @@ struct VirtioDeviceClass {void (*reset)(VirtIODevice *vdev);void (*set_status)(VirtIODevice *vdev,uint8_t val);void (*queue_reset)(VirtIODevice *vdev,uint32_t queue_index); +void (*queue_enable)(VirtIODevice *vdev,uint32_t queue_index);/* For transitional devices, this is a bitmap of features * that are only exposed on the legacy interface but not * the modern one.
@@ -270,6 +271,7 @@ int virtio_queue_set_host_notifier_mr(VirtIODevice *vdev, int n,int virtio_set_status (VirtIODevice *vdev,uint8_t val);void virtio_reset (void *opaque);void virtio_queue_reset (VirtIODevice *vdev,uint32_t queue_index); +void virtio_queue_enable (VirtIODevice *vdev,uint32_t queue_index);void virtio_update_irq (VirtIODevice *vdev);int virtio_set_features (VirtIODevice *vdev,uint64_t val);
Signature · 1 line
2.32.0
Re: [PATCH 13/16] virtio: introduce queue_enable in virtio
Quoted 3 lines · Kangjie Xu
Introduce the interface
queue_enable()in VirtioDeviceClass and the fucntionvirtio_queue_enable()in virtio, it can be called whenVIRTIO_PCI_COMMON_Q_ENABLEis written.
I'd suggest to split this series into two.
1) queue_enable
2) queue_reset
The patch looks good to me.
Thanks
Quoted 61 lines
Signed-off-by Kangjie Xu <[email protected]>Signed-off-by Xuan Zhuo <[email protected]>--- hw/virtio/virtio-pci.c | 1 + hw/virtio/virtio.c | 9 +++++++++ include/hw/virtio/virtio.h | 2 ++ 3 files changed, 12 insertions(+)hw/virtio/virtio-pci.c +1
-0@@ -1335,6 +1335,7 @@ static void virtio_pci_common_write(void *opaque, hwaddr addr, proxy->vqs[vdev->queue_sel].avail[0 ], ((uint64_t )proxy->vqs[vdev->queue_sel].used[1 ]) <<32 | proxy->vqs[vdev->queue_sel].used[0]); +virtio_queue_enable (vdev, vdev->queue_sel); proxy->vqs[vdev->queue_sel].enabled =1 ; proxy->vqs[vdev->queue_sel].reset =0 ; }else {hw/virtio/virtio.c +9
-0@@ -2050,6 +2050,15 @@ void virtio_queue_reset(VirtIODevice *vdev, uint32_t queue_index)__virtio_queue_reset (vdev, queue_index); } +void virtio_queue_enable (VirtIODevice *vdev,uint32_t queue_index) +{ +VirtioDeviceClass *k =VIRTIO_DEVICE_GET_CLASS (vdev); + +if (k->queue_enable) { + k->queue_enable (vdev, queue_index); + } +} +void virtio_reset (void *opaque) {VirtIODevice *vdev = opaque;include/hw/virtio/virtio.h +2
-0@@ -131,6 +131,7 @@ struct VirtioDeviceClass {void (*reset)(VirtIODevice *vdev);void (*set_status)(VirtIODevice *vdev,uint8_t val);void (*queue_reset)(VirtIODevice *vdev,uint32_t queue_index); +void (*queue_enable)(VirtIODevice *vdev,uint32_t queue_index);/* For transitional devices, this is a bitmap of features * that are only exposed on the legacy interface but not * the modern one. @@ -270,6 +271,7 @@ int virtio_queue_set_host_notifier_mr(VirtIODevice *vdev, int n,int virtio_set_status (VirtIODevice *vdev,uint8_t val);void virtio_reset (void *opaque);void virtio_queue_reset (VirtIODevice *vdev,uint32_t queue_index); +void virtio_queue_enable (VirtIODevice *vdev,uint32_t queue_index);void virtio_update_irq (VirtIODevice *vdev);int virtio_set_features (VirtIODevice *vdev,uint64_t val);
Re: [PATCH 13/16] virtio: introduce queue_enable in virtio
Quoted 9 lines · Jason Wang
Quoted 3 lines · Kangjie Xu
Introduce the interface
queue_enable()in VirtioDeviceClass and the fucntionvirtio_queue_enable()in virtio, it can be called whenVIRTIO_PCI_COMMON_Q_ENABLEis written.I'd suggest to split this series into two.
Quoted 8 lines
1)
queue_enable2)
queue_resetThe patch looks good to me.
Thanks
Okay, will fix.
Thanks
Quoted 63 lines
Quoted 63 lines
Signed-off-by Kangjie Xu <[email protected]>Signed-off-by Xuan Zhuo <[email protected]>---
hw/virtio/virtio-pci.c | 1 +
hw/virtio/virtio.c | 9 +++++++++
include/hw/virtio/virtio.h| 2 ++3 files changed, 12 insertions(+)hw/virtio/virtio-pci.c +0
-0@@ -1335,6 +1335,7 @@ static void virtio_pci_common_write(void *opaque, hwaddr addr, proxy->vqs[vdev->queue_sel].avail[0],((uint64_t)proxy->vqs[vdev->queue_sel].used[1]) << 32 | proxy->vqs[vdev->queue_sel].used[0]);
+virtio_queue_enable(vdev, vdev->queue_sel); proxy->vqs[vdev->queue_sel].enabled = 1; proxy->vqs[vdev->queue_sel].reset = 0; } else {hw/virtio/virtio.c +8
-0@@ -2050,6 +2050,15 @@ void virtio_queue_reset(VirtIODevice *vdev, uint32_t queue_index) __virtio_queue_reset(vdev, queue_index); } +void virtio_queue_enable(VirtIODevice *vdev, uint32_t queue_index) +{ + VirtioDeviceClass *k = VIRTIO_DEVICE_GET_CLASS(vdev); + + if (k->queue_enable) { + k->queue_enable(vdev, queue_index); + } +} + void virtio_reset(void *opaque) {VirtIODevice *vdev = opaque;
include/hw/virtio/virtio.h +2
-0@@ -131,6 +131,7 @@ struct VirtioDeviceClass { void (*reset)(VirtIODevice *vdev); void (*set_status)(VirtIODevice *vdev, uint8_t val); void (*queue_reset)(VirtIODevice *vdev, uint32_t queue_index); + void (*queue_enable)(VirtIODevice *vdev, uint32_t queue_index); /* For transitional devices, this is a bitmap of features* that are only exposed on the legacy interface but not * the modern one. @@ -270,6 +271,7 @@ int virtio_queue_set_host_notifier_mr(VirtIODevice *vdev, int n, int virtio_set_status(VirtIODevice *vdev, uint8_t val); void virtio_reset(void *opaque); void virtio_queue_reset(VirtIODevice *vdev, uint32_t queue_index); +void virtio_queue_enable (VirtIODevice *vdev,uint32_t queue_index); void virtio_update_irq(VirtIODevice *vdev); int virtio_set_features(VirtIODevice *vdev, uint64_t val);
PATCH 14/16 virtio-net: support queue_enable for vhost-user
In reply to Kangjie XuSupport queue enable in vhost-user scenario. It will be called when a vq reset operation is performed and the vq will be restared.
It should be noted that we can restart the vq when the vhost has already started. When launching a new vhost-user device, the vhost is not started and all vqs are not initalized until VIRTIO_PCI_COMMON_STATUS is written. Thus, we should use vhost_started to differentiate the two cases: vq reset and device start.
--- hw/net/virtio-net.c | 20 ++++++++++++++++++++ 1 file changed, 20 insertions(+)
hw/net/virtio-net.c +20 -0
@@ -544,6 +544,25 @@ static void virtio_net_queue_reset(VirtIODevice *vdev, uint32_t queue_index)assert (!virtio_net_get_subqueue (nc)->async_tx.elem); } +static void virtio_net_queue_enable (VirtIODevice *vdev,uint32_t queue_index) +{ +VirtIONet *n =VIRTIO_NET (vdev); +NetClientState *nc =qemu_get_subqueue (n->nic,vq2q (queue_index)); +int r; + +if (!nc->peer || !vdev->vhost_started) { +return ; + } + +if (nc->peer->info->type ==NET_CLIENT_DRIVER_VHOST_USER ) { + r =vhost_virtqueue_restart (vdev, nc, queue_index); +if (r <0 ) { +error_report ("unable to restart vhost net virtqueue: %d, " +"when resetting the queue" , queue_index); + } + } +} +static void virtio_net_reset (VirtIODevice *vdev) {VirtIONet *n =VIRTIO_NET (vdev);
@@ -3781,6 +3800,7 @@ static void virtio_net_class_init(ObjectClass *klass, void *data) vdc->bad_features = virtio_net_bad_features; vdc->reset = virtio_net_reset; vdc->queue_reset = virtio_net_queue_reset; + vdc->queue_enable = virtio_net_queue_enable; vdc->set_status = virtio_net_set_status; vdc->guest_notifier_mask = virtio_net_guest_notifier_mask; vdc->guest_notifier_pending = virtio_net_guest_notifier_pending;
Signature · 1 line
2.32.0
Re: [PATCH 14/16] virtio-net: support queue_enable for vhost-user
Quoted 31 lines · Kangjie Xu
Support queue enable in vhost-user scenario. It will be called when a vq reset operation is performed and the vq will be restared.
It should be noted that we can restart the vq when the vhost has already started. When launching a new vhost-user device, the vhost is not started and all vqs are not initalized until
VIRTIO_PCI_COMMON_STATUSis written. Thus, we should usevhost_startedto differentiate the two cases: vq reset and device start.Signed-off-by Kangjie Xu <[email protected]>Signed-off-by Xuan Zhuo <[email protected]>--- hw/net/virtio-net.c | 20 ++++++++++++++++++++ 1 file changed, 20 insertions(+)hw/net/virtio-net.c +8
-0@@ -544,6 +544,25 @@ static void virtio_net_queue_reset(VirtIODevice *vdev, uint32_t queue_index)assert (!virtio_net_get_subqueue (nc)->async_tx.elem); } +static void virtio_net_queue_enable (VirtIODevice *vdev,uint32_t queue_index) +{ +VirtIONet *n =VIRTIO_NET (vdev); +NetClientState *nc =qemu_get_subqueue (n->nic,vq2q (queue_index)); +int r; + +if (!nc->peer || !vdev->vhost_started) { +return ;
hw/net/virtio-net.c static void virtio_net_queue_reset(VirtIODevice *vdev, uint32_t queue_index) · line 564
Quoted 10 lines
hw/net/virtio-net.c +10
-0+ } + +if (nc->peer->info->type ==NET_CLIENT_DRIVER_VHOST_USER ) { + r =vhost_virtqueue_restart (vdev, nc, queue_index); +if (r <0 ) { +error_report ("unable to restart vhost net virtqueue: %d, " +"when resetting the queue" , queue_index); + } + } +}
So we don't check queue_enable in vhost_dev_start(), does this mean the queue_enable is actually meaningless (since the virtqueue is already started there)?
And another issue is
peet_attach/peer_detach() "abuse" vhost_set_vring_enable(). This probably means we need to invent new type of request instead of re-using VHOST_USER_SET_VRING_ENABLE.
Thanks
Quoted 12 lines
hw/net/virtio-net.c +2
-0+static void virtio_net_reset (VirtIODevice *vdev) {VirtIONet *n =VIRTIO_NET (vdev);@@ -3781,6 +3800,7 @@ static void virtio_net_class_init(ObjectClass *klass, void *data) vdc->bad_features = virtio_net_bad_features; vdc->reset = virtio_net_reset; vdc->queue_reset = virtio_net_queue_reset; + vdc->queue_enable = virtio_net_queue_enable; vdc->set_status = virtio_net_set_status; vdc->guest_notifier_mask = virtio_net_guest_notifier_mask; vdc->guest_notifier_pending = virtio_net_guest_notifier_pending;
Re: [PATCH 14/16] virtio-net: support queue_enable for vhost-user
Quoted 47 lines · Jason Wang
Quoted 31 lines · Kangjie Xu
Support queue enable in vhost-user scenario. It will be called when a vq reset operation is performed and the vq will be restared.
It should be noted that we can restart the vq when the vhost has already started. When launching a new vhost-user device, the vhost is not started and all vqs are not initalized until
VIRTIO_PCI_COMMON_STATUSis written. Thus, we should usevhost_startedto differentiate the two cases: vq reset and device start.Signed-off-by Kangjie Xu <[email protected]>Signed-off-by Xuan Zhuo <[email protected]>---
hw/net/virtio-net.c| 20 ++++++++++++++++++++1 file changed, 20 insertions(+)hw/net/virtio-net.c +7
-0@@ -544,6 +544,25 @@ static void virtio_net_queue_reset(VirtIODevice *vdev, uint32_t queue_index) assert(!virtio_net_get_subqueue(nc)->async_tx.elem); } +static void virtio_net_queue_enable(VirtIODevice *vdev, uint32_t queue_index) +{ + VirtIONet *n = VIRTIO_NET(vdev); + NetClientState *nc = qemu_get_subqueue(n->nic, vq2q(queue_index)); + int r; + + if (!nc->peer || !vdev->vhost_started) { + return;hw/net/virtio-net.c static void virtio_net_queue_reset(VirtIODevice · line 564
Quoted 10 lines
hw/net/virtio-net.c +10
-0+ } + + if (nc->peer->info->type == NET_CLIENT_DRIVER_VHOST_USER) { + r = vhost_virtqueue_restart(vdev, nc, queue_index); + if (r < 0) { + error_report("unable to restart vhost net virtqueue: %d, " + "when resetting the queue", queue_index); + } + } +}So we don't check
queue_enableinvhost_dev_start(), does this mean thequeue_enableis actually meaningless (since the virtqueue is already started there)?
Quoted 7 lines
And another issue is
peet_attach/peer_detach() "abuse"
vhost_set_vring_enable(). This probably means we need to invent new type of request instead of re-usingVHOST_USER_SET_VRING_ENABLE.Thanks
1. Yes, we don't need queue_enable in vhost_dev_start(), queue_enable is only useful when restarting the queue. I name it as queue_enable() simply because it is called when VIRTIO_PCI_COMMON_Q_ENABLE is written. Would it look better if we rename it as "queue_reenable"?
2. I think inventing a new type of vhost-protocol message can be a good choice. However, I don't know much about the vhost protocol. If we want to add a new message in vhost protocol, except the documentation and the code in qemu, Do we need to submit patches to other projects, e.g. some projects like virtio-spec?
Thanks
Quoted 13 lines
Quoted 13 lines
hw/net/virtio-net.c +1
-0+ static void virtio_net_reset(VirtIODevice *vdev) {VirtIONet *n =
VIRTIO_NET(vdev);hw/net/virtio-net.c +1
-0@@ -3781,6 +3800,7 @@ static void virtio_net_class_init(ObjectClass *klass, void *data) vdc->bad_features = virtio_net_bad_features; vdc->reset = virtio_net_reset; vdc->queue_reset = virtio_net_queue_reset; + vdc->queue_enable = virtio_net_queue_enable; vdc->set_status = virtio_net_set_status; vdc->guest_notifier_mask = virtio_net_guest_notifier_mask;
vdc->guest_notifier_pending=virtio_net_guest_notifier_pending;
Re: [PATCH 14/16] virtio-net: support queue_enable for vhost-user
Quoted 59 lines · Kangjie Xu
Quoted 56 lines · Jason Wang
Quoted 32 lines · Kangjie Xu
Support queue enable in vhost-user scenario. It will be called when a vq reset operation is performed and the vq will be restared.
It should be noted that we can restart the vq when the vhost has already started. When launching a new vhost-user device, the vhost is not started and all vqs are not initalized until
VIRTIO_PCI_COMMON_STATUSis written. Thus, we should usevhost_startedto differentiate the two cases: vq reset and device start.Signed-off-by Kangjie Xu <[email protected]>Signed-off-by Xuan Zhuo <[email protected]>--- hw/net/virtio-net.c | 20 ++++++++++++++++++++ 1 file changed, 20 insertions(+)hw/net/virtio-net.c +7
-0@@ -544,6 +544,25 @@ static void virtio_net_queue_reset(VirtIODevice *vdev, uint32_t queue_index)assert (!virtio_net_get_subqueue (nc)->async_tx.elem); } +static void virtio_net_queue_enable(VirtIODevice *vdev, uint32_t queue_index) +{ +VirtIONet *n =VIRTIO_NET (vdev); +NetClientState *nc =qemu_get_subqueue (n->nic,vq2q (queue_index)); +int r; + +if (!nc->peer || !vdev->vhost_started) { +return ;hw/net/virtio-net.c static void virtio_net_queue_reset(VirtIODevice · line 565
Quoted 10 lines
hw/net/virtio-net.c +10
-0+ } + +if (nc->peer->info->type ==NET_CLIENT_DRIVER_VHOST_USER ) { + r =vhost_virtqueue_restart (vdev, nc, queue_index); +if (r <0 ) { +error_report ("unable to restart vhost net virtqueue: %d, " +"when resetting the queue" , queue_index); + } + } +}So we don't check
queue_enableinvhost_dev_start(), does this mean thequeue_enableis actually meaningless (since the virtqueue is already started there)?And another issue is
peet_attach/peer_detach() "abuse"
vhost_set_vring_enable(). This probably means we need to invent new type of request instead of re-usingVHOST_USER_SET_VRING_ENABLE.
Quoted 7 lines
Quoted 3 lines
Thanks
1. Yes, we don't need
queue_enableinvhost_dev_start(),queue_enableis only useful when restarting the queue. I name it asqueue_enable()simply because it is called whenVIRTIO_PCI_COMMON_Q_ENABLEis written. Would it look better if we rename it as "queue_reenable"?
I think the right approach is probably:
1) when VERSION_1 is negotiated, only start the virtqueue when queue_enable is 1
2) when VERSION_1 is not negotiated, start virtqueue when DRIVER_OK (vhost_dev_start())
?
Quoted 6 lines
2. I think inventing a new type of vhost-protocol message can be a good choice. However, I don't know much about the vhost protocol. If we want to add a new message in vhost protocol, except the documentation and the code in qemu, Do we need to submit patches to other projects, e.g. some projects like virtio-spec?
Probably not since vhost-user doesn't belong to the spec currently. The doc in qemu should be sufficient.
Thanks
Quoted 17 lines
Thanks
Quoted 13 lines
Quoted 13 lines
hw/net/virtio-net.c +2
-0+static void virtio_net_reset (VirtIODevice *vdev) {VirtIONet *n =VIRTIO_NET (vdev);@@ -3781,6 +3800,7 @@ static void virtio_net_class_init(ObjectClass *klass, void *data) vdc->bad_features = virtio_net_bad_features; vdc->reset = virtio_net_reset; vdc->queue_reset = virtio_net_queue_reset; + vdc->queue_enable = virtio_net_queue_enable; vdc->set_status = virtio_net_set_status; vdc->guest_notifier_mask = virtio_net_guest_notifier_mask;vdc->guest_notifier_pending = virtio_net_guest_notifier_pending;
PATCH 15/16 virtio-net: support queue_reset for vhost-user
In reply to Kangjie XuSupport queue reset in vhost-user scenario.
--- hw/net/virtio-net.c | 4 ++++ 1 file changed, 4 insertions(+)
hw/net/virtio-net.c +4 -0
@@ -540,6 +540,10 @@ static void virtio_net_queue_reset(VirtIODevice *vdev, uint32_t queue_index)return ; } +if (nc->peer->info->type ==NET_CLIENT_DRIVER_VHOST_USER ) { +vhost_virtqueue_stop (vdev, nc, queue_index); + } +qemu_flush_or_purge_queued_packets (nc->peer, true);assert (!virtio_net_get_subqueue (nc)->async_tx.elem); }
Signature · 1 line
2.32.0
PATCH 16/16 vhost-net: vq reset feature bit support
In reply to Kangjie XuAdd support for negotation of vq reset feature bit.
--- hw/net/vhost_net.c | 1 + hw/net/virtio-net.c | 3 ++- 2 files changed, 3 insertions(+), 1 deletion(-)
hw/net/vhost_net.c +1 -0
@@ -73,6 +73,7 @@ static const int user_feature_bits[] = {VIRTIO_NET_F_MTU ,VIRTIO_F_IOMMU_PLATFORM ,VIRTIO_F_RING_PACKED , +VIRTIO_F_RING_RESET ,VIRTIO_NET_F_RSS , VIRTIO_NET_F_HASH_REPORT,
hw/net/virtio-net.c +2 -1
@@ -757,6 +757,8 @@ static uint64_t virtio_net_get_features(VirtIODevice *vdev, uint64_t features,virtio_add_feature (&features,VIRTIO_NET_F_MAC ); +virtio_add_feature (&features,VIRTIO_F_RING_RESET ); +if (!peer_has_vnet_hdr (n)) {virtio_clear_feature (&features,VIRTIO_NET_F_CSUM );virtio_clear_feature (&features,VIRTIO_NET_F_HOST_TSO4 );
@@ -777,7 +779,6 @@ static uint64_t virtio_net_get_features(VirtIODevice *vdev, uint64_t features, }if (!get_vhost_net (nc->peer)) { -virtio_add_feature (&features,VIRTIO_F_RING_RESET );return features; }
Signature · 1 line
2.32.0
Re: [PATCH 16/16] vhost-net: vq reset feature bit support
Quoted 4 lines · Kangjie Xu
Add support for negotation of vq reset feature bit.
Signed-off-by Kangjie Xu <[email protected]>Signed-off-by Xuan Zhuo <[email protected]>
I'd suggest to add support for vhost-net kernel as well. It looks much more easier than vhost-user (I guess a stop/start would do the trick).
Thanks
Quoted 38 lines
--- hw/net/vhost_net.c | 1 + hw/net/virtio-net.c | 3 ++- 2 files changed, 3 insertions(+), 1 deletion(-)hw/net/vhost_net.c +1
-0@@ -73,6 +73,7 @@ static const int user_feature_bits[] = {VIRTIO_NET_F_MTU ,VIRTIO_F_IOMMU_PLATFORM ,VIRTIO_F_RING_PACKED , +VIRTIO_F_RING_RESET ,VIRTIO_NET_F_RSS , VIRTIO_NET_F_HASH_REPORT,hw/net/virtio-net.c +2
-1@@ -757,6 +757,8 @@ static uint64_t virtio_net_get_features(VirtIODevice *vdev, uint64_t features,virtio_add_feature (&features,VIRTIO_NET_F_MAC ); +virtio_add_feature (&features,VIRTIO_F_RING_RESET ); +if (!peer_has_vnet_hdr (n)) {virtio_clear_feature (&features,VIRTIO_NET_F_CSUM );virtio_clear_feature (&features,VIRTIO_NET_F_HOST_TSO4 );@@ -777,7 +779,6 @@ static uint64_t virtio_net_get_features(VirtIODevice *vdev, uint64_t features, }if (!get_vhost_net (nc->peer)) { -virtio_add_feature (&features,VIRTIO_F_RING_RESET );return features; }
Re: [PATCH 16/16] vhost-net: vq reset feature bit support
Quoted 4 lines · Jason Wang
Quoted 2 lines · Kangjie Xu
Add support for negotation of vq reset feature bit.
Quoted 9 lines
Quoted 2 lines
Signed-off-by Kangjie Xu <[email protected]>Signed-off-by Xuan Zhuo <[email protected]>I'd suggest to add support for vhost-net kernel as well. It looks much more easier than vhost-user (I guess a stop/start would do the trick).
Thanks
Yeah, we've planned to support it in the future.
Thanks
Quoted 35 lines
Quoted 35 lines
---
hw/net/vhost_net.c | 1 +
hw/net/virtio-net.c| 3 ++-2 files changed, 3 insertions(+), 1 deletion(-)hw/net/vhost_net.c +1
-0@@ -73,6 +73,7 @@ static const int user_feature_bits[] = { VIRTIO_NET_F_MTU, VIRTIO_F_IOMMU_PLATFORM, VIRTIO_F_RING_PACKED, + VIRTIO_F_RING_RESET, VIRTIO_NET_F_RSS, VIRTIO_NET_F_HASH_REPORT,diff --git
a/hw/net/virtio-net.cb/hw/net/virtio-net.cindex0747ffe71c..a8b299067a100644hw/net/virtio-net.c +1
-1@@ -757,6 +757,8 @@ static uint64_t virtio_net_get_features(VirtIODevice *vdev, uint64_t features, virtio_add_feature(&features, VIRTIO_NET_F_MAC); + virtio_add_feature(&features, VIRTIO_F_RING_RESET); + if (!peer_has_vnet_hdr(n)) { virtio_clear_feature(&features, VIRTIO_NET_F_CSUM); virtio_clear_feature(&features, VIRTIO_NET_F_HOST_TSO4);@@ -777,7 +779,6 @@ static uint64_t virtio_net_get_features(VirtIODevice *vdev, uint64_t features, } if (!get_vhost_net(nc->peer)) { - virtio_add_feature(&features, VIRTIO_F_RING_RESET); return features; }
Re: [PATCH 16/16] vhost-net: vq reset feature bit support
Quoted 11 lines · Kangjie Xu
Quoted 8 lines · Jason Wang
Quoted 4 lines · Kangjie Xu
Add support for negotation of vq reset feature bit.
Signed-off-by Kangjie Xu <[email protected]>Signed-off-by Xuan Zhuo <[email protected]>
Quoted 7 lines
Quoted 6 lines
I'd suggest to add support for vhost-net kernel as well. It looks much more easier than vhost-user (I guess a stop/start would do the trick).
Thanks
Yeah, we've planned to support it in the future.
If it's possible, I suggest to implement in this series. It would be easier since it current kernel support it already (via SET_BACKEND).
Thanks
Quoted 39 lines
Thanks
Quoted 35 lines
Quoted 35 lines
--- hw/net/vhost_net.c | 1 + hw/net/virtio-net.c | 3 ++- 2 files changed, 3 insertions(+), 1 deletion(-)hw/net/vhost_net.c +1
-0@@ -73,6 +73,7 @@ static const int user_feature_bits[] = {VIRTIO_NET_F_MTU ,VIRTIO_F_IOMMU_PLATFORM ,VIRTIO_F_RING_PACKED , +VIRTIO_F_RING_RESET ,VIRTIO_NET_F_RSS ,VIRTIO_NET_F_HASH_REPORT , diff --git a/hw/net/virtio-net.c b/hw/net/virtio-net.cindex
0747ffe71c..a8b299067a100644hw/net/virtio-net.c +1
-1@@ -757,6 +757,8 @@ static uint64_t virtio_net_get_features(VirtIODevice *vdev, uint64_t features, virtio_add_feature(&features, VIRTIO_NET_F_MAC); +virtio_add_feature (&features,VIRTIO_F_RING_RESET ); +if (!peer_has_vnet_hdr (n)) {virtio_clear_feature (&features,VIRTIO_NET_F_CSUM );virtio_clear_feature (&features,VIRTIO_NET_F_HOST_TSO4 );@@ -777,7 +779,6 @@ static uint64_t virtio_net_get_features(VirtIODevice *vdev, uint64_t features, } if (!get_vhost_net(nc->peer)) { -virtio_add_feature (&features,VIRTIO_F_RING_RESET );return features; }
Re: [PATCH 16/16] vhost-net: vq reset feature bit support
Quoted 12 lines · Jason Wang
Quoted 11 lines · Kangjie Xu
Quoted 9 lines · Jason Wang
Quoted 4 lines · Kangjie Xu
Add support for negotation of vq reset feature bit.
Signed-off-by Kangjie Xu <[email protected]>Signed-off-by Xuan Zhuo <[email protected]>I'd suggest to add support for vhost-net kernel as well. It looks much more easier than vhost-user (I guess a stop/start would do the trick).
Quoted 8 lines
Quoted 4 lines
Quoted 3 lines
Thanks
Yeah, we've planned to support it in the future.
If it's possible, I suggest to implement in this series. It would be easier since it current kernel support it already (via
SET_BACKEND).Thanks
Okay, in the next version, we will add the implementation for vhost-kernel vq reset.
Thanks
Quoted 37 lines
Quoted 37 lines
Thanks
Quoted 35 lines
Quoted 35 lines
--- hw/net/vhost_net.c | 1 + hw/net/virtio-net.c | 3 ++- 2 files changed, 3 insertions(+), 1 deletion(-)hw/net/vhost_net.c +1
-0@@ -73,6 +73,7 @@ static const int user_feature_bits[] = {VIRTIO_NET_F_MTU ,VIRTIO_F_IOMMU_PLATFORM ,VIRTIO_F_RING_PACKED , +VIRTIO_F_RING_RESET ,VIRTIO_NET_F_RSS ,VIRTIO_NET_F_HASH_REPORT , diff --git a/hw/net/virtio-net.c b/hw/net/virtio-net.cindex
0747ffe71c..a8b299067a100644hw/net/virtio-net.c +1
-1@@ -757,6 +757,8 @@ static uint64_t virtio_net_get_features(VirtIODevice *vdev, uint64_t features, virtio_add_feature(&features, VIRTIO_NET_F_MAC); +virtio_add_feature (&features,VIRTIO_F_RING_RESET ); +if (!peer_has_vnet_hdr (n)) {virtio_clear_feature (&features,VIRTIO_NET_F_CSUM );virtio_clear_feature (&features,VIRTIO_NET_F_HOST_TSO4 );@@ -777,7 +779,6 @@ static uint64_t virtio_net_get_features(VirtIODevice *vdev, uint64_t features, } if (!get_vhost_net(nc->peer)) { -virtio_add_feature (&features,VIRTIO_F_RING_RESET );return features; }
Do you have any comments or suggestions about improvements to this patch set?
Regards,
Kangjie
Quoted 71 lines · Kangjie Xu
The virtio queue reset function has already been defined in the virtio spec 1.2. The relevant virtio spec information is here:
https://github.com/oasis-tcs/virtio-spec/issues/124 https://github.com/oasis-tcs/virtio-spec/issues/139This patch set is to support this function in QEMU. It consists of two parts: virtio-net and vhost-user. The patches 1-7 are the implementation for virtio-net and the patches 8-16 are for vhost-user.
The process of virt queue reset can be concluded as:
1. The virtqueue is disabled whenVIRTIO_PCI_COMMON_Q_RESETis written.
2. Then the virtqueue is restarted after the information of vrings is passed to QEMU andVIRTIO_PCI_COMMON_Q_ENABLEis written.Test environment:
Host: 5.4.189
Qemu: QEMU emulator version 7.0.50
Guest: 5.19.0-rc3 (With vq reset support)
DPDK: 22.07-rc1 (With vq reset support)
Test Cmd: ethtool -g eth1; ethtool -G eth1 rx $1 tx $2; ethtool -g eth1;The frontend can resize the virtio queue, then virtio queue reset function should be triggered.
The default is split mode, modify Qemu virtio-net to add PACKED feature to test packed mode.
Guest Kernel Patch:
https://lore.kernel.org/bpf/[email protected]/DPDK Code:
https://github.com/middaywords/dpdk/commit/098c8e1dfae10b747da8dd8950a82890efca7beePlease review. Thanks.
Kangjie Xu (9):
vhost: add op to enable or disable a single vring
vhost-user: enable/disable a single vring
vhost: extract the logic of unmapping the vrings and desc
vhost: introduce restart and release for vhost_dev's vqs
vhost-net: introduce restart and stop for vhost_net's vqs
virtio: introducequeue_enablein virtio
virtio-net: supportqueue_enablefor vhost-user
virtio-net: supportqueue_resetfor vhost-user
vhost-net: vq reset feature bit supportXuan Zhuo (7):
virtio-pci:virtio_pci_common_cfgaddqueue_notify_data
virtio: addVIRTIO_F_RING_RESET
virtio: pci:virtio_pci_common_cfgaddqueue_reset
virtio: introduce__virtio_queue_reset()
virtio: introducevirtio_queue_reset()
virtio-pci: support queue reset
virtio-net: support queue resethw/net/vhost_net.c | 56 ++++++++++++++++++ hw/net/virtio-net.c | 40 +++++++++++++ hw/virtio/vhost-user.c | 55 +++++++++++++++--- hw/virtio/vhost.c | 49 ++++++++++++++-- hw/virtio/virtio-pci.c | 17 ++++++ hw/virtio/virtio.c | 57 +++++++++++++------ include/hw/virtio/vhost-backend.h | 4 ++ include/hw/virtio/vhost.h | 6 ++ include/hw/virtio/virtio-pci.h | 1 + include/hw/virtio/virtio.h | 4 ++ include/net/vhost_net.h | 5 ++ .../standard-headers/linux/virtio_config.h | 5 ++ include/standard-headers/linux/virtio_pci.h | 4 ++ 13 files changed, 274 insertions(+), 29 deletions(-)
Quoted 3 lines · Kangjie Xu
Do you have any comments or suggestions about improvements to this patch set?
Will have a look and give feedback no later than the end of this week.
Thanks
Quoted 79 lines
Regards,
Kangjie
Quoted 71 lines · Kangjie Xu
The virtio queue reset function has already been defined in the virtio spec 1.2. The relevant virtio spec information is here:
https://github.com/oasis-tcs/virtio-spec/issues/124 https://github.com/oasis-tcs/virtio-spec/issues/139This patch set is to support this function in QEMU. It consists of two parts: virtio-net and vhost-user. The patches 1-7 are the implementation for virtio-net and the patches 8-16 are for vhost-user.
The process of virt queue reset can be concluded as:
1. The virtqueue is disabled whenVIRTIO_PCI_COMMON_Q_RESETis written.
2. Then the virtqueue is restarted after the information of vrings is passed to QEMU andVIRTIO_PCI_COMMON_Q_ENABLEis written.Test environment:
Host: 5.4.189
Qemu: QEMU emulator version 7.0.50
Guest: 5.19.0-rc3 (With vq reset support)
DPDK: 22.07-rc1 (With vq reset support)
Test Cmd: ethtool -g eth1; ethtool -G eth1 rx $1 tx $2; ethtool -g eth1;The frontend can resize the virtio queue, then virtio queue reset function should be triggered.
The default is split mode, modify Qemu virtio-net to add PACKED feature to test packed mode.
Guest Kernel Patch:
https://lore.kernel.org/bpf/[email protected]/DPDK Code:
https://github.com/middaywords/dpdk/commit/098c8e1dfae10b747da8dd8950a82890efca7beePlease review. Thanks.
Kangjie Xu (9):
vhost: add op to enable or disable a single vring
vhost-user: enable/disable a single vring
vhost: extract the logic of unmapping the vrings and desc
vhost: introduce restart and release for vhost_dev's vqs
vhost-net: introduce restart and stop for vhost_net's vqs
virtio: introducequeue_enablein virtio
virtio-net: supportqueue_enablefor vhost-user
virtio-net: supportqueue_resetfor vhost-user
vhost-net: vq reset feature bit supportXuan Zhuo (7):
virtio-pci:virtio_pci_common_cfgaddqueue_notify_data
virtio: addVIRTIO_F_RING_RESET
virtio: pci:virtio_pci_common_cfgaddqueue_reset
virtio: introduce__virtio_queue_reset()
virtio: introducevirtio_queue_reset()
virtio-pci: support queue reset
virtio-net: support queue resethw/net/vhost_net.c | 56 ++++++++++++++++++ hw/net/virtio-net.c | 40 +++++++++++++ hw/virtio/vhost-user.c | 55 +++++++++++++++--- hw/virtio/vhost.c | 49 ++++++++++++++-- hw/virtio/virtio-pci.c | 17 ++++++ hw/virtio/virtio.c | 57 +++++++++++++------ include/hw/virtio/vhost-backend.h | 4 ++ include/hw/virtio/vhost.h | 6 ++ include/hw/virtio/virtio-pci.h | 1 + include/hw/virtio/virtio.h | 4 ++ include/net/vhost_net.h | 5 ++ .../standard-headers/linux/virtio_config.h | 5 ++ include/standard-headers/linux/virtio_pci.h | 4 ++ 13 files changed, 274 insertions(+), 29 deletions(-)