Net Control VQ support in vDPA SVQ
cover Net Control VQ support in vDPA SVQ
Control virtqueue is used by networking device for accepting various commands from the driver. It's a must to support multiqueue and other configurations.
Shadow VirtQueue (SVQ) [1] already makes possible migration of virtqueue states, effectively intercepting them so qemu can track what regions of memory are dirty because device action and needs migration. However, this does not solve networking device state seen by the driver because CVQ messages, like changes on MAC addresses from the driver.
To solve that, this series uses SVQ infraestructure proposed at SVQ [1] to intercept networking control messages used by the device. This way, qemu is able to update VirtIONet device model and to migrate it. This series needs to be applied on top of [1].
Ideally, only the control VQ would be shadowed for all the run of qemu and the rest of virtqueues would be passthrough unless it's migration time. However, this requires the vDPA device to support address translations from more than one address space, something that is not possible at the moment with the current vhost-vDPA API. The API change has been proposed at [2], but use of it is left for future series.
Sending this as a RFC so some details like error control is still not 100% tested. Comments are welcomed on every aspect of the patch.
[1] https://lore.kernel.org/qemu-devel/[email protected]/
[2] https://lkml.org/lkml/2020/9/23/1243
Eugenio Pérez (9):
virtio-net: Expose ctrl virtqueue logic
vdpa: Extract get geatures part from vhost_vdpa_get_max_queue_pairs
virtio: Make virtqueue_alloc_element non-static
vhost: Add SVQElement
vhost: Add custom used buffer callback
vdpa: Add map/unmap operation callback to SVQ
vhost: Add vhost_svq_inject
vhost: Add vhost_svq_start_op
vdpa: control virtqueue support on shadow virtqueue
hw/virtio/vhost-shadow-virtqueue.h | 25 +++- include/hw/virtio/vhost-vdpa.h | 2 + include/hw/virtio/virtio-net.h | 3 + include/hw/virtio/virtio.h | 1 + hw/net/virtio-net.c | 83 ++++++----- hw/virtio/vhost-shadow-virtqueue.c | 217 +++++++++++++++++++++++------ hw/virtio/vhost-vdpa.c | 22 ++- hw/virtio/virtio.c | 2 +- net/vhost-vdpa.c | 140 +++++++++++++++++-- 9 files changed, 405 insertions(+), 90 deletions(-)
Signature · 1 line
2.27.0
RFC PATCH 1/9 virtio-net: Expose ctrl virtqueue logic
This allows external vhost-net devices to modify the state of the VirtIO device model once vhost-vdpa device has acknowledge the control commands.
--- include/hw/virtio/virtio-net.h | 3 ++ hw/net/virtio-net.c | 83 ++++++++++++++++++++-------------- 2 files changed, 52 insertions(+), 34 deletions(-)
include/hw/virtio/virtio-net.h +3 -0
@@ -218,6 +218,9 @@ struct VirtIONet {struct EBPFRSSContext ebpf_rss; }; +unsigned virtio_net_handle_ctrl_iov (VirtIODevice *vdev, +const struct iovec *in_sg,size_t in_num, +struct iovec *out_sg,unsigned out_num);void virtio_net_set_netclient_name (VirtIONet *n,const char *name,const char *type);
hw/net/virtio-net.c +49 -34
105 lines not on this page · raw message
Signature · 1 line
2.27.0
RFC PATCH 2/9 vdpa: Extract get geatures part from vhost_vdpa_get_max_queue_pairs
In reply to Eugenio PérezTo know the device features is also needed for CVQ SVQ. Extract from vhost_vdpa_get_max_queue_pairs so we can reuse it.
Report errno in case of failure getting them while we're at it.
--- net/vhost-vdpa.c | 30 ++++++++++++++++++++---------- 1 file changed, 20 insertions(+), 10 deletions(-)
net/vhost-vdpa.c +20 -10
@@ -241,20 +241,24 @@ static NetClientState *net_vhost_vdpa_init(NetClientState *peer,return nc; } -static int vhost_vdpa_get_max_queue_pairs (int fd,int *has_cvq,Error **errp) +static int vhost_vdpa_get_features (int fd,uint64_t *features,Error **errp) +{ +int ret =ioctl (fd,VHOST_GET_FEATURES , features); +if (ret) { +error_setg_errno (errp, errno, +"Fail to query features from vhost-vDPA device" ); + } +return ret; +} + +static int vhost_vdpa_get_max_queue_pairs (int fd,uint64_t features, +int *has_cvq,Error **errp) {unsigned long config_size = offsetof(struct vhost_vdpa_config , buf); g_autofree struct vhost_vdpa_config *config = NULL;__virtio16 *max_queue_pairs; -uint64_t features;int ret; - ret =ioctl (fd,VHOST_GET_FEATURES , &features); -if (ret) { -error_setg (errp,"Fail to query features from vhost-vDPA device" ); -return ret; - } -if (features & (1 <<VIRTIO_NET_F_CTRL_VQ )) { *has_cvq =1 ; }else {
@@ -285,10 +289,11 @@ int net_init_vhost_vdpa(const Netdev *netdev, const char *name, NetClientState *peer, Error **errp) {const NetdevVhostVDPAOptions *opts; +uint64_t features;int vdpa_device_fd; g_autofree NetClientState **ncs = NULL;NetClientState *nc; -int queue_pairs, i, has_cvq =0 ; +int queue_pairs, r, i, has_cvq =0 ;g_autoptr (VhostIOVATree) iova_tree =NULL ;struct vhost_vdpa_iova_range iova_range;
@@ -304,7 +309,12 @@ int net_init_vhost_vdpa(const Netdev *netdev, const char *name,return -errno; } - queue_pairs =vhost_vdpa_get_max_queue_pairs (vdpa_device_fd, + r =vhost_vdpa_get_features (vdpa_device_fd, &features, errp); +if (r) { +return r; + } + + queue_pairs =vhost_vdpa_get_max_queue_pairs (vdpa_device_fd, features, &has_cvq, errp);if (queue_pairs <0 ) {qemu_close (vdpa_device_fd);
Signature · 1 line
2.27.0
RFC PATCH 3/9 virtio: Make virtqueue_alloc_element non-static
In reply to Eugenio PérezSo SVQ can allocate elements using it
--- include/hw/virtio/virtio.h | 1 + hw/virtio/virtio.c | 2 +- 2 files changed, 2 insertions(+), 1 deletion(-)
include/hw/virtio/virtio.h +1 -0
@@ -195,6 +195,7 @@ void virtqueue_fill(VirtQueue *vq, const VirtQueueElement *elem, unsigned int len, unsigned int idx);void virtqueue_map (VirtIODevice *vdev,VirtQueueElement *elem); +void *virtqueue_alloc_element (size_t sz,unsigned out_num,unsigned in_num);void *virtqueue_pop (VirtQueue *vq,size_t sz);unsigned int virtqueue_drop_all (VirtQueue *vq);void *qemu_get_virtqueue_element (VirtIODevice *vdev,QEMUFile *f,size_t sz);
hw/virtio/virtio.c +1 -1
@@ -1376,7 +1376,7 @@ void virtqueue_map(VirtIODevice *vdev, VirtQueueElement *elem) false); } -static void *virtqueue_alloc_element (size_t sz,unsigned out_num,unsigned in_num) +void *virtqueue_alloc_element (size_t sz,unsigned out_num,unsigned in_num) {VirtQueueElement *elem; size_t in_addr_ofs = QEMU_ALIGN_UP(sz, __alignof__(elem->in_addr[0]));
Signature · 1 line
2.27.0
RFC PATCH 4/9 vhost: Add SVQElement
In reply to Eugenio PérezThis allows SVQ to add metadata to the different queue elements
--- hw/virtio/vhost-shadow-virtqueue.c | 52 ++++++++++++++++++------------ 1 file changed, 31 insertions(+), 21 deletions(-)
hw/virtio/vhost-shadow-virtqueue.c +31 -21
@@ -17,6 +17,10 @@#include "qemu/error-report.h" #include "qemu/main-loop.h" +typedef struct SVQElement { +VirtQueueElement elem; +}SVQElement ; +/* Shadow virtqueue to relay notifications */ struct VhostShadowVirtqueue {/* Shadow vring */
@@ -50,10 +54,10 @@ struct VhostShadowVirtqueue {VhostIOVATree *iova_tree;/* Map for returning guest's descriptors */ -VirtQueueElement **ring_id_maps; +SVQElement **ring_id_maps;/* Next VirtQueue element that guest made available */ -VirtQueueElement *next_guest_avail_elem; +SVQElement *next_guest_avail_elem;/* Next head to expose to device */ uint16_t avail_idx_shadow;
@@ -281,9 +285,10 @@ static void vhost_vring_write_descs(VhostShadowVirtqueue *svq, }static bool vhost_svq_add_split (VhostShadowVirtqueue *svq, -VirtQueueElement *elem, +SVQElement *svq_elem,unsigned *head) { +const VirtQueueElement *elem = &svq_elem->elem;unsigned avail_idx;vring_avail_t *avail = svq->vring.avail;bool ok;
@@ -324,7 +329,7 @@ static bool vhost_svq_add_split(VhostShadowVirtqueue *svq,return true; } -static bool vhost_svq_add (VhostShadowVirtqueue *svq,VirtQueueElement *elem) +static bool vhost_svq_add (VhostShadowVirtqueue *svq,SVQElement *elem) {unsigned qemu_head;bool ok =vhost_svq_add_split (svq, elem, &qemu_head);
@@ -372,19 +377,21 @@ static void vhost_handle_guest_kick(VhostShadowVirtqueue *svq) }while (true) { +SVQElement *svq_elem;VirtQueueElement *elem;bool ok;if (svq->next_guest_avail_elem) { - elem =g_steal_pointer (&svq->next_guest_avail_elem); + svq_elem =g_steal_pointer (&svq->next_guest_avail_elem); }else { - elem =virtqueue_pop (svq->vq,sizeof (*elem)); + svq_elem =virtqueue_pop (svq->vq,sizeof (*elem)); } -if (!elem) { +if (!svq_elem) {break ; } + elem = &svq_elem->elem;if (elem->out_num + elem->in_num >vhost_svq_available_slots (svq)) {/*
@@ -398,11 +405,11 @@ static void vhost_handle_guest_kick(VhostShadowVirtqueue *svq)* queue the current guest descriptor and ignore further kicks * until some elements are used. */ - svq->next_guest_avail_elem = elem; + svq->next_guest_avail_elem = svq_elem;return ; } - ok =vhost_svq_add (svq, elem); + ok =vhost_svq_add (svq, svq_elem);if (unlikely (!ok)) {/* VQ is broken, just return and ignore any other kicks */ return ;
@@ -437,7 +444,7 @@ static bool vhost_svq_more_used(VhostShadowVirtqueue *svq)return svq->last_used_idx != svq->shadow_used_idx; } -static VirtQueueElement *vhost_svq_get_buf (VhostShadowVirtqueue *svq) +static SVQElement *vhost_svq_get_buf (VhostShadowVirtqueue *svq) {vring_desc_t *descs = svq->vring.desc;const vring_used_t *used = svq->vring.used;
@@ -471,7 +478,7 @@ static VirtQueueElement *vhost_svq_get_buf(VhostShadowVirtqueue *svq) descs[used_elem.id].next = svq->free_head; svq->free_head = used_elem.id; - svq->ring_id_maps[used_elem.id]->len = used_elem.len; + svq->ring_id_maps[used_elem.id]->elem.len = used_elem.len;return g_steal_pointer (&svq->ring_id_maps[used_elem.id]); }
@@ -486,11 +493,13 @@ static void vhost_svq_flush(VhostShadowVirtqueue *svq,vhost_svq_set_notification (svq, false);while (true) { - g_autofree VirtQueueElement *elem = vhost_svq_get_buf(svq); -if (!elem) { + g_autofree SVQElement *svq_elem = vhost_svq_get_buf(svq); +VirtQueueElement *elem; +if (!svq_elem) {break ; } + elem = &svq_elem->elem;if (unlikely (i >= svq->vring.num)) { virtio_error(svq->vdev, "More than %u used buffers obtained in a %u size SVQ",
@@ -667,7 +676,7 @@ void vhost_svq_start(VhostShadowVirtqueue *svq, VirtIODevice *vdev,void vhost_svq_stop (VhostShadowVirtqueue *svq) {event_notifier_set_handler (&svq->svq_kick,NULL ); - g_autofree VirtQueueElement *next_avail_elem = NULL; + g_autofree SVQElement *next_avail_elem = NULL;if (!svq->vq) {return ;
@@ -677,17 +686,18 @@ void vhost_svq_stop(VhostShadowVirtqueue *svq)vhost_svq_flush (svq, false);for (unsigned i =0 ; i < svq->vring.num; ++i) { - g_autofree VirtQueueElement *elem = NULL; - elem =g_steal_pointer (&svq->ring_id_maps[i]); -if (elem) { -virtqueue_detach_element (svq->vq, elem, elem->len); + g_autofree SVQElement *svq_elem = NULL; + svq_elem =g_steal_pointer (&svq->ring_id_maps[i]); +if (svq_elem) { +virtqueue_detach_element (svq->vq, &svq_elem->elem, + svq_elem->elem.len); } } next_avail_elem =g_steal_pointer (&svq->next_guest_avail_elem);if (next_avail_elem) { -virtqueue_detach_element (svq->vq, next_avail_elem, - next_avail_elem->len); +virtqueue_detach_element (svq->vq, &next_avail_elem->elem, + next_avail_elem->elem.len); } }
@@ -735,7 +745,7 @@ VhostShadowVirtqueue *vhost_svq_new(uint16_t qsize, VhostIOVATree *iova_tree) svq->vring.used =qemu_memalign (qemu_real_host_page_size, device_size);memset (svq->vring.used,0 , device_size); svq->iova_tree = iova_tree; - svq->ring_id_maps = g_new0(VirtQueueElement *, qsize); + svq->ring_id_maps = g_new0(SVQElement *, qsize);event_notifier_set_handler (&svq->hdev_call, vhost_svq_handle_call);return g_steal_pointer (&svq);
Signature · 1 line
2.27.0
RFC PATCH 5/9 vhost: Add custom used buffer callback
In reply to Eugenio PérezThe callback allows SVQ users to know the VirtQueue requests and responses. QEMU can use this to synchronize virtio device model state, allowing to migrate it with minimum changes to the migration code.
In the case of networking, this will be used to inspect control virtqueue messages.
--- hw/virtio/vhost-shadow-virtqueue.h | 10 +++++++++- include/hw/virtio/vhost-vdpa.h | 2 ++ hw/virtio/vhost-shadow-virtqueue.c | 15 ++++++++++++++- hw/virtio/vhost-vdpa.c | 4 ++-- 4 files changed, 27 insertions(+), 4 deletions(-)
hw/virtio/vhost-shadow-virtqueue.h +9 -1
@@ -16,6 +16,13 @@typedef struct VhostShadowVirtqueue VhostShadowVirtqueue ; +typedef void (*VirtQueueElementCallback )(VirtIODevice *vdev, +const VirtQueueElement *elem); + +typedef struct VhostShadowVirtqueueOps { +VirtQueueElementCallback used_elem_handler; +}VhostShadowVirtqueueOps ; +bool vhost_svq_valid_device_features (uint64_t *features);bool vhost_svq_valid_guest_features (uint64_t *features); bool vhost_svq_ack_guest_features(uint64_t dev_features,
@@ -39,7 +46,8 @@ void vhost_svq_start(VhostShadowVirtqueue *svq, VirtIODevice *vdev, VirtQueue *vq);void vhost_svq_stop (VhostShadowVirtqueue *svq); -VhostShadowVirtqueue *vhost_svq_new (uint16_t qsize,VhostIOVATree *iova_map); +VhostShadowVirtqueue *vhost_svq_new (uint16_t qsize,VhostIOVATree *iova_map, +const VhostShadowVirtqueueOps *ops);void vhost_svq_free (VhostShadowVirtqueue *vq);
include/hw/virtio/vhost-vdpa.h +2 -0
@@ -17,6 +17,7 @@#include "hw/virtio/vhost-iova-tree.h" #include "hw/virtio/virtio.h" #include "standard-headers/linux/vhost_types.h" +#include "hw/virtio/vhost-shadow-virtqueue.h" typedef struct VhostVDPAHostNotifier { MemoryRegion mr;
@@ -34,6 +35,7 @@ typedef struct vhost_vdpa {/* IOVA mapping used by Shadow Virtqueue */ VhostIOVATree *iova_tree;GPtrArray *shadow_vqs; +const VhostShadowVirtqueueOps *shadow_vq_ops;struct vhost_dev *dev;VhostVDPAHostNotifier notifier[VIRTIO_QUEUE_MAX ]; } VhostVDPA;
hw/virtio/vhost-shadow-virtqueue.c +14 -1
@@ -59,6 +59,12 @@ struct VhostShadowVirtqueue {/* Next VirtQueue element that guest made available */ SVQElement *next_guest_avail_elem; +/* Optional callbacks */ +const VhostShadowVirtqueueOps *ops; + +/* Optional custom used virtqueue element handler */ +VirtQueueElementCallback used_elem_cb; +/* Next head to expose to device */ uint16_t avail_idx_shadow;
@@ -509,6 +515,10 @@ static void vhost_svq_flush(VhostShadowVirtqueue *svq, i =0 ; }virtqueue_fill (vq, elem, elem->len, i++); + +if (svq->ops && svq->ops->used_elem_handler) { + svq->ops->used_elem_handler (svq->vdev, elem); + } }virtqueue_flush (vq, i);
@@ -707,12 +717,14 @@ void vhost_svq_stop(VhostShadowVirtqueue *svq)* * @qsize Shadow VirtQueue size * @iova_tree Tree to perform descriptors translations +* @used_cb Optional callback for each device's used buffer * * Returns the new virtqueue or NULL. * * In case of error, reason is reported through error_report. */ -VhostShadowVirtqueue *vhost_svq_new (uint16_t qsize,VhostIOVATree *iova_tree) +VhostShadowVirtqueue *vhost_svq_new (uint16_t qsize,VhostIOVATree *iova_tree, +const VhostShadowVirtqueueOps *ops) {size_t desc_size =sizeof (vring_desc_t) * qsize;size_t device_size, driver_size;
@@ -747,6 +759,7 @@ VhostShadowVirtqueue *vhost_svq_new(uint16_t qsize, VhostIOVATree *iova_tree) svq->iova_tree = iova_tree; svq->ring_id_maps = g_new0(SVQElement *, qsize);event_notifier_set_handler (&svq->hdev_call, vhost_svq_handle_call); + svq->ops = ops;return g_steal_pointer (&svq); err_init_hdev_call:
hw/virtio/vhost-vdpa.c +2 -2
@@ -17,7 +17,6 @@#include "hw/virtio/vhost.h" #include "hw/virtio/vhost-backend.h" #include "hw/virtio/virtio-net.h" -#include "hw/virtio/vhost-shadow-virtqueue.h" #include "hw/virtio/vhost-vdpa.h" #include "exec/address-spaces.h" #include "qemu/main-loop.h"
@@ -1219,7 +1218,8 @@ static int vhost_vdpa_init_svq(struct vhost_dev *hdev, struct vhost_vdpa *v,for (unsigned n =0 ; n < hdev->nvqs; ++n) {DMAMap device_region, driver_region;struct vhost_vring_addr addr; -VhostShadowVirtqueue *svq =vhost_svq_new (qsize, v->iova_tree); +VhostShadowVirtqueue *svq =vhost_svq_new (qsize, v->iova_tree, + v->shadow_vq_ops);if (unlikely (!svq)) {error_setg (errp,"Cannot create svq %u" , n);return -1 ;
Signature · 1 line
2.27.0
RFC PATCH 6/9 vdpa: Add map/unmap operation callback to SVQ
In reply to Eugenio Pérez--- hw/virtio/vhost-shadow-virtqueue.h | 13 ++++++++++++- hw/virtio/vhost-shadow-virtqueue.c | 12 +++++++++++- hw/virtio/vhost-vdpa.c | 20 +++++++++++++++++++- 3 files changed, 42 insertions(+), 3 deletions(-)
hw/virtio/vhost-shadow-virtqueue.h +12 -1
@@ -23,6 +23,15 @@ typedef struct VhostShadowVirtqueueOps {VirtQueueElementCallback used_elem_handler; } VhostShadowVirtqueueOps; +typedef int (*vhost_svq_map_op )(hwaddr iova,hwaddr size,void *vaddr, +bool readonly,void *opaque); +typedef int (*vhost_svq_unmap_op )(hwaddr iova,hwaddr size,void *opaque); + +typedef struct VhostShadowVirtqueueMapOps { +vhost_svq_map_op map; +vhost_svq_unmap_op unmap; +}VhostShadowVirtqueueMapOps ; +bool vhost_svq_valid_device_features (uint64_t *features);bool vhost_svq_valid_guest_features (uint64_t *features); bool vhost_svq_ack_guest_features(uint64_t dev_features,
@@ -47,7 +56,9 @@ void vhost_svq_start(VhostShadowVirtqueue *svq, VirtIODevice *vdev,void vhost_svq_stop (VhostShadowVirtqueue *svq);VhostShadowVirtqueue *vhost_svq_new (uint16_t qsize,VhostIOVATree *iova_map, -const VhostShadowVirtqueueOps *ops); +const VhostShadowVirtqueueOps *ops, +const VhostShadowVirtqueueMapOps *map_ops, +void *map_ops_opaque);void vhost_svq_free (VhostShadowVirtqueue *vq);
hw/virtio/vhost-shadow-virtqueue.c +11 -1
@@ -62,6 +62,12 @@ struct VhostShadowVirtqueue {/* Optional callbacks */ const VhostShadowVirtqueueOps *ops; +/* Device memory mapping callbacks */ +const VhostShadowVirtqueueMapOps *map_ops; + +/* Device memory mapping callbacks opaque */ +void *map_ops_opaque; +/* Optional custom used virtqueue element handler */ VirtQueueElementCallback used_elem_cb;
@@ -724,7 +730,9 @@ void vhost_svq_stop(VhostShadowVirtqueue *svq)* In case of error, reason is reported through error_report. */ VhostShadowVirtqueue *vhost_svq_new (uint16_t qsize,VhostIOVATree *iova_tree, -const VhostShadowVirtqueueOps *ops) +const VhostShadowVirtqueueOps *ops, +const VhostShadowVirtqueueMapOps *map_ops, +void *map_ops_opaque) {size_t desc_size =sizeof (vring_desc_t) * qsize;size_t device_size, driver_size;
@@ -760,6 +768,8 @@ VhostShadowVirtqueue *vhost_svq_new(uint16_t qsize, VhostIOVATree *iova_tree, svq->ring_id_maps = g_new0(SVQElement *, qsize);event_notifier_set_handler (&svq->hdev_call, vhost_svq_handle_call); svq->ops = ops; + svq->map_ops = map_ops; + svq->map_ops_opaque = map_ops_opaque;return g_steal_pointer (&svq); err_init_hdev_call:
hw/virtio/vhost-vdpa.c +19 -1
@@ -1178,6 +1178,22 @@ static int vhost_vdpa_get_max_queue_size(struct vhost_dev *dev,return vhost_vdpa_call (dev,VHOST_VDPA_GET_VRING_NUM , qsize); } +static int vhost_vdpa_svq_map (hwaddr iova,hwaddr size,void *vaddr, +bool readonly,void *opaque) +{ +return vhost_vdpa_dma_map (opaque, iova, size, vaddr, readonly); +} + +static int vhost_vdpa_svq_unmap (hwaddr iova,hwaddr size,void *opaque) +{ +return vhost_vdpa_dma_unmap (opaque, iova, size); +} + +static const VhostShadowVirtqueueMapOps vhost_vdpa_svq_map_ops = { + .map = vhost_vdpa_svq_map, + .unmap = vhost_vdpa_svq_unmap, +}; +static int vhost_vdpa_init_svq (struct vhost_dev *hdev,struct vhost_vdpa *v,Error **errp) {
@@ -1219,7 +1235,9 @@ static int vhost_vdpa_init_svq(struct vhost_dev *hdev, struct vhost_vdpa *v,DMAMap device_region, driver_region;struct vhost_vring_addr addr;VhostShadowVirtqueue *svq =vhost_svq_new (qsize, v->iova_tree, - v->shadow_vq_ops); + v->shadow_vq_ops, + &vhost_vdpa_svq_map_ops, + v);if (unlikely (!svq)) {error_setg (errp,"Cannot create svq %u" , n);return -1 ;
Signature · 1 line
2.27.0
RFC PATCH 7/9 vhost: Add vhost_svq_inject
In reply to Eugenio PérezThis allows qemu to inject packets to the device without guest's notice.
This will be use to inject net CVQ messages to restore status in the destination
--- hw/virtio/vhost-shadow-virtqueue.h | 2 + hw/virtio/vhost-shadow-virtqueue.c | 142 ++++++++++++++++++++++++----- 2 files changed, 123 insertions(+), 21 deletions(-)
hw/virtio/vhost-shadow-virtqueue.h +2 -0
@@ -38,6 +38,8 @@ bool vhost_svq_ack_guest_features(uint64_t dev_features,uint64_t guest_features, uint64_t *acked_features); +bool vhost_svq_inject (VhostShadowVirtqueue *svq,const struct iovec *iov, +size_t out_num,size_t in_num);void vhost_svq_set_svq_kick_fd (VhostShadowVirtqueue *svq,int svq_kick_fd);void vhost_svq_set_guest_call_notifier (VhostShadowVirtqueue *svq,int call_fd); const EventNotifier *vhost_svq_get_dev_kick_notifier(
hw/virtio/vhost-shadow-virtqueue.c +121 -21
192 lines not on this page · raw message
Signature · 1 line
2.27.0
Re: [RFC PATCH 7/9] vhost: Add vhost_svq_inject
Quoted 166 lines · Eugenio Pérez
This allows qemu to inject packets to the device without guest's notice.
This will be use to inject net CVQ messages to restore status in the destination
Signed-off-by Eugenio Pérez <[email protected]>--- hw/virtio/vhost-shadow-virtqueue.h | 2 + hw/virtio/vhost-shadow-virtqueue.c | 142 ++++++++++++++++++++++++----- 2 files changed, 123 insertions(+), 21 deletions(-)hw/virtio/vhost-shadow-virtqueue.h +2
-0@@ -38,6 +38,8 @@ bool vhost_svq_ack_guest_features(uint64_t dev_features,uint64_t guest_features, uint64_t *acked_features); +bool vhost_svq_inject (VhostShadowVirtqueue *svq,const struct iovec *iov, +size_t out_num,size_t in_num);void vhost_svq_set_svq_kick_fd (VhostShadowVirtqueue *svq,int svq_kick_fd);void vhost_svq_set_guest_call_notifier (VhostShadowVirtqueue *svq,int call_fd); const EventNotifier *vhost_svq_get_dev_kick_notifier(hw/virtio/vhost-shadow-virtqueue.c +84
-17@@ -16,9 +16,11 @@#include "qemu/error-report.h" #include "qemu/main-loop.h" +#include "qemu/iov.h" typedef struct SVQElement {VirtQueueElement elem; +bool not_from_guest; }SVQElement ;/* Shadow virtqueue to relay notifications */ @@ -311,19 +313,53 @@ static bool vhost_svq_add_split(VhostShadowVirtqueue *svq,/* We need some descriptors here */ assert (elem->out_num || elem->in_num); - ok =vhost_svq_translate_addr (svq, sgs, elem->out_sg, elem->out_num); -if (unlikely (!ok)) { -return false; +if (elem->out_sg && svq_elem->not_from_guest) { +DMAMap map = { + .translated_addr = (hwaddr )svq_elem->elem.out_sg->iov_base, + .size =ROUND_UP (elem->out_sg->iov_len,4096 ) -1 , + .perm =IOMMU_RO , + }; +int r =vhost_iova_tree_map_alloc (svq->iova_tree, &map); + + elem->out_addr[0 ] = map.iova; +assert (elem->out_num ==1 ); +assert (r ==IOVA_OK ); + + r = svq->map_ops->map (map.iova, map.size, (void *)map.translated_addr, + true, svq->map_ops_opaque); +assert (r ==0 ); + sgs[0 ] = (void *)map.iova; + }else { + ok =vhost_svq_translate_addr (svq, sgs, elem->out_sg, elem->out_num); +if (unlikely (!ok)) { +return false; + } }vhost_vring_write_descs (svq, sgs, elem->out_sg, elem->out_num, elem->in_num >0 , false); +if (elem->in_sg && svq_elem->not_from_guest) { +DMAMap map = { + .translated_addr = (hwaddr )svq_elem->elem.in_sg->iov_base, + .size =ROUND_UP (elem->in_sg->iov_len,4096 ) -1 , + .perm =IOMMU_RW , + }; +int r =vhost_iova_tree_map_alloc (svq->iova_tree, &map); - ok =vhost_svq_translate_addr (svq, sgs, elem->in_sg, elem->in_num); -if (unlikely (!ok)) { -return false; - } + elem->in_addr[0 ] = map.iova; +assert (elem->out_num ==1 ); +assert (r ==IOVA_OK ); + r = svq->map_ops->map (map.iova, map.size, (void *)map.translated_addr, + false, svq->map_ops_opaque); +assert (r ==0 ); + sgs[0 ] = (void *)map.iova; + }else { + ok =vhost_svq_translate_addr (svq, sgs, elem->in_sg, elem->in_num); +if (unlikely (!ok)) { +return false; + } + }vhost_vring_write_descs (svq, sgs, elem->in_sg, elem->in_num, false, true);/* @@ -364,6 +400,43 @@ static void vhost_svq_kick(VhostShadowVirtqueue *svq)event_notifier_set (&svq->hdev_kick); } +bool vhost_svq_inject (VhostShadowVirtqueue *svq,const struct iovec *iov, +size_t out_num,size_t in_num) +{ +size_t out_size =iov_size (iov, out_num); +size_t out_buf_size =ROUND_UP (out_size,4096 ); +size_t in_size =iov_size (iov + out_num, in_num); +size_t in_buf_size =ROUND_UP (in_size,4096 ); +SVQElement *svq_elem; +uint16_t num_slots = (in_num ?1 :0 ) + (out_num ?1 :0 ); + +if (unlikely (num_slots ==0 || svq->next_guest_avail_elem || +vhost_svq_available_slots (svq) < num_slots)) { +return false; + } + + svq_elem =virtqueue_alloc_element (sizeof (SVQElement),1 ,1 ); +if (out_num) { +void *out =qemu_memalign (4096 , out_buf_size); + svq_elem->elem.out_sg[0 ].iov_base = out; + svq_elem->elem.out_sg[0 ].iov_len = out_size; +iov_to_buf (iov, out_num,0 , out, out_size); +memset (out + out_size,0 , out_buf_size - out_size); + } +if (in_num) { +void *in =qemu_memalign (4096 , in_buf_size); + svq_elem->elem.in_sg[0 ].iov_base = in; + svq_elem->elem.in_sg[0 ].iov_len = in_size; +memset (in,0 , in_buf_size); + } + + svq_elem->not_from_guest = true; +vhost_svq_add (svq, svq_elem); +vhost_svq_kick (svq); + +return true; +} +/** * Forward available buffers. * @@ -512,23 +585,50 @@ static void vhost_svq_flush(VhostShadowVirtqueue *svq, } elem = &svq_elem->elem; -if (unlikely (i >= svq->vring.num)) { -virtio_error (svq->vdev, -"More than %u used buffers obtained in a %u size SVQ" , - i, svq->vring.num); -virtqueue_fill (vq, elem, elem->len, i); -virtqueue_flush (vq, i); - i =0 ; - } -virtqueue_fill (vq, elem, elem->len, i++); -if (svq->ops && svq->ops->used_elem_handler) { svq->ops->used_elem_handler (svq->vdev, elem); } + +if (svq_elem->not_from_guest) { +const DMAMap out_map = { + .iova = elem->out_addr[0 ],
hw/virtio/vhost-shadow-virtqueue.c static void vhost_svq_flush(VhostShadowVirtqueue *svq, · line 604
Quoted 10 lines
hw/virtio/vhost-shadow-virtqueue.c +10
-0+ .translated_addr = (hwaddr)elem->out_sg[0].iov_base, + .size = elem->out_sg[0].iov_len, + }; +const DMAMap in_map = { + .iova = elem->in_addr[0 ], + .translated_addr = (hwaddr )elem->in_sg[0 ].iov_base, + .size = elem->in_sg[0 ].iov_len, + }; +vhost_iova_tree_remove (svq->iova_tree, &out_map); +if (svq->map_ops->unmap) {
This is actually bad, as is removing the SVQ element size, not the map size.
Same for the "in_map". I will fix it for the next revision.
Thanks!
Quoted 53 lines
hw/virtio/vhost-shadow-virtqueue.c +27
-5+ svq->map_ops->unmap (out_map.iova, in_map.size, + svq->map_ops_opaque); + } +qemu_vfree (elem->out_sg[0 ].iov_base); +vhost_iova_tree_remove (svq->iova_tree, &in_map); +if (svq->map_ops->unmap) { + svq->map_ops->unmap (out_map.iova, out_map.size, + svq->map_ops_opaque); + } +qemu_vfree (elem->in_sg[0 ].iov_base); + }else { +if (unlikely (i >= svq->vring.num)) { +virtio_error (svq->vdev, +"More than %u used buffers obtained in a %u size SVQ" , + i, svq->vring.num); +virtqueue_fill (vq, elem, elem->len, i); +virtqueue_flush (vq, i); + i =0 ; + } +virtqueue_fill (vq, elem, elem->len, i++); + } } -virtqueue_flush (vq, i); -event_notifier_set (&svq->svq_call); +if (i >0 ) { +virtqueue_flush (vq, i); +event_notifier_set (&svq->svq_call); + }if (check_for_avail_queue && svq->next_guest_avail_elem) {/* @@ -704,14 +804,14 @@ void vhost_svq_stop(VhostShadowVirtqueue *svq)for (unsigned i =0 ; i < svq->vring.num; ++i) { g_autofree SVQElement *svq_elem = NULL; svq_elem =g_steal_pointer (&svq->ring_id_maps[i]); -if (svq_elem) { +if (svq_elem && !svq_elem->not_from_guest) {virtqueue_detach_element (svq->vq, &svq_elem->elem, svq_elem->elem.len); } } next_avail_elem =g_steal_pointer (&svq->next_guest_avail_elem); -if (next_avail_elem) { +if (next_avail_elem && !next_avail_elem->not_from_guest) {virtqueue_detach_element (svq->vq, &next_avail_elem->elem, next_avail_elem->elem.len); } --2.27.0
RFC PATCH 8/9 vhost: Add vhost_svq_start_op
In reply to Eugenio Pérez--- hw/virtio/vhost-shadow-virtqueue.h | 4 +++- hw/virtio/vhost-shadow-virtqueue.c | 4 ++++ 2 files changed, 7 insertions(+), 1 deletion(-)
hw/virtio/vhost-shadow-virtqueue.h +3 -1
@@ -15,11 +15,13 @@#include "hw/virtio/vhost-iova-tree.h" typedef struct VhostShadowVirtqueue VhostShadowVirtqueue ; - +typedef bool (*vhost_svq_start_op )(VhostShadowVirtqueue *svq, +VirtIODevice *vdev);typedef void (*VirtQueueElementCallback )(VirtIODevice *vdev,const VirtQueueElement *elem);typedef struct VhostShadowVirtqueueOps { +vhost_svq_start_op start;VirtQueueElementCallback used_elem_handler; }VhostShadowVirtqueueOps ;
hw/virtio/vhost-shadow-virtqueue.c +4 -0
@@ -783,6 +783,10 @@ void vhost_svq_start(VhostShadowVirtqueue *svq, VirtIODevice *vdev,for (unsigned i =0 ; i < svq->vring.num -1 ; i++) { svq->vring.desc[i].next =cpu_to_le16 (i +1 ); } + +if (svq->ops && svq->ops->start) { + svq->ops->start (svq, vdev); + } }/**
Signature · 1 line
2.27.0
RFC PATCH 9/9 vdpa: control virtqueue support on shadow virtqueue
In reply to Eugenio PérezIntroduce the control virtqueue support for vDPA shadow virtqueue. This is needed for advanced networking features like multiqueue.
To demonstrate command handling, VIRTIO_NET_F_CTROL_MACADDR is implemented. If vDPA device is started with SVQ support, and MAC changes in the source VM, it will be transfered with the rest of properties in the emulated virtio-net device model.
A new CVQ command will be reproduced at destination so that NIC is aware of the changed MAC.
--- net/vhost-vdpa.c | 110 ++++++++++++++++++++++++++++++++++++++++++++++- 1 file changed, 108 insertions(+), 2 deletions(-)
net/vhost-vdpa.c +108 -2
@@ -11,6 +11,7 @@#include "qemu/osdep.h" #include "clients.h" +#include "hw/virtio/virtio-net.h" #include "net/vhost_net.h" #include "net/vhost-vdpa.h" #include "hw/virtio/vhost-vdpa.h"
@@ -70,6 +71,28 @@ const int vdpa_feature_bits[] = {VHOST_INVALID_FEATURE_BIT }; +/** Supported device specific feature bits with SVQ */ +static const uint64_t vdpa_svq_device_features = +BIT_ULL (VIRTIO_NET_F_CSUM ) | +BIT_ULL (VIRTIO_NET_F_GUEST_CSUM ) | +BIT_ULL (VIRTIO_NET_F_CTRL_GUEST_OFFLOADS ) | +BIT_ULL (VIRTIO_NET_F_MTU ) | +BIT_ULL (VIRTIO_NET_F_MAC ) | +BIT_ULL (VIRTIO_NET_F_GUEST_TSO4 ) | +BIT_ULL (VIRTIO_NET_F_GUEST_TSO6 ) | +BIT_ULL (VIRTIO_NET_F_GUEST_ECN ) | +BIT_ULL (VIRTIO_NET_F_GUEST_UFO ) | +BIT_ULL (VIRTIO_NET_F_HOST_TSO4 ) | +BIT_ULL (VIRTIO_NET_F_HOST_TSO6 ) | +BIT_ULL (VIRTIO_NET_F_HOST_ECN ) | +BIT_ULL (VIRTIO_NET_F_HOST_UFO ) | +BIT_ULL (VIRTIO_NET_F_MRG_RXBUF ) | +BIT_ULL (VIRTIO_NET_F_STATUS ) | +BIT_ULL (VIRTIO_NET_F_CTRL_VQ ) | +BIT_ULL (VIRTIO_NET_F_CTRL_MAC_ADDR ) | +BIT_ULL (VIRTIO_NET_F_RSC_EXT ) | +BIT_ULL (VIRTIO_NET_F_STANDBY ); +VHostNetState *vhost_vdpa_get_vhost_net (NetClientState *nc) {VhostVDPAState *s =DO_UPCAST (VhostVDPAState, nc, nc);
@@ -203,6 +226,79 @@ static void vhost_vdpa_get_iova_range(int fd, } } +static bool vhost_vdpa_start_control_svq (VhostShadowVirtqueue *svq, +VirtIODevice *vdev) +{ +VirtIONet *n =VIRTIO_NET (vdev); +NetClientState *nc =qemu_get_subqueue (n->nic, n->max_queue_pairs); +VhostVDPAState *s =DO_UPCAST (VhostVDPAState, nc, nc->peer); +uint64_t features = vdev->host_features; +assert (s->nc.info->type ==NET_CLIENT_DRIVER_VHOST_VDPA ); + +if (features &BIT_ULL (VIRTIO_NET_F_CTRL_MAC_ADDR )) { +const struct virtio_net_ctrl_hdr ctrl = { + .class =VIRTIO_NET_CTRL_MAC , + .cmd =VIRTIO_NET_CTRL_MAC_ADDR_SET , + }; +uint8_t mac[6 ]; +const struct iovec data[] = { + { + .iov_base = (void *)&ctrl, + .iov_len =sizeof (ctrl), + },{ + .iov_base = mac, + .iov_len =sizeof (mac), + },{ + .iov_base =NULL , + .iov_len =sizeof (virtio_net_ctrl_ack), + } + }; +bool ret; + +/* TODO: Only best effort? */ +memcpy (mac, n->mac,sizeof (mac)); + ret =vhost_svq_inject (svq, data,2 ,1 ); +if (!ret) { +return false; + } + } + +return true; +} + +static void vhost_vdpa_net_handle_ctrl (VirtIODevice *vdev, +const VirtQueueElement *elem) +{ +struct virtio_net_ctrl_hdr ctrl; +virtio_net_ctrl_ack status =VIRTIO_NET_ERR ; +size_t s; +struct iovec in = { + .iov_base = &status, + .iov_len =sizeof (status), + }; + + s =iov_to_buf (elem->out_sg, elem->out_num,0 , &ctrl,sizeof (ctrl.class)); +if (s !=sizeof (ctrl.class) || + ctrl.class !=VIRTIO_NET_CTRL_MAC_ADDR_SET ) { +return ; + } + s =iov_to_buf (elem->in_sg, elem->in_num,0 , &status,sizeof (status)); +if (s !=sizeof (status) || status !=VIRTIO_NET_OK ) { +return ; + } + + status =VIRTIO_NET_ERR ; +virtio_net_handle_ctrl_iov (vdev, &in,1 , elem->out_sg, elem->out_num); +if (status !=VIRTIO_NET_OK ) { +error_report ("Bad CVQ processing in model" ); + } +} + +static const VhostShadowVirtqueueOps vhost_vdpa_net_svq_ops = { + .start = vhost_vdpa_start_control_svq, + .used_elem_handler = vhost_vdpa_net_handle_ctrl, +}; + static NetClientState *net_vhost_vdpa_init(NetClientState *peer, const char *device, const char *name,
@@ -232,6 +328,9 @@ static NetClientState *net_vhost_vdpa_init(NetClientState *peer, s->vhost_vdpa.index = queue_pair_index; s->vhost_vdpa.iova_range = iova_range; s->vhost_vdpa.shadow_vqs_enabled = svq; +if (!is_datapath) { + s->vhost_vdpa.shadow_vq_ops = &vhost_vdpa_net_svq_ops; + } s->vhost_vdpa.iova_tree = iova_tree; ret =vhost_vdpa_add (nc, (void *)&s->vhost_vdpa, queue_pair_index, nvqs);if (ret) {
@@ -322,8 +421,15 @@ int net_init_vhost_vdpa(const Netdev *netdev, const char *name, }vhost_vdpa_get_iova_range (vdpa_device_fd, &iova_range);if (opts->x_svq) { -if (has_cvq) { -error_setg (errp,"vdpa svq does not work with cvq" ); +uint64_t invalid_dev_features = features & + ~vdpa_svq_device_features & +/* Transport are all accepted at this point */ + ~MAKE_64BIT_MASK (VIRTIO_TRANSPORT_F_START , +VIRTIO_TRANSPORT_F_END -VIRTIO_TRANSPORT_F_START ); + +if (invalid_dev_features) { +error_setg (errp,"vdpa svq does not work with features 0x%" PRIx64, + invalid_dev_features); goto err_svq; } iova_tree =vhost_iova_tree_new (iova_range.first, iova_range.last);
Signature · 1 line
2.27.0
Quoted 40 lines · Eugenio Pérez
Control virtqueue is used by networking device for accepting various commands from the driver. It's a must to support multiqueue and other configurations.
Shadow VirtQueue (SVQ) [1] already makes possible migration of virtqueue states, effectively intercepting them so qemu can track what regions of memory are dirty because device action and needs migration. However, this does not solve networking device state seen by the driver because CVQ messages, like changes on MAC addresses from the driver.
To solve that, this series uses SVQ infraestructure proposed at SVQ [1] to intercept networking control messages used by the device. This way, qemu is able to update VirtIONet device model and to migrate it. This series needs to be applied on top of [1].
Ideally, only the control VQ would be shadowed for all the run of qemu and the rest of virtqueues would be passthrough unless it's migration time. However, this requires the vDPA device to support address translations from more than one address space, something that is not possible at the moment with the current vhost-vDPA API. The API change has been proposed at [2], but use of it is left for future series.
Sending this as a RFC so some details like error control is still not 100% tested. Comments are welcomed on every aspect of the patch.
[1] https://lore.kernel.org/qemu-devel/[email protected]/
[2] https://lkml.org/lkml/2020/9/23/1243Eugenio Pérez (9):
virtio-net: Expose ctrl virtqueue logic
vdpa: Extract get geatures part fromvhost_vdpa_get_max_queue_pairs
virtio: Makevirtqueue_alloc_elementnon-static
vhost: Add SVQElement
vhost: Add custom used buffer callback
vdpa: Add map/unmap operation callback to SVQ
vhost: Addvhost_svq_inject
vhost: Addvhost_svq_start_op
vdpa: control virtqueue support on shadow virtqueue
Quoted 11 lines
hw/virtio/vhost-shadow-virtqueue.h | 25 +++- include/hw/virtio/vhost-vdpa.h | 2 + include/hw/virtio/virtio-net.h | 3 + include/hw/virtio/virtio.h | 1 + hw/net/virtio-net.c | 83 ++++++----- hw/virtio/vhost-shadow-virtqueue.c | 217 +++++++++++++++++++++++------ hw/virtio/vhost-vdpa.c | 22 ++- hw/virtio/virtio.c | 2 +- net/vhost-vdpa.c | 140 +++++++++++++++++-- 9 files changed, 405 insertions(+), 90 deletions(-)
This series is available to test at [1].
Thanks!
[1] https://github.com/eugpermar/qemu/releases/tag/vdpa_sw_live_migration.d%2Fcvq-v1
Quoted 5 lines
--
2.27.0