Skip to content

RFC vDPA-net inflight descriptors migration with SVQ

Eugenio Pérez qemu-devel 1 version In review

Get series with b4
b4 am [email protected]

Versions newest first

  1. v1 RFC Latest13 patches 16 repliesIn review

Patches in v1 No applies recorded yet

#Subject and filesSizeRepliesApplied
1/13vhost: add available descriptor list in SVQhw/virtio/vhost-shadow-virtqueue.c hw/virtio/vhost-shadow-virtqueue.h+11 -00 replies
2/13vhost: iterate only available descriptors at SVQ stophw/virtio/vhost-shadow-virtqueue.c+6 -50 replies
3/13vhost: merge avail list and next avail descriptors detachhw/virtio/vhost-shadow-virtqueue.c+9 -90 replies
4/13vhost: add vhost_svq_save_inflighthw/virtio/vhost-shadow-virtqueue.c hw/virtio/vhost-shadow-virtqueue.h+22 -00 replies
5/13virtio: Specify uint32_t as VirtQueueElementOld members typehw/virtio/virtio.c+3 -30 replies
6/13virtio: refactor qemu_get_virtqueue_elementhw/virtio/virtio.c+42 -260 replies
7/13virtio: refactor qemu_put_virtqueue_elementhw/virtio/virtio.c+26 -190 replies
8/13virtio: expose VirtQueueElementOldhw/virtio/virtio.c include/hw/virtio/virtio.h+36 -220 replies
9/13virtio: add vmstate_virtqueue_element_oldhw/virtio/virtio.c include/hw/virtio/virtio.h include/migration/vmstate.h+44 -00 replies
10/13virtio-net: Migrate vhost inflight descriptorshw/net/virtio-net.c include/hw/virtio/virtio-net.h include/migration/vmstate.h+142 -013 replies
11/13virtio-net: save inflight descriptors at vhost shutdownhw/net/vhost_net.c+22 -00 replies
12/13vhost: expose vhost_svq_add_elementhw/virtio/vhost-shadow-virtqueue.c hw/virtio/vhost-shadow-virtqueue.h+10 -30 replies
13/13vdpa: Recover inflight descriptorshw/net/vhost_net.c hw/virtio/vhost-vdpa.c+34 -110 replies

Reviews trailers per patch

No Reviewed-by, Acked-by or Tested-by trailers for v1 yet.

Read the v1 thread

Cover letter [RFC PATCH for 8.0 00/13] vDPA-net inflight descriptors migration with SVQ

The state of the descriptors (avail or used) may not be recoverable just looking at the guest memory. Out of order used descriptor may override previous avail ones in the descriptor table or avail vring.

Currently we're not migrating this status in net devices because virtio-net, vhost-kernel etc use the descriptors in order, so the information always recoverable from guest's memory. However, vDPA devices may use them out of order, and other kind of devices like block need this support.

Shadow virtqueue is able to track these and resend them at the destination. Add them to the virtio-net migration description so they are not lose in the process.

This is a very early RFC just to validate the first draft so expect leftovers. To fetch and request the descriptors from a device without SVQ need to be implemented on top. Some other notable pending items are:
* Do not send the descriptors actually recoverable from the guest memory.
* Properly version the migrate data.
* Properly abstract the descriptors access from virtio-net to SVQ.
* Do not use VirtQueueElementOld but migrate directly VirtQueueElement.
* Replace lots of assertions with runtime conditionals.
* Other TODOs in the patch message or code changes.

Thanks.

Eugenio Pérez (13):
vhost: add available descriptor list in SVQ
vhost: iterate only available descriptors at SVQ stop
vhost: merge avail list and next avail descriptors detach
vhost: add vhost_svq_save_inflight
virtio: Specify uint32_t as VirtQueueElementOld members type
virtio: refactor qemu_get_virtqueue_element
virtio: refactor qemu_put_virtqueue_element
virtio: expose VirtQueueElementOld
virtio: add vmstate_virtqueue_element_old
virtio-net: Migrate vhost inflight descriptors
virtio-net: save inflight descriptors at vhost shutdown
vhost: expose vhost_svq_add_element
vdpa: Recover inflight descriptors

 hw/virtio/vhost-shadow-virtqueue.h |   9 ++
 include/hw/virtio/virtio-net.h     |   2 +
 include/hw/virtio/virtio.h         |  32 ++++++
 include/migration/vmstate.h        |  22 ++++
 hw/net/vhost_net.c                 |  56 ++++++++++
 hw/net/virtio-net.c                | 129 +++++++++++++++++++++++
 hw/virtio/vhost-shadow-virtqueue.c |  52 +++++++--
 hw/virtio/vhost-vdpa.c             |  11 --
 hw/virtio/virtio.c                 | 162 ++++++++++++++++++-----------
 9 files changed, 392 insertions(+), 83 deletions(-)
Signature · 1 line
2.31.1