Skip to content

Control VQ support in vDPA

Jason Wang kvm lkml netdev oe-kbuild-all virtualization 1 version In review

Get series with b4
b4 am [email protected]

Versions newest first

  1. v1 Latest21 patches 29 repliesIn review

Changes since RFC from the cover letter

- tweak vhost uAPI documentation
- switch to use device specific IOTLB really in patch 4
- tweak the commit log
- fix that ASID in vhost is claimed to be 32 actually but 16bit actually
- fix use after free when using ASID with IOTLB batching requests
- switch to use Stefano's patch for having separated iov
- remove unused "used_as" variable
- fix the iotlb/asid checking in vhost_vdpa_unmap()

Patches in v1 No applies recorded yet

#Subject and filesSizeRepliesApplied
1/21vhost: move the backend feature bits to vhost_types.hinclude/uapi/linux/vhost.h include/uapi/linux/vhost_types.h+5 -50 replies
2/21virtio-vdpa: don't set callback if virtio doesn't need itdrivers/virtio/virtio_vdpa.c+1 -10 replies
3/21vhost-vdpa: passing iotlb to IOMMU mapping helpersdrivers/vhost/vdpa.c+23 -180 replies
4/21vhost-vdpa: switch to use vhost-vdpa specific IOTLBdrivers/vhost/vdpa.c+10 -80 replies
5/21vdpa: add the missing comment for nvqs in struct vdpa_deviceinclude/linux/vdpa.h+1 -00 replies
6/21vdpa: introduce virtqueue groupsdrivers/vdpa/ifcvf/ifcvf_main.c drivers/vdpa/mlx5/net/mlx5_vnet.c drivers/vdpa/vdpa.c +2 more+37 -72 replies
7/21vdpa: multiple address spaces supportdrivers/vdpa/ifcvf/ifcvf_main.c drivers/vdpa/mlx5/net/mlx5_vnet.c drivers/vdpa/vdpa.c +3 more+38 -204 replies
8/21vdpa: introduce config operations for associating ASID to a virtqueue groupinclude/linux/vdpa.h+10 -00 replies
9/21vhost_iotlb: split out IOTLB initializationdrivers/vhost/iotlb.c include/linux/vhost_iotlb.h+20 -50 replies
10/21vhost: support ASID in IOTLB APIdrivers/vhost/vdpa.c drivers/vhost/vhost.c drivers/vhost/vhost.h +1 more+28 -92 replies
11/21vhost-vdpa: introduce asid based IOTLBdrivers/vhost/vdpa.c+80 -266 replies
12/21vhost-vdpa: introduce uAPI to get the number of virtqueue groupsdrivers/vhost/vdpa.c include/uapi/linux/vhost.h+7 -04 replies
13/21vhost-vdpa: introduce uAPI to get the number of address spacesdrivers/vhost/vdpa.c include/uapi/linux/vhost.h+5 -00 replies
14/21vhost-vdpa: uAPI to get virtqueue group iddrivers/vhost/vdpa.c include/uapi/linux/vhost.h+16 -00 replies
15/21vhost-vdpa: introduce uAPI to set group ASIDdrivers/vhost/vdpa.c include/uapi/linux/vhost.h+15 -00 replies
16/21vhost-vdpa: support ASID based IOTLB APIdrivers/vhost/vdpa.c drivers/vhost/vhost.c+99 -320 replies
17/21vdpa_sim: split vdpasim_virtqueue's iov field in out_iov and in_iovdrivers/vdpa/vdpa_sim/vdpa_sim.c+6 -50 replies
18/21vdpa_sim: advertise VIRTIO_NET_F_MTUdrivers/vdpa/vdpa_sim/vdpa_sim.c+2 -10 replies
19/21vdpa_sim: factor out buffer completion logicdrivers/vdpa/vdpa_sim/vdpa_sim.c+18 -150 replies
20/21vdpa_sim: filter destination mac addressdrivers/vdpa/vdpa_sim/vdpa_sim.c+31 -180 replies
21/21vdpasim: control virtqueue supportdrivers/vdpa/vdpa_sim/vdpa_sim.c+166 -234 replies

Reviews trailers per patch

1 of 21 patches has a Reviewed-by or Acked-by.

PatchJason Wang
1/21
2/21
3/21
4/21
5/21
6/21
7/21
8/21
9/21
10/21
11/21
12/21
13/21
14/21
15/21
16/21
17/21A-b
18/21
19/21
20/21
21/21

Trailers given in replies are filled; those carried in the patches from earlier versions are outlined with dashes.

Read the v1 thread

Cover letter [PATCH 00/21] Control VQ support in vDPA

Hi All:

This series tries to add the support for control virtqueue in vDPA.

Control virtqueue is used by networking device for accepting various commands from the driver. It's a must to support multiqueue and other configurations.

When used by vhost-vDPA bus driver for VM, the control virtqueue should be shadowed via userspace VMM (Qemu) instead of being assigned directly to Guest. This is because Qemu needs to know the device state in order to start and stop device correctly (e.g for Live Migration).

This requies to isolate the memory mapping for control virtqueue presented by vhost-vDPA to prevent guest from accesing it directly.

To achieve this, vDPA introduce two new abstractions:

- address space: identified through address space id (ASID) and a set of memory mapping in maintained
- virtqueue group: the minimal set of virtqueues that must share an address space

Device needs to advertise the following attributes to vDPA:

- the number of address spaces supported in the device
- the number of virtqueue groups supported in the device
- the mappings from a specific virtqueue to its virtqueue groups

The mappings from virtqueue to virtqueue groups is fixed and defined by vDPA device driver. E.g:

- For the device that has hardware ASID support, it can simply advertise a per virtqueue virtqueue group.
- For the device that does not have hardware ASID support, it can simply advertise a single virtqueue group that contains all virtqueues. Or if it wants a software emulated control virtqueue, it can advertise two virtqueue groups, one is for cvq, another is for the rest virtqueues.

vDPA also allow to change the association between virtqueue group and address space. So in the case of control virtqueue, userspace VMM(Qemu) may use a dedicated address space for the control virtqueue group to isolate the memory mapping.

The vhost/vhost-vDPA is also extend for the userspace to:

- query the number of virtqueue groups and address spaces supported by the device
- query the virtqueue group for a specific virtqueue
- assocaite a virtqueue group with an address space
- send ASID based IOTLB commands

This will help userspace VMM(Qemu) to detect whether the control vq could be supported and isolate memory mappings of control virtqueue from the others.

To demonstrate the usage, vDPA simulator is extended to support setting MAC address via a emulated control virtqueue.

Please review.

Changes since RFC:

- tweak vhost uAPI documentation
- switch to use device specific IOTLB really in patch 4
- tweak the commit log
- fix that ASID in vhost is claimed to be 32 actually but 16bit actually
- fix use after free when using ASID with IOTLB batching requests
- switch to use Stefano's patch for having separated iov
- remove unused "used_as" variable
- fix the iotlb/asid checking in vhost_vdpa_unmap()

Thanks

Jason Wang (20):
vhost: move the backend feature bits to vhost_types.h
virtio-vdpa: don't set callback if virtio doesn't need it
vhost-vdpa: passing iotlb to IOMMU mapping helpers
vhost-vdpa: switch to use vhost-vdpa specific IOTLB
vdpa: add the missing comment for nvqs in struct vdpa_device
vdpa: introduce virtqueue groups
vdpa: multiple address spaces support
vdpa: introduce config operations for associating ASID to a virtqueue
group
vhost_iotlb: split out IOTLB initialization
vhost: support ASID in IOTLB API
vhost-vdpa: introduce asid based IOTLB
vhost-vdpa: introduce uAPI to get the number of virtqueue groups
vhost-vdpa: introduce uAPI to get the number of address spaces
vhost-vdpa: uAPI to get virtqueue group id
vhost-vdpa: introduce uAPI to set group ASID
vhost-vdpa: support ASID based IOTLB API
vdpa_sim: advertise VIRTIO_NET_F_MTU
vdpa_sim: factor out buffer completion logic
vdpa_sim: filter destination mac address
vdpasim: control virtqueue support

Stefano Garzarella (1):
vdpa_sim: split vdpasim_virtqueue's iov field in out_iov and in_iov

 drivers/vdpa/ifcvf/ifcvf_main.c   |   9 +-
 drivers/vdpa/mlx5/net/mlx5_vnet.c |  11 +-
 drivers/vdpa/vdpa.c               |   8 +-
 drivers/vdpa/vdpa_sim/vdpa_sim.c  | 292 ++++++++++++++++++++++++------
 drivers/vhost/iotlb.c             |  23 ++-
 drivers/vhost/vdpa.c              | 246 ++++++++++++++++++++-----
 drivers/vhost/vhost.c             |  23 ++-
 drivers/vhost/vhost.h             |   4 +-
 drivers/virtio/virtio_vdpa.c      |   2 +-
 include/linux/vdpa.h              |  42 ++++-
 include/linux/vhost_iotlb.h       |   2 +
 include/uapi/linux/vhost.h        |  25 ++-
 include/uapi/linux/vhost_types.h  |  10 +-
 13 files changed, 561 insertions(+), 136 deletions(-)
Signature · 6 lines
2.25.1

_______________________________________________
Virtualization mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/virtualization