vDPA support in qemu
v3 cover vDPA support in qemu
vDPA device is a device that uses a datapath which complies with the virtio specifications with vendor specific control path. vDPA devices can be both physically located on the hardware or emulated by software. This PATCH introduce the vDPA support in qemu
TODO
1) vIOMMU support
2) live migration support
3) docs for vhost-vdpa
4) config interrupt support
Change from v1
separate the patch of vhost_vq_get_addr separate the patch of vhost_dev_start introduce the docmation for vhost-vdpa.rst other comments form last version
github address:
https://github.com/lulu-github-name/qemutmp.git PATCHV2
Change from v2
fix the complie problem
separate the patch of vhost_force_iommu
other comments form last version
github address:
https://github.com/lulu-github-name/qemutmp.git PATCHV3
Cindy Lu (10):
net: introduce qemu_get_peer
vhost_net: use the function qemu_get_peer
vhost: introduce new VhostOps vhost_dev_start
vhost: implement vhost_dev_start method
vhost: introduce new VhostOps vhost_vq_get_addr
vhost: implement vhost_vq_get_addr method
vhost: introduce new VhostOps vhost_force_iommu
vhost: implement vhost_force_iommu method
vhost-vdpa: introduce vhost-vdpa backend
vhost-vdpa: introduce vhost-vdpa net client
Jason Wang (3):
virtio-bus: introduce queue_enabled method
virtio-pci: implement queue_enabled method
vhost: check the existence of vhost_set_iotlb_callback
configure | 21 ++ docs/interop/index.rst | 1 + docs/interop/vhost-vdpa.rst | 17 ++ hw/net/vhost_net.c | 35 ++- hw/net/virtio-net.c | 20 ++ hw/virtio/Makefile.objs | 1 + hw/virtio/vhost-backend.c | 6 + hw/virtio/vhost-vdpa.c | 475 ++++++++++++++++++++++++++++++ hw/virtio/vhost.c | 52 +++- hw/virtio/virtio-pci.c | 13 + hw/virtio/virtio.c | 6 + include/hw/virtio/vhost-backend.h | 19 +- include/hw/virtio/vhost-vdpa.h | 26 ++ include/hw/virtio/vhost.h | 7 + include/hw/virtio/virtio-bus.h | 4 + include/net/net.h | 1 + include/net/vhost-vdpa.h | 22 ++ net/Makefile.objs | 2 +- net/clients.h | 2 + net/net.c | 10 + net/vhost-vdpa.c | 228 ++++++++++++++ qapi/net.json | 28 +- qemu-options.hx | 12 + 23 files changed, 978 insertions(+), 30 deletions(-) create mode 100644 docs/interop/vhost-vdpa.rst create mode 100644 hw/virtio/vhost-vdpa.c create mode 100644 include/hw/virtio/vhost-vdpa.h create mode 100644 include/net/vhost-vdpa.h create mode 100644 net/vhost-vdpa.c
Signature · 1 line
2.21.1
PATCH v3 1/13 net: introduce qemu_get_peer
This is a small function that can get the peer
from given NetClientState and queue_index
--- include/net/net.h | 1 + net/net.c | 7 +++++++ 2 files changed, 8 insertions(+)
include/net/net.h +1 -0
@@ -176,6 +176,7 @@ void hmp_info_network(Monitor *mon, const QDict *qdict);void net_socket_rs_init (SocketReadState *rs,SocketReadStateFinalize *finalize,bool vnet_hdr); +NetClientState *qemu_get_peer (NetClientState *nc,int queue_index);/* NIC info */
net/net.c +7 -0
@@ -325,6 +325,13 @@ void *qemu_get_nic_opaque(NetClientState *nc)return nic->opaque; } +NetClientState *qemu_get_peer (NetClientState *nc,int queue_index) +{ +assert (nc !=NULL ); +NetClientState *ncs = nc + queue_index; +return ncs->peer; +} +static void qemu_cleanup_net_client (NetClientState *nc) {QTAILQ_REMOVE (&net_clients, nc, next);
Signature · 1 line
2.21.1
PATCH v3 2/13 vhost_net: use the function qemu_get_peer
In reply to Cindy Luuser the qemu_get_peer to replace the old process
--- hw/net/vhost_net.c | 16 ++++++++++------ 1 file changed, 10 insertions(+), 6 deletions(-)
hw/net/vhost_net.c +10 -6
@@ -306,7 +306,9 @@ int vhost_net_start(VirtIODevice *dev, NetClientState *ncs,BusState *qbus =BUS (qdev_get_parent_bus (DEVICE (dev)));VirtioBusState *vbus =VIRTIO_BUS (qbus);VirtioBusClass *k =VIRTIO_BUS_GET_CLASS (vbus); +struct vhost_net *net;int r, e, i; +NetClientState *peer;if (!k->set_guest_notifiers) {error_report ("binding does not support guest notifiers" );
@@ -314,9 +316,9 @@ int vhost_net_start(VirtIODevice *dev, NetClientState *ncs, }for (i =0 ; i < total_queues; i++) { -struct vhost_net *net; - net =get_vhost_net (ncs[i].peer); + peer =qemu_get_peer (ncs, i); + net =get_vhost_net (peer);vhost_net_set_vq_index (net, i *2 );/* Suppress the masking guest notifiers on vhost user
@@ -335,15 +337,16 @@ int vhost_net_start(VirtIODevice *dev, NetClientState *ncs, }for (i =0 ; i < total_queues; i++) { - r =vhost_net_start_one (get_vhost_net (ncs[i].peer), dev); + peer =qemu_get_peer (ncs, i); + r =vhost_net_start_one (get_vhost_net (peer), dev);if (r <0 ) { goto err_start; } -if (ncs[i].peer->vring_enable) { +if (peer->vring_enable) {/* restore vring enable state */ - r =vhost_set_vring_enable (ncs[i].peer, ncs[i].peer->vring_enable); + r =vhost_set_vring_enable (peer, peer->vring_enable);if (r <0 ) { goto err_start;
@@ -355,7 +358,8 @@ int vhost_net_start(VirtIODevice *dev, NetClientState *ncs, err_start:while (--i >=0 ) { -vhost_net_stop_one (get_vhost_net (ncs[i].peer), dev); + peer =qemu_get_peer (ncs , i); +vhost_net_stop_one (get_vhost_net (peer), dev); } e = k->set_guest_notifiers (qbus->parent, total_queues *2 , false);if (e <0 ) {
Signature · 1 line
2.21.1
PATCH v3 3/13 virtio-bus: introduce queue_enabled method
In reply to Cindy LuFrom: Jason Wang <[email protected]>
This patch introduces queue_enabled() method which allows the transport to implement its own way to report whether or not a queue is enabled.
--- hw/virtio/virtio.c | 6 ++++++ include/hw/virtio/virtio-bus.h | 4 ++++ 2 files changed, 10 insertions(+)
hw/virtio/virtio.c +6 -0
@@ -3286,6 +3286,12 @@ hwaddr virtio_queue_get_desc_addr(VirtIODevice *vdev, int n)bool virtio_queue_enabled (VirtIODevice *vdev,int n) { +BusState *qbus =qdev_get_parent_bus (DEVICE (vdev)); +VirtioBusClass *k =VIRTIO_BUS_GET_CLASS (qbus); + +if (k->queue_enabled) { +return k->queue_enabled (qbus->parent, n); + }return virtio_queue_get_desc_addr (vdev, n) !=0 ; }
include/hw/virtio/virtio-bus.h +4 -0
@@ -83,6 +83,10 @@ typedef struct VirtioBusClass {*/ int (*ioeventfd_assign)(DeviceState *d,EventNotifier *notifier,int n,bool assign); +/* +* Whether queue number n is enabled. +*/ +bool (*queue_enabled)(DeviceState *d,int n);/* * Does the transport have variable vring alignment? * (ie can it ever call virtio_queue_set_align()?)
Signature · 1 line
2.21.1
PATCH v3 4/13 virtio-pci: implement queue_enabled method
In reply to Cindy LuFrom: Jason Wang <[email protected]>
With version 1, we can detect whether a queue is enabled via queue_enabled.
--- hw/virtio/virtio-pci.c | 13 +++++++++++++ 1 file changed, 13 insertions(+)
hw/virtio/virtio-pci.c +13 -0
@@ -1107,6 +1107,18 @@ static AddressSpace *virtio_pci_get_dma_as(DeviceState *d)return pci_get_address_space (dev); } +static bool virtio_pci_queue_enabled (DeviceState *d,int n) +{ +VirtIOPCIProxy *proxy =VIRTIO_PCI (d); +VirtIODevice *vdev =virtio_bus_get_device (&proxy->bus); + +if (virtio_vdev_has_feature (vdev,VIRTIO_F_VERSION_1 )) { +return proxy->vqs[vdev->queue_sel].enabled; + } + +return virtio_queue_enabled (vdev, n); +} +static int virtio_pci_add_mem_cap (VirtIOPCIProxy *proxy,struct virtio_pci_cap *cap) {
@@ -2064,6 +2076,7 @@ static void virtio_pci_bus_class_init(ObjectClass *klass, void *data) k->ioeventfd_enabled = virtio_pci_ioeventfd_enabled; k->ioeventfd_assign = virtio_pci_ioeventfd_assign; k->get_dma_as = virtio_pci_get_dma_as; + k->queue_enabled = virtio_pci_queue_enabled; }static const TypeInfo virtio_pci_bus_info = {
Signature · 1 line
2.21.1
PATCH v3 5/13 vhost: check the existence of vhost_set_iotlb_callback
In reply to Cindy LuFrom: Jason Wang <[email protected]>
Add the check of vhost_set_iotlb_callback
before calling
--- hw/virtio/vhost.c | 9 ++++++--- 1 file changed, 6 insertions(+), 3 deletions(-)
hw/virtio/vhost.c +6 -3
@@ -1686,8 +1686,9 @@ int vhost_dev_start(struct vhost_dev *hdev, VirtIODevice *vdev) } } -if (vhost_dev_has_iommu (hdev)) { - hdev->vhost_ops->vhost_set_iotlb_callback (hdev, true); +if (vhost_dev_has_iommu (hdev) && + hdev->vhost_ops->vhost_set_iotlb_callback) { + hdev->vhost_ops->vhost_set_iotlb_callback (hdev, true);/* Update used ring information for IOTLB to work correctly, * vhost-kernel code requires for this.*/
@@ -1730,7 +1731,9 @@ void vhost_dev_stop(struct vhost_dev *hdev, VirtIODevice *vdev) }if (vhost_dev_has_iommu (hdev)) { - hdev->vhost_ops->vhost_set_iotlb_callback (hdev, false); +if (hdev->vhost_ops->vhost_set_iotlb_callback) { + hdev->vhost_ops->vhost_set_iotlb_callback (hdev, false); + }memory_listener_unregister (&hdev->iommu_listener); }vhost_log_put (hdev, true);
Signature · 1 line
2.21.1
PATCH v3 6/13 vhost: introduce new VhostOps vhost_dev_start
In reply to Cindy LuThis patch introduces new VhostOps vhost_dev_start callback which allows the vhost_net set the start/stop status to backend
--- include/hw/virtio/vhost-backend.h | 2 ++ 1 file changed, 2 insertions(+)
include/hw/virtio/vhost-backend.h +2 -0
@@ -112,6 +112,7 @@ typedef int (*vhost_get_inflight_fd_op)(struct vhost_dev *dev,typedef int (*vhost_set_inflight_fd_op )(struct vhost_dev *dev,struct vhost_inflight *inflight); +typedef int (*vhost_dev_start_op )(struct vhost_dev *dev,bool started); typedef struct VhostOps { VhostBackendType backend_type; vhost_backend_init vhost_backend_init;
@@ -152,6 +153,7 @@ typedef struct VhostOps {vhost_backend_mem_section_filter_op vhost_backend_mem_section_filter;vhost_get_inflight_fd_op vhost_get_inflight_fd;vhost_set_inflight_fd_op vhost_set_inflight_fd; +vhost_dev_start_op vhost_dev_start; } VhostOps;extern const VhostOps user_ops;
Signature · 1 line
2.21.1
PATCH v3 7/13 vhost: implement vhost_dev_start method
In reply to Cindy Luuse the vhost_dev_start callback to send the status to backend
--- hw/virtio/vhost.c | 10 +++++++++- 1 file changed, 9 insertions(+), 1 deletion(-)
hw/virtio/vhost.c +9 -1
@@ -1685,7 +1685,12 @@ int vhost_dev_start(struct vhost_dev *hdev, VirtIODevice *vdev) goto fail_log; } } - +if (hdev->vhost_ops->vhost_dev_start) { + r = hdev->vhost_ops->vhost_dev_start (hdev, true); +if (r) { + goto fail_log; + } + }if (vhost_dev_has_iommu (hdev) && hdev->vhost_ops->vhost_set_iotlb_callback) { hdev->vhost_ops->vhost_set_iotlb_callback (hdev, true);
@@ -1723,6 +1728,9 @@ void vhost_dev_stop(struct vhost_dev *hdev, VirtIODevice *vdev)/* should only be called after backend is connected */ assert (hdev->vhost_ops); +if (hdev->vhost_ops->vhost_dev_start) { + hdev->vhost_ops->vhost_dev_start (hdev, false); + }for (i =0 ; i < hdev->nvqs; ++i) { vhost_virtqueue_stop(hdev, vdev,
Signature · 1 line
2.21.1
PATCH v3 8/13 vhost: introduce new VhostOps vhost_vq_get_addr
In reply to Cindy LuThis patch introduces new VhostOps vhost_vq_get_addr_op callback to get the vring addr from the backend
--- include/hw/virtio/vhost-backend.h | 6 ++++++ 1 file changed, 6 insertions(+)
include/hw/virtio/vhost-backend.h +6 -0
@@ -34,6 +34,7 @@ struct vhost_vring_state;struct vhost_vring_addr ;struct vhost_scsi_target ;struct vhost_iotlb_msg ; +struct vhost_virtqueue ;typedef int (*vhost_backend_init )(struct vhost_dev *dev,void *opaque);typedef int (*vhost_backend_cleanup )(struct vhost_dev *dev);
@@ -113,6 +114,10 @@ typedef int (*vhost_set_inflight_fd_op)(struct vhost_dev *dev, struct vhost_inflight *inflight);typedef int (*vhost_dev_start_op )(struct vhost_dev *dev,bool started); + +typedef int (*vhost_vq_get_addr_op )(struct vhost_dev *dev, +struct vhost_vring_addr *addr, +struct vhost_virtqueue *vq); typedef struct VhostOps { VhostBackendType backend_type; vhost_backend_init vhost_backend_init;
@@ -154,6 +159,7 @@ typedef struct VhostOps {vhost_get_inflight_fd_op vhost_get_inflight_fd;vhost_set_inflight_fd_op vhost_set_inflight_fd;vhost_dev_start_op vhost_dev_start; +vhost_vq_get_addr_op vhost_vq_get_addr; } VhostOps;extern const VhostOps user_ops;
Signature · 1 line
2.21.1
PATCH v3 9/13 vhost: implement vhost_vq_get_addr method
In reply to Cindy Luuse vhost_vq_get_addr callback to get the vq address from backend
--- hw/virtio/vhost.c | 28 +++++++++++++++++++--------- include/hw/virtio/vhost-backend.h | 4 ++++ 2 files changed, 23 insertions(+), 9 deletions(-)
hw/virtio/vhost.c +19 -9
@@ -773,15 +773,25 @@ static int vhost_virtqueue_set_addr(struct vhost_dev *dev,struct vhost_virtqueue *vq, unsigned idx, bool enable_log) { - struct vhost_vring_addr addr = { - .index = idx, - .desc_user_addr = (uint64_t )(unsigned long )vq->desc, - .avail_user_addr = (uint64_t )(unsigned long )vq->avail, - .used_user_addr = (uint64_t )(unsigned long )vq->used, - .log_guest_addr = vq->used_phys, - .flags = enable_log ? (1 <<VHOST_VRING_F_LOG ) :0 , - }; -int r = dev->vhost_ops->vhost_set_vring_addr (dev, &addr); + struct vhost_vring_addr addr; +int r; +memset (&addr,0 ,sizeof (struct vhost_vring_addr )); + +if (dev->vhost_ops->vhost_vq_get_addr) { + r = dev->vhost_ops->vhost_vq_get_addr (dev, &addr, vq); +if (r <0 ) { +VHOST_OPS_DEBUG ("vhost_vq_get_addr failed" ); +return -errno; + } + }else { + addr.desc_user_addr = (uint64_t )(unsigned long )vq->desc; + addr.avail_user_addr = (uint64_t )(unsigned long )vq->avail; + addr.used_user_addr = (uint64_t )(unsigned long )vq->used; + } + addr.index = idx; + addr.log_guest_addr = vq->used_phys; + addr.flags = enable_log ? (1 <<VHOST_VRING_F_LOG ) :0 ; + r = dev->vhost_ops->vhost_set_vring_addr (dev, &addr);if (r <0 ) {VHOST_OPS_DEBUG ("vhost_set_vring_addr failed" );return -errno;
include/hw/virtio/vhost-backend.h +4 -0
@@ -118,6 +118,9 @@ typedef int (*vhost_dev_start_op)(struct vhost_dev *dev, bool started);typedef int (*vhost_vq_get_addr_op )(struct vhost_dev *dev,struct vhost_vring_addr *addr,struct vhost_virtqueue *vq); + +typedef int (*vhost_get_device_id_op )(struct vhost_dev *dev,uint32_t *dev_id); + typedef struct VhostOps { VhostBackendType backend_type; vhost_backend_init vhost_backend_init;
@@ -160,6 +163,7 @@ typedef struct VhostOps {vhost_set_inflight_fd_op vhost_set_inflight_fd;vhost_dev_start_op vhost_dev_start;vhost_vq_get_addr_op vhost_vq_get_addr; +vhost_get_device_id_op vhost_get_device_id; } VhostOps;extern const VhostOps user_ops;
Signature · 1 line
2.21.1
PATCH v3 10/13 vhost: introduce new VhostOps vhost_force_iommu
In reply to Cindy LuThis patch introduces new VhostOps vhost_force_iommu callback to force enable features bit VIRTIO_F_IOMMU_PLATFORM.
--- include/hw/virtio/vhost-backend.h | 3 +++ 1 file changed, 3 insertions(+)
include/hw/virtio/vhost-backend.h +3 -0
@@ -121,6 +121,8 @@ typedef int (*vhost_vq_get_addr_op)(struct vhost_dev *dev,typedef int (*vhost_get_device_id_op )(struct vhost_dev *dev,uint32_t *dev_id); +typedef bool (*vhost_force_iommu_op )(struct vhost_dev *dev); + typedef struct VhostOps { VhostBackendType backend_type; vhost_backend_init vhost_backend_init;
@@ -164,6 +166,7 @@ typedef struct VhostOps {vhost_dev_start_op vhost_dev_start;vhost_vq_get_addr_op vhost_vq_get_addr;vhost_get_device_id_op vhost_get_device_id; +vhost_force_iommu_op vhost_force_iommu; } VhostOps;extern const VhostOps user_ops;
Signature · 1 line
2.21.1
PATCH v3 11/13 vhost: implement vhost_force_iommu method
In reply to Cindy Luuse the vhost_force_iommu callback to force enable feature bit VIRTIO_F_IOMMU_PLATFORM
--- hw/virtio/vhost.c | 5 +++++ 1 file changed, 5 insertions(+)
hw/virtio/vhost.c +5 -0
@@ -810,6 +810,11 @@ static int vhost_dev_set_features(struct vhost_dev *dev,if (!vhost_dev_has_iommu (dev)) { features &= ~(0x1ULL <<VIRTIO_F_IOMMU_PLATFORM ); } +if (dev->vhost_ops->vhost_force_iommu) { +if (dev->vhost_ops->vhost_force_iommu (dev) == true) { + features |=0x1ULL <<VIRTIO_F_IOMMU_PLATFORM ; + } + } r = dev->vhost_ops->vhost_set_features (dev, features);if (r <0 ) {VHOST_OPS_DEBUG ("vhost_set_features failed" );
Signature · 1 line
2.21.1
PATCH v3 12/13 vhost-vdpa: introduce vhost-vdpa backend
In reply to Cindy LuCurrently we have 2 types of vhost backends in QEMU: vhost kernel and vhost-user. The above patch provides a generic device for vDPA purpose, this vDPA device exposes to user space a non-vendor-specific configuration interface for setting up a vhost HW accelerator, this patch set introduces a third vhost backend called vhost-vdpa based on the vDPA interface.
Vhost-vdpa usage:
qemu-system-x86_64 -cpu host -enable-kvm \
......
-netdev type=vhost-vdpa,vhostdev=/dev/vhost-vdpa-id,id=vhost-vdpa0 \
-device virtio-net-pci,netdev=vhost-vdpa0,page-per-vq=on \--- configure | 21 ++ docs/interop/index.rst | 1 + docs/interop/vhost-vdpa.rst | 17 ++ hw/net/vhost_net.c | 19 +- hw/net/virtio-net.c | 20 ++ hw/virtio/Makefile.objs | 1 + hw/virtio/vhost-backend.c | 6 + hw/virtio/vhost-vdpa.c | 475 ++++++++++++++++++++++++++++++ include/hw/virtio/vhost-backend.h | 4 +- include/hw/virtio/vhost-vdpa.h | 26 ++ include/hw/virtio/vhost.h | 7 + qemu-options.hx | 12 + 12 files changed, 602 insertions(+), 7 deletions(-) create mode 100644 docs/interop/vhost-vdpa.rst create mode 100644 hw/virtio/vhost-vdpa.c create mode 100644 include/hw/virtio/vhost-vdpa.h
configure +21 -0
@@ -1575,6 +1575,10 @@ for opt do ;; --enable-vhost-user) vhost_user="yes" ;; + --disable-vhost-vdpa) vhost_vdpa="no" + ;; + --enable-vhost-vdpa) vhost_vdpa="yes" + ;; --disable-vhost-kernel) vhost_kernel="no" ;; --enable-vhost-kernel) vhost_kernel="yes"
@@ -1883,6 +1887,7 @@ disabled with --disable-FEATURE, default is enabled if available: vhost-crypto vhost-user-crypto backend support vhost-kernel vhost kernel backend support vhost-user vhost-user backend support + vhost-vdpa vhost-vdpa kernel backend support spice spice rbd rados block device (rbd) libiscsi iscsi support
@@ -2394,6 +2399,10 @@ test "$vhost_user" = "" && vhost_user=yes if test "$vhost_user" = "yes" && test "$mingw32" = "yes"; then error_exit "vhost-user isn't available on win32" fi +test "$vhost_vdpa" = "" && vhost_vdpa=$linux +if test "$vhost_vdpa" = "yes" && test "$linux" != "yes"; then + error_exit "vhost-vdpa is only available on Linux" +fi test "$vhost_kernel" = "" && vhost_kernel=$linux if test "$vhost_kernel" = "yes" && test "$linux" != "yes"; then error_exit "vhost-kernel is only available on Linux"
@@ -2422,6 +2431,11 @@ test "$vhost_user_fs" = "" && vhost_user_fs=$vhost_user if test "$vhost_user_fs" = "yes" && test "$vhost_user" = "no"; then error_exit "--enable-vhost-user-fs requires --enable-vhost-user" fi +#vhost-vdpa backends +test "$vhost_net_vdpa" = "" && vhost_net_vdpa=$vhost_vdpa +if test "$vhost_net_vdpa" = "yes" && test "$vhost_vdpa" = "no"; then + error_exit "--enable-vhost-net-vdpa requires --enable-vhost-vdpa" +fi # OR the vhost-kernel and vhost-user values for simplicity if test "$vhost_net" = ""; then
@@ -6936,6 +6950,7 @@ echo "vhost-scsi support $vhost_scsi" echo "vhost-vsock support $vhost_vsock" echo "vhost-user support $vhost_user" echo "vhost-user-fs support $vhost_user_fs" +echo "vhost-vdpa support $vhost_vdpa" echo "Trace backends $trace_backends" if have_backend "simple"; then echo "Trace output file $trace_file-<pid>"
@@ -7437,6 +7452,9 @@ fi if test "$vhost_net_user" = "yes" ; then echo "CONFIG_VHOST_NET_USER=y" >> $config_host_mak fi +if test "$vhost_net_vdpa" = "yes" ; then + echo "CONFIG_VHOST_NET_VDPA=y" >> $config_host_mak +fi if test "$vhost_crypto" = "yes" ; then echo "CONFIG_VHOST_CRYPTO=y" >> $config_host_mak fi
@@ -7452,6 +7470,9 @@ fi if test "$vhost_user" = "yes" ; then echo "CONFIG_VHOST_USER=y" >> $config_host_mak fi +if test "$vhost_vdpa" = "yes" ; then + echo "CONFIG_VHOST_VDPA=y" >> $config_host_mak +fi if test "$vhost_user_fs" = "yes" ; then echo "CONFIG_VHOST_USER_FS=y" >> $config_host_mak fi
docs/interop/index.rst +1 -0
@@ -20,3 +20,4 @@ Contents: qemu-ga vhost-user vhost-user-gpu + vhost-vdpa
docs/interop/vhost-vdpa.rst new file +17 -0
@@ -0,0 +1,17 @@ +===================== +Vhost-vdpa Protocol +===================== + +Introduction +============= +vDPA(Virtual data path acceleration) device is a device that uses +a datapath which complies with the virtio specifications with vendor +specific control path. vDPA devices can be both physically located on +the hardware or emulated by software. + +This document describes the vDPA support in qemu + +Here is the kernel commit here +https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=4c8cf31885f69e86be0b5b9e6677a26797365e1d + +TODO : More information will add later
hw/net/vhost_net.c +13 -6
@@ -17,8 +17,10 @@#include "net/net.h" #include "net/tap.h" #include "net/vhost-user.h" +#include "net/vhost-vdpa.h" #include "standard-headers/linux/vhost_types.h" +#include "linux-headers/linux/vhost.h" #include "hw/virtio/virtio-net.h" #include "net/vhost_net.h" #include "qemu/error-report.h"
@@ -33,12 +35,6 @@#include "hw/virtio/vhost.h" #include "hw/virtio/virtio-bus.h" -struct vhost_net { -struct vhost_dev dev; -struct vhost_virtqueue vqs[2 ]; -int backend; -NetClientState *nc; -};/* Features supported by host kernel. */ static const int kernel_feature_bits[] = {
@@ -96,6 +92,11 @@ static const int *vhost_net_get_feature_bits(struct vhost_net *net)case NET_CLIENT_DRIVER_VHOST_USER : feature_bits = user_feature_bits;break ; +#ifdef CONFIG_VHOST_NET_VDPA +case NET_CLIENT_DRIVER_VHOST_VDPA : + feature_bits = vdpa_feature_bits; +break ; +#endif default :error_report ("Feature bits not defined for this type: %d" , net->nc->info->type);
@@ -433,6 +434,12 @@ VHostNetState *get_vhost_net(NetClientState *nc) vhost_net =vhost_user_get_vhost_net (nc);assert (vhost_net);break ; +#endif +#ifdef CONFIG_VHOST_NET_VDPA +case NET_CLIENT_DRIVER_VHOST_VDPA : + vhost_net =vhost_vdpa_get_vhost_net (nc); +assert (vhost_net); +break ;#endif default :break ;
hw/net/virtio-net.c +20 -0
@@ -43,6 +43,7 @@#include "monitor/qdev.h" #include "hw/pci/pci.h" #include "net_rx_pkt.h" +#include "hw/virtio/vhost.h" #define VIRTIO_NET_VM_VERSION 11
@@ -125,6 +126,7 @@ static void virtio_net_get_config(VirtIODevice *vdev, uint8_t *config)VirtIONet *n =VIRTIO_NET (vdev);struct virtio_net_config netcfg; +memset (&netcfg,0 ,sizeof (struct virtio_net_config ));virtio_stw_p (vdev, &netcfg.status, n->status);virtio_stw_p (vdev, &netcfg.max_virtqueue_pairs, n->max_queues);virtio_stw_p (vdev, &netcfg.mtu, n->net_conf.mtu);
@@ -138,6 +140,17 @@ static void virtio_net_get_config(VirtIODevice *vdev, uint8_t *config)virtio_stl_p (vdev, &netcfg.supported_hash_types,VIRTIO_NET_RSS_SUPPORTED_HASHES );memcpy (config, &netcfg, n->config_size); + +#ifdef CONFIG_VHOST_NET_VDPA +struct vhost_dev *hdev; +int ret; +NetClientState *nc =qemu_get_queue (n->nic); + hdev = &(get_vhost_net (nc->peer)->dev); + ret =vhost_dev_get_config (hdev, (uint8_t *)&netcfg, n->config_size); +if (ret !=-1 ) { +memcpy (config, &netcfg, n->config_size); + } +#endif }static void virtio_net_set_config (VirtIODevice *vdev,const uint8_t *config)
@@ -153,6 +166,13 @@ static void virtio_net_set_config(VirtIODevice *vdev, const uint8_t *config)memcpy (n->mac, netcfg.mac,ETH_ALEN );qemu_format_nic_info_str (qemu_get_queue (n->nic), n->mac); } +#ifdef CONFIG_VHOST_NET_VDPA +struct vhost_dev *hdev; +NetClientState *nc =qemu_get_queue (n->nic); + hdev = &(get_vhost_net (nc->peer)->dev); +vhost_dev_set_config (hdev, (uint8_t *)&netcfg,0 , n->config_size, +VHOST_SET_CONFIG_TYPE_MASTER ); +#endif }static bool virtio_net_started (VirtIONet *n,uint8_t status)
hw/virtio/Makefile.objs +1 -0
@@ -5,6 +5,7 @@ obj-y += virtio.o obj-$(CONFIG_VHOST) += vhost.o vhost-backend.o common-obj-$(call lnot,$(CONFIG_VHOST)) += vhost-stub.o obj-$(CONFIG_VHOST_USER) += vhost-user.o +obj-$(CONFIG_VHOST_VDPA) += vhost-vdpa.o common-obj-$(CONFIG_VIRTIO_RNG) += virtio-rng.o common-obj-$(CONFIG_VIRTIO_PCI) += virtio-pci.o
hw/virtio/vhost-backend.c +6 -0
@@ -15,6 +15,7 @@#include "qemu/main-loop.h" #include "standard-headers/linux/vhost_types.h" +#include "hw/virtio/vhost-vdpa.h" #ifdef CONFIG_VHOST_KERNEL #include <linux/vhost.h> #include <sys/ioctl.h>
@@ -285,6 +286,11 @@ int vhost_set_backend_type(struct vhost_dev *dev, VhostBackendType backend_type)case VHOST_BACKEND_TYPE_USER : dev->vhost_ops = &user_ops;break ; +#endif +#ifdef CONFIG_VHOST_VDPA +case VHOST_BACKEND_TYPE_VDPA : + dev->vhost_ops = &vdpa_ops; +break ;#endif default :error_report ("Unknown vhost backend type" );
hw/virtio/vhost-vdpa.c new file +475 -0
475 lines not on this page · raw message
include/hw/virtio/vhost-backend.h +3 -1
@@ -17,7 +17,8 @@ typedef enum VhostBackendType {VHOST_BACKEND_TYPE_NONE =0 ,VHOST_BACKEND_TYPE_KERNEL =1 ,VHOST_BACKEND_TYPE_USER =2 , -VHOST_BACKEND_TYPE_MAX =3 , +VHOST_BACKEND_TYPE_VDPA =3 , +VHOST_BACKEND_TYPE_MAX =4 , } VhostBackendType; typedef enum VhostSetConfigType {
@@ -170,6 +171,7 @@ typedef struct VhostOps { } VhostOps;extern const VhostOps user_ops; +extern const VhostOps vdpa_ops;int vhost_set_backend_type (struct vhost_dev *dev,VhostBackendType backend_type);
include/hw/virtio/vhost-vdpa.h new file +26 -0
@@ -0,0 +1,26 @@ +/* +* vhost-vdpa.h +* +* Copyright(c) 2017-2018 Intel Corporation. +* Copyright(c) 2020 Red Hat, Inc. +* +* This work is licensed under the terms of the GNU GPL, version 2 or later. +* See the COPYING file in the top-level directory. +* +*/ + +#ifndef HW_VIRTIO_VHOST_VDPA_H +#define HW_VIRTIO_VHOST_VDPA_H + +#include "hw/virtio/virtio.h" + +typedef struct vhost_vdpa { +int device_fd; +uint32_t msg_type; +MemoryListener listener; +}VhostVDPA ; + +extern AddressSpace address_space_memory; +extern int vhost_vdpa_get_device_id (struct vhost_dev *dev, +uint32_t *device_id); +#endif
include/hw/virtio/vhost.h +7 -0
@@ -92,6 +92,13 @@ struct vhost_dev {const VhostDevConfigOps *config_ops; }; +struct vhost_net { +struct vhost_dev dev; +struct vhost_virtqueue vqs[2 ]; +int backend; +NetClientState *nc; +}; +int vhost_dev_init (struct vhost_dev *hdev,void *opaque,VhostBackendType backend_type,uint32_t busyloop_timeout);
qemu-options.hx +12 -0
@@ -2418,6 +2418,10 @@ DEF("netdev", HAS_ARG, QEMU_OPTION_netdev, #ifdef CONFIG_POSIX "-netdev vhost-user,id=str,chardev=dev[,vhostforce=on|off]\n" " configure a vhost-user network, backed by a chardev 'dev'\n" +#endif +#ifdef __linux__ + "-netdev vhost-vdpa,id=str,vhostdev=/path/to/dev\n" + " configure a vhost-vdpa network,Establish a vhost-vdpa netdev\n" #endif "-netdev hubport,id=str,hubid=n[,netdev=nd]\n" " configure a hub port on the hub with ID 'n'\n", QEMU_ARCH_ALL)
@@ -2897,6 +2901,14 @@ SRST -netdev type=vhost-user,id=net0,chardev=chr0 \ -device virtio-net-pci,netdev=net0 +``-netdev vhost-vdpa,vhostdev=/path/to/dev`` + Establish a vhost-vdpa netdev. + + vDPA device is a device that uses a datapath which complies with + the virtio specifications with a vendor specific control path. + vDPA devices can be both physically located on the hardware or + emulated by software. + ``-netdev hubport,id=id,hubid=hubid[,netdev=nd]`` Create a hub port on the emulated hub with ID hubid.
Signature · 1 line
2.21.1
PATCH v3 13/13 vhost-vdpa: introduce vhost-vdpa net client
In reply to Cindy LuThis patch set introduces a new net client type: vhost-vdpa. vhost-vdpa net client will set up a vDPA device which is specified by a "vhostdev" parameter.
--- include/net/vhost-vdpa.h | 22 ++++ net/Makefile.objs | 2 +- net/clients.h | 2 + net/net.c | 3 + net/vhost-vdpa.c | 228 +++++++++++++++++++++++++++++++++++++++ qapi/net.json | 28 ++++- 6 files changed, 281 insertions(+), 4 deletions(-) create mode 100644 include/net/vhost-vdpa.h create mode 100644 net/vhost-vdpa.c
include/net/vhost-vdpa.h new file +22 -0
@@ -0,0 +1,22 @@ +/* +* vhost-vdpa.h +* +* Copyright(c) 2017-2018 Intel Corporation. +* Copyright(c) 2020 Red Hat, Inc. +* +* This work is licensed under the terms of the GNU GPL, version 2 or later. +* See the COPYING file in the top-level directory. +* +*/ + +#ifndef VHOST_VDPA_H +#define VHOST_VDPA_H + +#define TYPE_VHOST_VDPA "vhost-vdpa" + +struct vhost_net *vhost_vdpa_get_vhost_net (NetClientState *nc); +uint64_t vhost_vdpa_get_acked_features (NetClientState *nc); + +extern const int vdpa_feature_bits[]; + +#endif /* VHOST_VDPA_H */
net/Makefile.objs +1 -1
@@ -26,7 +26,7 @@ tap-obj-$(CONFIG_SOLARIS) = tap-solaris.o tap-obj-y ?= tap-stub.o common-obj-$(CONFIG_POSIX) += tap.o $(tap-obj-y) common-obj-$(CONFIG_WIN32) += tap-win32.o - +common-obj-$(CONFIG_VHOST_NET_VDPA) += vhost-vdpa.o vde.o-libs = $(VDE_LIBS) common-obj-$(CONFIG_CAN_BUS) += can/
net/clients.h +2 -0
@@ -61,4 +61,6 @@ int net_init_netmap(const Netdev *netdev, const char *name,int net_init_vhost_user (const Netdev *netdev,const char *name,NetClientState *peer,Error **errp); +int net_init_vhost_vdpa (const Netdev *netdev,const char *name, +NetClientState *peer,Error **errp);#endif /* QEMU_NET_CLIENTS_H */
net/net.c +3 -0
@@ -966,6 +966,9 @@ static int (* const net_client_init_fun[NET_CLIENT_DRIVER__MAX])( #ifdef CONFIG_VHOST_NET_USER [NET_CLIENT_DRIVER_VHOST_USER ] = net_init_vhost_user, #endif +#ifdef CONFIG_VHOST_NET_VDPA + [NET_CLIENT_DRIVER_VHOST_VDPA ] = net_init_vhost_vdpa, +#endif #ifdef CONFIG_L2TPV3 [NET_CLIENT_DRIVER_L2TPV3] = net_init_l2tpv3, #endif
net/vhost-vdpa.c new file +228 -0
228 lines not on this page · raw message
qapi/net.json +25 -3
@@ -428,16 +428,37 @@ '*vhostforce': 'bool', '*queues': 'int' } } +## +# @NetdevVhostVDPAOptions: +# +# Vhost-vdpa network backend +# +# vDPA device is a device that uses a datapath which complies with the virtio +# specifications with a vendor specific control path. +# +# @vhostdev: path of vhost-vdpa device +# (default:'/dev/vhost-vdpa-0') +# +# @queues: number of queues to be created for multiqueue vhost-vdpa +# (default: 1) +# +# Since: 5.1 +## +{ 'struct': 'NetdevVhostVDPAOptions', + 'data': { + '*vhostdev': 'str', + '*queues': 'int' } } + ## # @NetClientDriver: # # Available netdev drivers. # -# Since: 2.7 +# Since: 5.1 ## { 'enum': 'NetClientDriver', 'data': [ 'none', 'nic', 'user', 'tap', 'l2tpv3', 'socket', 'vde', - 'bridge', 'hubport', 'netmap', 'vhost-user' ] } + 'bridge', 'hubport', 'netmap', 'vhost-user', 'vhost-vdpa' ] } ## # @Netdev:
@@ -465,7 +486,8 @@ 'bridge': 'NetdevBridgeOptions', 'hubport': 'NetdevHubPortOptions', 'netmap': 'NetdevNetmapOptions', - 'vhost-user': 'NetdevVhostUserOptions' } } + 'vhost-user': 'NetdevVhostUserOptions', + 'vhost-vdpa': 'NetdevVhostVDPAOptions' } } ## # @NetFilterDirection:
Signature · 1 line
2.21.1
Re: [PATCH v3 13/13] vhost-vdpa: introduce vhost-vdpa net client
Quoted 17 lines · Cindy Lu
This patch set introduces a new net client type: vhost-vdpa. vhost-vdpa net client will set up a vDPA device which is specified by a "vhostdev" parameter.
Signed-off-by Lingshan Zhu <[email protected]>Signed-off-by Tiwei Bie <[email protected]>Signed-off-by Cindy Lu <[email protected]>Signed-off-by Jason Wang <[email protected]>---
+++
b/qapi/net.json##
# @NetClientDriver:
#
# Available netdev drivers.
#
qapi/net.json line 457
Quoted 2 lines
qapi/net.json +1
-1-# Since: 2.7 +# Since: 5.1
This should be:
# Since 2.7
# @vhost-vdpa since 5.1
since the enum itself is still available in older releases, it is only the new member that was introduced in this series.
Quoted 6 lines
##
{ 'enum': 'NetClientDriver','data': [ 'none', 'nic', 'user', 'tap', 'l2tpv3', 'socket', 'vde',qapi/net.json +1
-1- 'bridge', 'hubport', 'netmap', 'vhost-user' ] } + 'bridge', 'hubport', 'netmap', 'vhost-user', 'vhost-vdpa' ] }
Signature · 3 lines
Eric Blake, Principal Software Engineer Red Hat, Inc. +1-919-301-3226 Virtualization: qemu.org | libvirt.org
Re: [PATCH v3 13/13] vhost-vdpa: introduce vhost-vdpa net client
Quoted 23 lines · Eric Blake
Quoted 17 lines · Cindy Lu
This patch set introduces a new net client type: vhost-vdpa. vhost-vdpa net client will set up a vDPA device which is specified by a "vhostdev" parameter.
Signed-off-by Lingshan Zhu <[email protected]>Signed-off-by Tiwei Bie <[email protected]>Signed-off-by Cindy Lu <[email protected]>Signed-off-by Jason Wang <[email protected]>---
+++
b/qapi/net.json##
# @NetClientDriver:
#
# Available netdev drivers.
#qapi/net.json line 457
Quoted 2 lines
qapi/net.json +1
-1-# Since: 2.7 +# Since: 5.1This should be:
Quoted 12 lines
# Since 2.7
# @vhost-vdpa since 5.1since the enum itself is still available in older releases, it is only the new member that was introduced in this series.
Quoted 5 lines
##
{ 'enum': 'NetClientDriver','data': [ 'none', 'nic', 'user', 'tap', 'l2tpv3', 'socket', 'vde',qapi/net.json +1
-1- 'bridge', 'hubport', 'netmap', 'vhost-user' ] } + 'bridge', 'hubport', 'netmap', 'vhost-user', 'vhost-vdpa' ] }
Thanks! I will fix it up when applying.
Ack with that fixed?
Quoted 4 lines
Signature · 3 lines
Eric Blake, Principal Software Engineer Red Hat, Inc. +1-919-301-3226 Virtualization: qemu.org | libvirt.org
Re: [PATCH v3 13/13] vhost-vdpa: introduce vhost-vdpa net client
Hi Michael, Eric,
This was fix in the latest version, v4
https://patchew.org/QEMU/[email protected]/[email protected]/
Quoted 48 lines · Michael S. Tsirkin
Quoted 23 lines · Eric Blake
Quoted 17 lines · Cindy Lu
This patch set introduces a new net client type: vhost-vdpa. vhost-vdpa net client will set up a vDPA device which is specified by a "vhostdev" parameter.
Signed-off-by Lingshan Zhu <[email protected]>Signed-off-by Tiwei Bie <[email protected]>Signed-off-by Cindy Lu <[email protected]>Signed-off-by Jason Wang <[email protected]>---
+++
b/qapi/net.json##
# @NetClientDriver:
#
# Available netdev drivers.
#qapi/net.json line 457
Quoted 2 lines
qapi/net.json +1
-1-# Since: 2.7 +# Since: 5.1This should be:
Quoted 12 lines
# Since 2.7
# @vhost-vdpa since 5.1since the enum itself is still available in older releases, it is only the new member that was introduced in this series.
Quoted 5 lines
##
{ 'enum': 'NetClientDriver','data': [ 'none', 'nic', 'user', 'tap', 'l2tpv3', 'socket', 'vde',qapi/net.json +1
-1- 'bridge', 'hubport', 'netmap', 'vhost-user' ] } + 'bridge', 'hubport', 'netmap', 'vhost-user', 'vhost-vdpa' ] }Thanks! I will fix it up when applying.
Ack with that fixed?Quoted 4 lines
--
Eric Blake, Principal Software Engineer
Red Hat, Inc. +1-919-301-3226
Virtualization: qemu.org | libvirt.org
Patchew URL: https://patchew.org/QEMU/[email protected]/
Patchew URL: https://patchew.org/QEMU/[email protected]/
Hi,
This series failed the docker-quick@centos7 build test. Please find the testing commands and their output below. If you have Docker installed, you can probably reproduce it locally.
=== TEST SCRIPT BEGIN ===
#!/bin/bash
make docker-image-centos7 V=1 NETWORK=1
time make docker-test-quick@centos7 SHOW_ENV=1 J=14 NETWORK=1 === TEST SCRIPT END ===
TEST iotest-qcow2: 039
Broken pipe/tmp/qemu-test/src/tests/qtest/libqtest.c:175: kill_qemu() detected QEMU death from signal 11 (Segmentation fault) (core dumped)
ERROR - too few tests run (expected 4, got 1)
make: *** [check-qtest-x86_64] Error 1
make: *** Waiting for unfinished jobs....
qemu-system-aarch64: -accel kvm: invalid accelerator kvm
qemu-system-aarch64: falling back to tcg
---
raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['sudo', '-n', 'docker', 'run', '--label', 'com.qemu.instance.uuid=63e80a61b9a949878e719ebcc62e165b', '-u', '1003', '--security-opt', 'seccomp=unconfined', '--rm', '-e', 'TARGET_LIST=', '-e', 'EXTRA_CONFIGURE_OPTS=', '-e', 'V=', '-e', 'J=14', '-e', 'DEBUG=', '-e', 'SHOW_ENV=1', '-e', 'CCACHE_DIR=/var/tmp/ccache', '-v', '/home/patchew2/.cache/qemu-docker-ccache:/var/tmp/ccache:z', '-v', '/var/tmp/patchew-tester-tmp-vriwkat5/src/docker-src.2020-07-01-05.38.01.23726:/var/tmp/qemu:z,ro', 'qemu:centos7', '/var/tmp/qemu/run', 'test-quick']' returned non-zero exit status 2. filter=--filter=label=com.qemu.instance.uuid=63e80a61b9a949878e719ebcc62e165b
make[1]: *** [docker-run] Error 1
make[1]: Leaving directory `/var/tmp/patchew-tester-tmp-vriwkat5/src'
make: *** [docker-run-test-quick@centos7] Error 2
real 15m51.159s
user 0m9.070s
The full log is available at
http://patchew.org/logs/[email protected]/testing.docker-quick@centos7/?type=message.
---
Email generated automatically by Patchew [https://patchew.org/].
Please send your feedback to [email protected]