VDPA support for Mellanox ConnectX devices
Subsystems (4)
v4 cover VDPA support for Mellanox ConnectX devices
Hi Michael,
please note that this series depends on mlx5 core device driver patches in mlx5-next branch ingit://git.kernel.org/pub/scm/linux/kernel/git/mellanox/linux.git.
git pull git://git.kernel.org/pub/scm/linux/kernel/git/mellanox/linux.git mlx5-next
They also depend Jason Wang's patches: https://lkml.org/lkml/2020/7/1/301
Jason, I had to resolve some conflicts so I would appreciate of you can verify that it is ok.
The following series of patches provide VDPA support for Mellanox devices. The supported devices are ConnectX6 DX and newer.
Currently, only a network driver is implemented; future patches will introduce a block device driver. iperf performance on a single queue is around 12 Gbps. Future patches will introduce multi queue support.
The files are organized in such a way that code that can be used by different VDPA implementations will be placed in a common are resides in drivers/vdpa/mlx5/core.
Only virtual functions are currently supported. Also, certain firmware capabilities must be set to enable the driver. Physical functions (PFs) are skipped by the driver.
To make use of the VDPA net driver, one must load mlx5_vdpa. In such case, VFs will be operated by the VDPA driver. Although one can see a regular instance of a network driver on the VF, the VDPA driver takes precedence over the NIC driver, steering-wize.
Currently, the device/interface infrastructure in mlx5_core is used to probe drivers. Future patches will introduce virtbus as a means to register devices and drivers and VDPA will be adapted to it.
The mlx5 mode of operation required to support VDPA is switchdev mode. Once can use Linux or OVS bridge to take care of layer 2 switching.
In order to provide virtio networking to a guest, an updated version of qemu is required. This version has been tested by the following quemu version:
url: https://github.com/jasowang/qemu.git
branch: vdpa
Commit ID: 6f4e59b807db
V2->V3
Fix makefile to use include path relative to the root of the kernel
V3-V4
Rebase Jason's patches on linux-next branch
Fix krobot error on mips arch
Make use of the free callback to destroy resoruces on unload Use VIRTIO_F_ACCESS_PLATFORM instead of legacy VIRTIO_F_IOMMU_PLATFORM Add empty implementations for get_vq_notification() and get_vq_irq()
Eli Cohen (6):
net/vdpa: Use struct for set/get vq state
vdpa: Modify get_vq_state() to return error code vdpa/mlx5: Add hardware descriptive header file vdpa/mlx5: Add support library for mlx5 VDPA implementation vdpa/mlx5: Add shared memory registration code vdpa/mlx5: Add VDPA driver for supported mlx5 devices
Jason Wang (5):
vhost-vdpa: refine ioctl pre-processing
vhost: generialize backend features setting/getting
vhost-vdpa: support get/set backend features
vhost-vdpa: support IOTLB batching hints
vdpasim: support batch updating
Max Gurtovoy (1):
vdpa: remove hard coded virtq num
drivers/vdpa/Kconfig | 19 + drivers/vdpa/Makefile | 1 + drivers/vdpa/ifcvf/ifcvf_base.c | 4 +- drivers/vdpa/ifcvf/ifcvf_base.h | 4 +- drivers/vdpa/ifcvf/ifcvf_main.c | 13 +- drivers/vdpa/mlx5/Makefile | 4 + drivers/vdpa/mlx5/core/mlx5_vdpa.h | 91 ++ drivers/vdpa/mlx5/core/mlx5_vdpa_ifc.h | 168 ++ drivers/vdpa/mlx5/core/mr.c | 484 ++++++ drivers/vdpa/mlx5/core/resources.c | 284 ++++ drivers/vdpa/mlx5/net/main.c | 76 + drivers/vdpa/mlx5/net/mlx5_vnet.c | 1965 ++++++++++++++++++++++++ drivers/vdpa/mlx5/net/mlx5_vnet.h | 24 + drivers/vdpa/vdpa.c | 3 + drivers/vdpa/vdpa_sim/vdpa_sim.c | 53 +- drivers/vhost/net.c | 18 +- drivers/vhost/vdpa.c | 76 +- drivers/vhost/vhost.c | 15 + drivers/vhost/vhost.h | 2 + include/linux/vdpa.h | 24 +- include/uapi/linux/vhost.h | 2 + include/uapi/linux/vhost_types.h | 11 + 22 files changed, 3284 insertions(+), 57 deletions(-) create mode 100644 drivers/vdpa/mlx5/Makefile create mode 100644 drivers/vdpa/mlx5/core/mlx5_vdpa.h create mode 100644 drivers/vdpa/mlx5/core/mlx5_vdpa_ifc.h create mode 100644 drivers/vdpa/mlx5/core/mr.c create mode 100644 drivers/vdpa/mlx5/core/resources.c create mode 100644 drivers/vdpa/mlx5/net/main.c create mode 100644 drivers/vdpa/mlx5/net/mlx5_vnet.c create mode 100644 drivers/vdpa/mlx5/net/mlx5_vnet.h
Signature · 1 line
2.26.0
PATCH v4 1/12 vhost-vdpa: refine ioctl pre-processing
From: Jason Wang <[email protected]>
Switch to use 'switch' to make the codes more easier to be extended.
--- drivers/vhost/vdpa.c | 9 +++++---- 1 file changed, 5 insertions(+), 4 deletions(-)
drivers/vhost/vdpa.c +5 -4
@@ -377,15 +377,16 @@ static long vhost_vdpa_vring_ioctl(struct vhost_vdpa *v, unsigned int cmd, idx =array_index_nospec (idx, v->nvqs); vq = &v->vqs[idx]; -if (cmd ==VHOST_VDPA_SET_VRING_ENABLE ) { +switch (cmd) { +case VHOST_VDPA_SET_VRING_ENABLE :if (copy_from_user (&s, argp,sizeof (s)))return -EFAULT ; ops->set_vq_ready (vdpa, idx, s.num);return 0 ; - } - -if (cmd ==VHOST_GET_VRING_BASE ) +case VHOST_GET_VRING_BASE : vq->last_avail_idx = ops->get_vq_state (v->vdpa, idx); +break ; + } r =vhost_vring_ioctl (&v->vdev, cmd, argp);if (r)
Signature · 1 line
2.26.0
PATCH v4 2/12 vhost: generialize backend features setting/getting
In reply to Eli CohenFrom: Jason Wang <[email protected]>
Move the backend features setting/getting from net.c to vhost.c to be reused by vhost-vdpa.
--- drivers/vhost/net.c | 18 ++---------------- drivers/vhost/vhost.c | 15 +++++++++++++++ drivers/vhost/vhost.h | 2 ++ 3 files changed, 19 insertions(+), 16 deletions(-)
drivers/vhost/net.c +2 -16
@@ -1615,21 +1615,6 @@ static long vhost_net_reset_owner(struct vhost_net *n)return err; } -static int vhost_net_set_backend_features (struct vhost_net *n,u64 features) -{ -int i; - -mutex_lock (&n->dev.mutex); -for (i =0 ; i <VHOST_NET_VQ_MAX ; ++i) { -mutex_lock (&n->vqs[i].vq.mutex); - n->vqs[i].vq.acked_backend_features = features; -mutex_unlock (&n->vqs[i].vq.mutex); - } -mutex_unlock (&n->dev.mutex); - -return 0 ; -} -static int vhost_net_set_features (struct vhost_net *n,u64 features) {size_t vhost_hlen, sock_hlen, hdr_len;
@@ -1730,7 +1715,8 @@ static long vhost_net_ioctl(struct file *f, unsigned int ioctl,return -EFAULT ;if (features & ~VHOST_NET_BACKEND_FEATURES )return -EOPNOTSUPP ; -return vhost_net_set_backend_features (n, features); +vhost_set_backend_features (&n->dev, features); +return 0 ;case VHOST_RESET_OWNER :return vhost_net_reset_owner (n);case VHOST_SET_OWNER :
drivers/vhost/vhost.c +15 -0
@@ -2591,6 +2591,21 @@ struct vhost_msg_node *vhost_dequeue_msg(struct vhost_dev *dev, }EXPORT_SYMBOL_GPL (vhost_dequeue_msg); +void vhost_set_backend_features (struct vhost_dev *dev,u64 features) +{ +struct vhost_virtqueue *vq; +int i; + +mutex_lock (&dev->mutex); +for (i =0 ; i < dev->nvqs; ++i) { + vq = dev->vqs[i]; +mutex_lock (&vq->mutex); + vq->acked_backend_features = features; +mutex_unlock (&vq->mutex); + } +mutex_unlock (&dev->mutex); +} +EXPORT_SYMBOL_GPL (vhost_set_backend_features);static int __init vhost_init (void ) {
drivers/vhost/vhost.h +2 -0
@@ -214,6 +214,8 @@ void vhost_enqueue_msg(struct vhost_dev *dev, struct vhost_msg_node *node);struct vhost_msg_node *vhost_dequeue_msg (struct vhost_dev *dev,struct list_head *head); +void vhost_set_backend_features (struct vhost_dev *dev,u64 features); +__poll_t vhost_chr_poll (struct file *file,struct vhost_dev *dev,poll_table *wait); ssize_t vhost_chr_read_iter(struct vhost_dev *dev, struct iov_iter *to,
Signature · 1 line
2.26.0
PATCH v4 3/12 vhost-vdpa: support get/set backend features
In reply to Eli CohenFrom: Jason Wang <[email protected]>
This patch makes userspace can get and set backend features to vhost-vdpa.
--- drivers/vhost/vdpa.c | 18 ++++++++++++++++++ 1 file changed, 18 insertions(+)
drivers/vhost/vdpa.c +18 -0
@@ -26,6 +26,10 @@#include "vhost.h" +enum { +VHOST_VDPA_BACKEND_FEATURES = (1ULL <<VHOST_BACKEND_F_IOTLB_MSG_V2 ) +}; +/* Currently, only network backend w/o multiqueue is supported. */ #define VHOST_VDPA_VQ_MAX 2
@@ -364,6 +368,8 @@ static long vhost_vdpa_vring_ioctl(struct vhost_vdpa *v, unsigned int cmd,struct vdpa_callback cb;struct vhost_virtqueue *vq;struct vhost_vring_state s; +u64 __user *featurep = argp; +u64 features;u32 idx;long r;
@@ -386,6 +392,18 @@ static long vhost_vdpa_vring_ioctl(struct vhost_vdpa *v, unsigned int cmd,case VHOST_GET_VRING_BASE : vq->last_avail_idx = ops->get_vq_state (v->vdpa, idx);break ; +case VHOST_GET_BACKEND_FEATURES : + features =VHOST_VDPA_BACKEND_FEATURES ; +if (copy_to_user (featurep, &features,sizeof (features))) +return -EFAULT ; +return 0 ; +case VHOST_SET_BACKEND_FEATURES : +if (copy_from_user (&features, featurep,sizeof (features))) +return -EFAULT ; +if (features & ~VHOST_VDPA_BACKEND_FEATURES ) +return -EOPNOTSUPP ; +vhost_set_backend_features (&v->vdev, features); +return 0 ; } r =vhost_vring_ioctl (&v->vdev, cmd, argp);
Signature · 1 line
2.26.0
PATCH v4 4/12 vhost-vdpa: support IOTLB batching hints
In reply to Eli CohenFrom: Jason Wang <[email protected]>
This patches extend the vhost IOTLB API to accept batch updating hints form userspace. When userspace wants update the device IOTLB in a batch, it may do:
1) Write vhost_iotlb_msg with VHOST_IOTLB_BATCH_BEGIN flag
2) Perform a batch of IOTLB updating via VHOST_IOTLB_UPDATE/INVALIDATE
3) Write vhost_iotlb_msg with VHOST_IOTLB_BATCH_END flag
Vhost-vdpa may decide to batch the IOMMU/IOTLB updating in step 3 when vDPA device support set_map() ops. This is useful for the vDPA device that want to know all the mappings to tweak their own DMA translation logic.
For vDPA device that doesn't require set_map(), no behavior changes.
This capability is advertised via VHOST_BACKEND_F_IOTLB_BATCH capability.
--- drivers/vhost/vdpa.c | 36 ++++++++++++++++++++++++-------- include/uapi/linux/vhost.h | 2 ++ include/uapi/linux/vhost_types.h | 11 ++++++++++ 3 files changed, 40 insertions(+), 9 deletions(-)
drivers/vhost/vdpa.c +27 -9
@@ -27,7 +27,9 @@#include "vhost.h" enum { -VHOST_VDPA_BACKEND_FEATURES = (1ULL <<VHOST_BACKEND_F_IOTLB_MSG_V2 ) +VHOST_VDPA_BACKEND_FEATURES = + (1ULL <<VHOST_BACKEND_F_IOTLB_MSG_V2 ) | + (1ULL <<VHOST_BACKEND_F_IOTLB_BATCH ), };/* Currently, only network backend w/o multiqueue is supported. */
@@ -48,6 +50,7 @@ struct vhost_vdpa {int virtio_id;int minor;struct eventfd_ctx *config_ctx; +int in_batch; };static DEFINE_IDA (vhost_vdpa_ida);
@@ -140,6 +143,7 @@ static void vhost_vdpa_reset(struct vhost_vdpa *v)struct vdpa_device *vdpa = v->vdpa;vdpa_reset (vdpa); + v->in_batch =0 ; }static long vhost_vdpa_get_device_id (struct vhost_vdpa *v,u8 __user *argp)
@@ -563,13 +567,15 @@ static int vhost_vdpa_map(struct vhost_vdpa *v,if (r)return r; -if (ops->dma_map) +if (ops->dma_map) { r = ops->dma_map (vdpa, iova, size, pa, perm); -else if (ops->set_map) - r = ops->set_map (vdpa, dev->iotlb); -else + }else if (ops->set_map) { +if (!v->in_batch) + r = ops->set_map (vdpa, dev->iotlb); + }else { r =iommu_map (v->domain, iova, pa, size,perm_to_iommu_flags (perm)); + }return r; }
@@ -582,12 +588,14 @@ static void vhost_vdpa_unmap(struct vhost_vdpa *v, u64 iova, u64 size)vhost_vdpa_iotlb_unmap (v, iova, iova + size -1 ); -if (ops->dma_map) +if (ops->dma_map) { ops->dma_unmap (vdpa, iova, size); -else if (ops->set_map) - ops->set_map (vdpa, dev->iotlb); -else + }else if (ops->set_map) { +if (!v->in_batch) + ops->set_map (vdpa, dev->iotlb); + }else {iommu_unmap (v->domain, iova, size); + } } static int vhost_vdpa_process_iotlb_update(struct vhost_vdpa *v,
@@ -680,6 +688,8 @@ static int vhost_vdpa_process_iotlb_msg(struct vhost_dev *dev, struct vhost_iotlb_msg *msg) { struct vhost_vdpa *v = container_of(dev, struct vhost_vdpa, vdev); +struct vdpa_device *vdpa = v->vdpa; +const struct vdpa_config_ops *ops = vdpa->config;int r =0 ; r =vhost_dev_check_owner (dev);
@@ -693,6 +703,14 @@ static int vhost_vdpa_process_iotlb_msg(struct vhost_dev *dev,case VHOST_IOTLB_INVALIDATE :vhost_vdpa_unmap (v, msg->iova, msg->size);break ; +case VHOST_IOTLB_BATCH_BEGIN : + v->in_batch = true; +break ; +case VHOST_IOTLB_BATCH_END : +if (v->in_batch && ops->set_map) + ops->set_map (vdpa, dev->iotlb); + v->in_batch = false; +break ;default : r = -EINVAL ;break ;
include/uapi/linux/vhost.h +2 -0
@@ -91,6 +91,8 @@/* Use message type V2 */ #define VHOST_BACKEND_F_IOTLB_MSG_V2 0x1 +/* IOTLB can accept batching hints */ +#define VHOST_BACKEND_F_IOTLB_BATCH 0x2#define VHOST_SET_BACKEND_FEATURES _IOW(VHOST_VIRTIO, 0x25, __u64)#define VHOST_GET_BACKEND_FEATURES _IOR(VHOST_VIRTIO, 0x26, __u64)
include/uapi/linux/vhost_types.h +11 -0
@@ -60,6 +60,17 @@ struct vhost_iotlb_msg {#define VHOST_IOTLB_UPDATE 2#define VHOST_IOTLB_INVALIDATE 3#define VHOST_IOTLB_ACCESS_FAIL 4 +/* +* VHOST_IOTLB_BATCH_BEGIN and VHOST_IOTLB_BATCH_END allow modifying +* multiple mappings in one go: beginning with +* VHOST_IOTLB_BATCH_BEGIN, followed by any number of +* VHOST_IOTLB_UPDATE messages, and ending with VHOST_IOTLB_BATCH_END. +* When one of these two values is used as the message type, the rest +* of the fields in the message are ignored. There's no guarantee that +* these changes take place automatically in the device. +*/ +#define VHOST_IOTLB_BATCH_BEGIN 5 +#define VHOST_IOTLB_BATCH_END 6__u8 type; };
Signature · 1 line
2.26.0
PATCH v4 5/12 vdpasim: support batch updating
In reply to Eli CohenFrom: Jason Wang <[email protected]>
The vDPA simulator support both set_map() and dma_map()/dma_unmap() operations. But vhost-vdpa can only use one of them. So this patch introduce a module parameter (batch_mapping) that let vpda_sim to support only one of those dma operations. The batched mapping via set_map() is enabled by default.
--- drivers/vdpa/vdpa_sim/vdpa_sim.c | 40 +++++++++++++++++++++++++++++--- 1 file changed, 37 insertions(+), 3 deletions(-)
drivers/vdpa/vdpa_sim/vdpa_sim.c +37 -3
@@ -34,6 +34,10 @@#define DRV_DESC "vDPA Device Simulator"#define DRV_LICENSE "GPL v2" +static int batch_mapping =1 ; +module_param (batch_mapping, int,0444 ); +MODULE_PARM_DESC (batch_mapping,"Batched mapping 1 -Enable; 0 - Disable" ); +struct vdpasim_virtqueue {struct vringh vring;struct vringh_kiov iov;
@@ -334,15 +338,21 @@ static const struct dma_map_ops vdpasim_dma_ops = { };static const struct vdpa_config_ops vdpasim_net_config_ops; +static const struct vdpa_config_ops vdpasim_net_batch_config_ops;static struct vdpasim *vdpasim_create (void ) { +const struct vdpa_config_ops *ops;struct vdpasim *vdpasim;struct device *dev;int ret = -ENOMEM ; - vdpasim = vdpa_alloc_device(struct vdpasim, vdpa, NULL, - &vdpasim_net_config_ops); +if (batch_mapping) + ops = &vdpasim_net_batch_config_ops; +else + ops = &vdpasim_net_config_ops; + + vdpasim = vdpa_alloc_device(struct vdpasim, vdpa, NULL, ops);if (!vdpasim) goto err_alloc;
@@ -641,12 +651,36 @@ static const struct vdpa_config_ops vdpasim_net_config_ops = { .get_config = vdpasim_get_config, .set_config = vdpasim_set_config, .get_generation = vdpasim_get_generation, - .set_map = vdpasim_set_map, .dma_map = vdpasim_dma_map, .dma_unmap = vdpasim_dma_unmap, .free = vdpasim_free, }; +static const struct vdpa_config_ops vdpasim_net_batch_config_ops = { + .set_vq_address = vdpasim_set_vq_address, + .set_vq_num = vdpasim_set_vq_num, + .kick_vq = vdpasim_kick_vq, + .set_vq_cb = vdpasim_set_vq_cb, + .set_vq_ready = vdpasim_set_vq_ready, + .get_vq_ready = vdpasim_get_vq_ready, + .set_vq_state = vdpasim_set_vq_state, + .get_vq_state = vdpasim_get_vq_state, + .get_vq_align = vdpasim_get_vq_align, + .get_features = vdpasim_get_features, + .set_features = vdpasim_set_features, + .set_config_cb = vdpasim_set_config_cb, + .get_vq_num_max = vdpasim_get_vq_num_max, + .get_device_id = vdpasim_get_device_id, + .get_vendor_id = vdpasim_get_vendor_id, + .get_status = vdpasim_get_status, + .set_status = vdpasim_set_status, + .get_config = vdpasim_get_config, + .set_config = vdpasim_set_config, + .get_generation = vdpasim_get_generation, + .set_map = vdpasim_set_map, + .free = vdpasim_free, +}; +static int __init vdpasim_dev_init (void ) { vdpasim_dev =vdpasim_create ();
Signature · 1 line
2.26.0
PATCH v4 6/12 vdpa: remove hard coded virtq num
In reply to Eli CohenFrom: Max Gurtovoy <[email protected]>
This will enable vdpa providers to add support for multi queue feature and publish it to upper layers (vhost and virtio).
--- drivers/vdpa/ifcvf/ifcvf_main.c | 3 ++- drivers/vdpa/vdpa.c | 3 +++ drivers/vdpa/vdpa_sim/vdpa_sim.c | 4 ++-- drivers/vhost/vdpa.c | 9 +++------ include/linux/vdpa.h | 6 ++++-- 5 files changed, 14 insertions(+), 11 deletions(-)
drivers/vdpa/ifcvf/ifcvf_main.c +2 -1
@@ -431,7 +431,8 @@ static int ifcvf_probe(struct pci_dev *pdev, const struct pci_device_id *id) } adapter = vdpa_alloc_device(struct ifcvf_adapter, vdpa, - dev, &ifc_vdpa_ops); + dev, &ifc_vdpa_ops, +IFCVF_MAX_QUEUE_PAIRS *2 );if (adapter ==NULL ) {IFCVF_ERR (pdev,"Failed to allocate vDPA structure" );return -ENOMEM ;
drivers/vdpa/vdpa.c +3 -0
@@ -61,6 +61,7 @@ static void vdpa_release_dev(struct device *d)* initialized but before registered. * @parent: the parent device * @config: the bus operations that is supported by this device +* @nvqs: number of virtqueues supported by this device * @size: size of the parent structure that contains private data * * Driver should use vdpa_alloc_device() wrapper macro instead of
@@ -71,6 +72,7 @@ static void vdpa_release_dev(struct device *d)*/ struct vdpa_device *__vdpa_alloc_device (struct device *parent,const struct vdpa_config_ops *config, +int nvqs,size_t size) {struct vdpa_device *vdev;
@@ -97,6 +99,7 @@ struct vdpa_device *__vdpa_alloc_device(struct device *parent, vdev->index = err; vdev->config = config; vdev->features_valid = false; + vdev->nvqs = nvqs; err =dev_set_name (&vdev->dev,"vdpa%u" , vdev->index);if (err)
drivers/vdpa/vdpa_sim/vdpa_sim.c +2 -2
@@ -65,7 +65,7 @@ static u64 vdpasim_features = (1ULL << VIRTIO_F_ANY_LAYOUT) |/* State of each vdpasim device */ struct vdpasim {struct vdpa_device vdpa; -struct vdpasim_virtqueue vqs[2 ]; +struct vdpasim_virtqueue vqs[VDPASIM_VQ_NUM ];struct work_struct work;/* spinlock to synchronize virtqueue state */ spinlock_t lock;
@@ -352,7 +352,7 @@ static struct vdpasim *vdpasim_create(void)else ops = &vdpasim_net_config_ops; - vdpasim = vdpa_alloc_device(struct vdpasim, vdpa, NULL, ops); + vdpasim = vdpa_alloc_device(struct vdpasim, vdpa, NULL, ops, VDPASIM_VQ_NUM);if (!vdpasim) goto err_alloc;
drivers/vhost/vdpa.c +3 -6
@@ -32,9 +32,6 @@ enum { (1ULL << VHOST_BACKEND_F_IOTLB_BATCH), }; -/* Currently, only network backend w/o multiqueue is supported. */ -#define VHOST_VDPA_VQ_MAX 2 -#define VHOST_VDPA_DEV_MAX (1U << MINORBITS)struct vhost_vdpa {
@@ -947,7 +944,7 @@ static int vhost_vdpa_probe(struct vdpa_device *vdpa) {const struct vdpa_config_ops *ops = vdpa->config;struct vhost_vdpa *v; -int minor, nvqs =VHOST_VDPA_VQ_MAX ; +int minor;int r;/* Currently, we only accept the network devices. */
@@ -968,14 +965,14 @@ static int vhost_vdpa_probe(struct vdpa_device *vdpa)atomic_set (&v->opened,0 ); v->minor = minor; v->vdpa = vdpa; - v->nvqs = nvqs; + v->nvqs = vdpa->nvqs; v->virtio_id = ops->get_device_id (vdpa);device_initialize (&v->dev); v->dev.release = vhost_vdpa_release_dev; v->dev.parent = &vdpa->dev; v->dev.devt =MKDEV (MAJOR (vhost_vdpa_major), minor); - v->vqs =kmalloc_array (nvqs,sizeof (struct vhost_virtqueue ), + v->vqs =kmalloc_array (v->nvqs,sizeof (struct vhost_virtqueue ),GFP_KERNEL );if (!v->vqs) { r = -ENOMEM ;
include/linux/vdpa.h +4 -2
@@ -41,6 +41,7 @@ struct vdpa_device {const struct vdpa_config_ops *config;unsigned int index;bool features_valid; +int nvqs; };/**
@@ -216,11 +217,12 @@ struct vdpa_config_ops {struct vdpa_device *__vdpa_alloc_device (struct device *parent,const struct vdpa_config_ops *config, +int nvqs,size_t size); -#define vdpa_alloc_device (dev_struct, member, parent, config) \ +#define vdpa_alloc_device (dev_struct, member, parent, config, nvqs) \ container_of(__vdpa_alloc_device( \ - parent, config, \ + parent, config, nvqs, \ sizeof(dev_struct) + \ BUILD_BUG_ON_ZERO(offsetof( \ dev_struct, member))), \
Signature · 1 line
2.26.0
PATCH v4 7/12 net/vdpa: Use struct for set/get vq state
In reply to Eli CohenFor now VQ state involves 16 bit available index value encoded in u64 variable. In the future it will be extended to contain more fields. Use struct to contain the state, now containing only a single u16 for the available index. In the future we can add fields to this struct.
--- drivers/vdpa/ifcvf/ifcvf_base.c | 4 ++-- drivers/vdpa/ifcvf/ifcvf_base.h | 4 ++-- drivers/vdpa/ifcvf/ifcvf_main.c | 9 +++++---- drivers/vdpa/vdpa_sim/vdpa_sim.c | 10 ++++++---- drivers/vhost/vdpa.c | 7 +++++-- include/linux/vdpa.h | 18 ++++++++++++++---- 6 files changed, 34 insertions(+), 18 deletions(-)
drivers/vdpa/ifcvf/ifcvf_base.c +2 -2
@@ -272,7 +272,7 @@ static int ifcvf_config_features(struct ifcvf_hw *hw)return 0 ; } -u64 ifcvf_get_vq_state (struct ifcvf_hw *hw,u16 qid) +u16 ifcvf_get_vq_state (struct ifcvf_hw *hw,u16 qid) {struct ifcvf_lm_cfg __iomem *ifcvf_lm;void __iomem *avail_idx_addr;
@@ -287,7 +287,7 @@ u64 ifcvf_get_vq_state(struct ifcvf_hw *hw, u16 qid)return last_avail_idx; } -int ifcvf_set_vq_state (struct ifcvf_hw *hw,u16 qid,u64 num) +int ifcvf_set_vq_state (struct ifcvf_hw *hw,u16 qid,u16 num) {struct ifcvf_lm_cfg __iomem *ifcvf_lm;void __iomem *avail_idx_addr;
drivers/vdpa/ifcvf/ifcvf_base.h +2 -2
@@ -116,7 +116,7 @@ void ifcvf_set_status(struct ifcvf_hw *hw, u8 status);void io_write64_twopart (u64 val,u32 *lo,u32 *hi);void ifcvf_reset (struct ifcvf_hw *hw);u64 ifcvf_get_features (struct ifcvf_hw *hw); -u64 ifcvf_get_vq_state (struct ifcvf_hw *hw,u16 qid); -int ifcvf_set_vq_state (struct ifcvf_hw *hw,u16 qid,u64 num); +u16 ifcvf_get_vq_state (struct ifcvf_hw *hw,u16 qid); +int ifcvf_set_vq_state (struct ifcvf_hw *hw,u16 qid,u16 num);struct ifcvf_adapter *vf_to_adapter (struct ifcvf_hw *hw);#endif /* _IFCVF_H_ */
drivers/vdpa/ifcvf/ifcvf_main.c +5 -4
@@ -237,19 +237,20 @@ static u16 ifcvf_vdpa_get_vq_num_max(struct vdpa_device *vdpa_dev)return IFCVF_QUEUE_MAX ; } -static u64 ifcvf_vdpa_get_vq_state (struct vdpa_device *vdpa_dev,u16 qid) +static void ifcvf_vdpa_get_vq_state (struct vdpa_device *vdpa_dev,u16 qid, +struct vdpa_vq_state *state) {struct ifcvf_hw *vf =vdpa_to_vf (vdpa_dev); -return ifcvf_get_vq_state (vf, qid); + state->avail_index =ifcvf_get_vq_state (vf, qid); }static int ifcvf_vdpa_set_vq_state (struct vdpa_device *vdpa_dev,u16 qid, -u64 num) +const struct vdpa_vq_state *state) {struct ifcvf_hw *vf =vdpa_to_vf (vdpa_dev); -return ifcvf_set_vq_state (vf, qid, num); +return ifcvf_set_vq_state (vf, qid, state->avail_index); } static void ifcvf_vdpa_set_vq_cb(struct vdpa_device *vdpa_dev, u16 qid,
drivers/vdpa/vdpa_sim/vdpa_sim.c +6 -4
@@ -450,26 +450,28 @@ static bool vdpasim_get_vq_ready(struct vdpa_device *vdpa, u16 idx)return vq->ready; } -static int vdpasim_set_vq_state (struct vdpa_device *vdpa,u16 idx,u64 state) +static int vdpasim_set_vq_state (struct vdpa_device *vdpa,u16 idx, +const struct vdpa_vq_state *state) {struct vdpasim *vdpasim =vdpa_to_sim (vdpa);struct vdpasim_virtqueue *vq = &vdpasim->vqs[idx];struct vringh *vrh = &vq->vring;spin_lock (&vdpasim->lock); - vrh->last_avail_idx = state; + vrh->last_avail_idx = state->avail_index;spin_unlock (&vdpasim->lock);return 0 ; } -static u64 vdpasim_get_vq_state (struct vdpa_device *vdpa,u16 idx) +static void vdpasim_get_vq_state (struct vdpa_device *vdpa,u16 idx, +struct vdpa_vq_state *state) {struct vdpasim *vdpasim =vdpa_to_sim (vdpa);struct vdpasim_virtqueue *vq = &vdpasim->vqs[idx];struct vringh *vrh = &vq->vring; -return vrh->last_avail_idx; + state->avail_index = vrh->last_avail_idx; }static u32 vdpasim_get_vq_align (struct vdpa_device *vdpa)
drivers/vhost/vdpa.c +5 -2
@@ -366,6 +366,7 @@ static long vhost_vdpa_vring_ioctl(struct vhost_vdpa *v, unsigned int cmd, {struct vdpa_device *vdpa = v->vdpa;const struct vdpa_config_ops *ops = vdpa->config; +struct vdpa_vq_state vq_state;struct vdpa_callback cb;struct vhost_virtqueue *vq;struct vhost_vring_state s;
@@ -391,7 +392,8 @@ static long vhost_vdpa_vring_ioctl(struct vhost_vdpa *v, unsigned int cmd, ops->set_vq_ready (vdpa, idx, s.num);return 0 ;case VHOST_GET_VRING_BASE : - vq->last_avail_idx = ops->get_vq_state (v->vdpa, idx); + ops->get_vq_state (v->vdpa, idx, &vq_state); + vq->last_avail_idx = vq_state.avail_index;break ;case VHOST_GET_BACKEND_FEATURES : features =VHOST_VDPA_BACKEND_FEATURES ;
@@ -421,7 +423,8 @@ static long vhost_vdpa_vring_ioctl(struct vhost_vdpa *v, unsigned int cmd,break ;case VHOST_SET_VRING_BASE : -if (ops->set_vq_state (vdpa, idx, vq->last_avail_idx)) + vq_state.avail_index = vq->last_avail_idx; +if (ops->set_vq_state (vdpa, idx, &vq_state)) r = -EINVAL ;break ;
include/linux/vdpa.h +14 -4
@@ -27,6 +27,14 @@ struct vdpa_notification_area {resource_size_t size; }; +/** +* vDPA vq_state definition +* @avail_index: available index +*/ +struct vdpa_vq_state { +u16 avail_index; +}; +/** * vDPA device - representation of a vDPA device * @dev: underlying device
@@ -80,12 +88,12 @@ struct vdpa_device {* @set_vq_state: Set the state for a virtqueue * @vdev: vdpa device * @idx: virtqueue index -* @state: virtqueue state (last_avail_idx) +* @state: pointer to set virtqueue state (last_avail_idx) * Returns integer: success (0) or error (< 0) * @get_vq_state: Get the state for a virtqueue * @vdev: vdpa device * @idx: virtqueue index -* Returns virtqueue state (last_avail_idx) +* @state: pointer to returned state (last_avail_idx) * @get_vq_notification: Get the notification area for a virtqueue * @vdev: vdpa device * @idx: virtqueue index
@@ -182,8 +190,10 @@ struct vdpa_config_ops { struct vdpa_callback *cb);void (*set_vq_ready)(struct vdpa_device *vdev,u16 idx,bool ready);bool (*get_vq_ready)(struct vdpa_device *vdev,u16 idx); -int (*set_vq_state)(struct vdpa_device *vdev,u16 idx,u64 state); -u64 (*get_vq_state)(struct vdpa_device *vdev,u16 idx); +int (*set_vq_state)(struct vdpa_device *vdev,u16 idx, +const struct vdpa_vq_state *state); +void (*get_vq_state)(struct vdpa_device *vdev,u16 idx, +struct vdpa_vq_state *state);struct vdpa_notification_area (*get_vq_notification)(struct vdpa_device *vdev,u16 idx);int (*get_vq_irq)(struct vdpa_device *vdv,u16 idx);
Signature · 1 line
2.26.0
PATCH v4 8/12 vdpa: Modify get_vq_state() to return error code
In reply to Eli CohenModify get_vq_state() so it returns an error code. In case of hardware acceleration, the available index may be retrieved from the device, an operation that can possibly fail.
--- drivers/vdpa/ifcvf/ifcvf_main.c | 5 +++-- drivers/vdpa/vdpa_sim/vdpa_sim.c | 5 +++-- drivers/vhost/vdpa.c | 5 ++++- include/linux/vdpa.h | 4 ++-- 4 files changed, 12 insertions(+), 7 deletions(-)
drivers/vdpa/ifcvf/ifcvf_main.c +3 -2
@@ -237,12 +237,13 @@ static u16 ifcvf_vdpa_get_vq_num_max(struct vdpa_device *vdpa_dev)return IFCVF_QUEUE_MAX ; } -static void ifcvf_vdpa_get_vq_state (struct vdpa_device *vdpa_dev,u16 qid, -struct vdpa_vq_state *state) +static int ifcvf_vdpa_get_vq_state (struct vdpa_device *vdpa_dev,u16 qid, +struct vdpa_vq_state *state) {struct ifcvf_hw *vf =vdpa_to_vf (vdpa_dev); state->avail_index =ifcvf_get_vq_state (vf, qid); +return 0 ; } static int ifcvf_vdpa_set_vq_state(struct vdpa_device *vdpa_dev, u16 qid,
drivers/vdpa/vdpa_sim/vdpa_sim.c +3 -2
@@ -464,14 +464,15 @@ static int vdpasim_set_vq_state(struct vdpa_device *vdpa, u16 idx,return 0 ; } -static void vdpasim_get_vq_state (struct vdpa_device *vdpa,u16 idx, -struct vdpa_vq_state *state) +static int vdpasim_get_vq_state (struct vdpa_device *vdpa,u16 idx, +struct vdpa_vq_state *state) {struct vdpasim *vdpasim =vdpa_to_sim (vdpa);struct vdpasim_virtqueue *vq = &vdpasim->vqs[idx];struct vringh *vrh = &vq->vring; state->avail_index = vrh->last_avail_idx; +return 0 ; }static u32 vdpasim_get_vq_align (struct vdpa_device *vdpa)
drivers/vhost/vdpa.c +4 -1
@@ -392,7 +392,10 @@ static long vhost_vdpa_vring_ioctl(struct vhost_vdpa *v, unsigned int cmd, ops->set_vq_ready (vdpa, idx, s.num);return 0 ;case VHOST_GET_VRING_BASE : - ops->get_vq_state (v->vdpa, idx, &vq_state); + r = ops->get_vq_state (v->vdpa, idx, &vq_state); +if (r) +return r; + vq->last_avail_idx = vq_state.avail_index;break ;case VHOST_GET_BACKEND_FEATURES :
include/linux/vdpa.h +2 -2
@@ -192,8 +192,8 @@ struct vdpa_config_ops {bool (*get_vq_ready)(struct vdpa_device *vdev,u16 idx);int (*set_vq_state)(struct vdpa_device *vdev,u16 idx,const struct vdpa_vq_state *state); -void (*get_vq_state)(struct vdpa_device *vdev,u16 idx, -struct vdpa_vq_state *state); +int (*get_vq_state)(struct vdpa_device *vdev,u16 idx, +struct vdpa_vq_state *state);struct vdpa_notification_area (*get_vq_notification)(struct vdpa_device *vdev,u16 idx);int (*get_vq_irq)(struct vdpa_device *vdv,u16 idx);
Signature · 1 line
2.26.0
Re: [PATCH V4 linux-next 08/12] vdpa: Modify get_vq_state() to return error code
Quoted 6 lines · Eli Cohen
Modify
get_vq_state()so it returns an error code. In case of hardware acceleration, the available index may be retrieved from the device, an operation that can possibly fail.Reviewed-by Parav Pandit <[email protected]>Signed-off-by Eli Cohen <[email protected]>
Quoted 70 lines
--- drivers/vdpa/ifcvf/ifcvf_main.c | 5 +++-- drivers/vdpa/vdpa_sim/vdpa_sim.c | 5 +++-- drivers/vhost/vdpa.c | 5 ++++- include/linux/vdpa.h | 4 ++-- 4 files changed, 12 insertions(+), 7 deletions(-)drivers/vdpa/ifcvf/ifcvf_main.c +3
-2@@ -237,12 +237,13 @@ static u16 ifcvf_vdpa_get_vq_num_max(struct vdpa_device *vdpa_dev)return IFCVF_QUEUE_MAX ; } -static void ifcvf_vdpa_get_vq_state (struct vdpa_device *vdpa_dev,u16 qid, -struct vdpa_vq_state *state) +static int ifcvf_vdpa_get_vq_state (struct vdpa_device *vdpa_dev,u16 qid, +struct vdpa_vq_state *state) {struct ifcvf_hw *vf =vdpa_to_vf (vdpa_dev); state->avail_index =ifcvf_get_vq_state (vf, qid); +return 0 ; } static int ifcvf_vdpa_set_vq_state(struct vdpa_device *vdpa_dev, u16 qid,drivers/vdpa/vdpa_sim/vdpa_sim.c +3
-2@@ -464,14 +464,15 @@ static int vdpasim_set_vq_state(struct vdpa_device *vdpa, u16 idx,return 0 ; } -static void vdpasim_get_vq_state (struct vdpa_device *vdpa,u16 idx, -struct vdpa_vq_state *state) +static int vdpasim_get_vq_state (struct vdpa_device *vdpa,u16 idx, +struct vdpa_vq_state *state) {struct vdpasim *vdpasim =vdpa_to_sim (vdpa);struct vdpasim_virtqueue *vq = &vdpasim->vqs[idx];struct vringh *vrh = &vq->vring; state->avail_index = vrh->last_avail_idx; +return 0 ; }static u32 vdpasim_get_vq_align (struct vdpa_device *vdpa)drivers/vhost/vdpa.c +4
-1@@ -392,7 +392,10 @@ static long vhost_vdpa_vring_ioctl(struct vhost_vdpa *v, unsigned int cmd, ops->set_vq_ready (vdpa, idx, s.num);return 0 ;case VHOST_GET_VRING_BASE : - ops->get_vq_state (v->vdpa, idx, &vq_state); + r = ops->get_vq_state (v->vdpa, idx, &vq_state); +if (r) +return r; + vq->last_avail_idx = vq_state.avail_index;break ;case VHOST_GET_BACKEND_FEATURES :include/linux/vdpa.h +0
-0
include/linux/vdpa.h struct vdpa_config_ops { · line 199
Quoted 10 lines
include/linux/vdpa.h +2
-2@@ -192,8 +192,8 @@ struct vdpa_config_ops {bool (*get_vq_ready)(struct vdpa_device *vdev,u16 idx);int (*set_vq_state)(struct vdpa_device *vdev,u16 idx,const struct vdpa_vq_state *state); -void (*get_vq_state)(struct vdpa_device *vdev,u16 idx, -struct vdpa_vq_state *state); +int (*get_vq_state)(struct vdpa_device *vdev,u16 idx, +struct vdpa_vq_state *state);struct vdpa_notification_area (*get_vq_notification)(struct vdpa_device *vdev,u16 idx);int (*get_vq_irq)(struct vdpa_device *vdv,u16 idx);
_______________________________________________
Virtualization mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/virtualization
PATCH v4 9/12 vdpa/mlx5: Add hardware descriptive header file
In reply to Eli CohenKeep all vdpa related hardware definitions in this file.
--- drivers/vdpa/mlx5/core/mlx5_vdpa_ifc.h | 168 +++++++++++++++++++++++++ 1 file changed, 168 insertions(+) create mode 100644 drivers/vdpa/mlx5/core/mlx5_vdpa_ifc.h
drivers/vdpa/mlx5/core/mlx5_vdpa_ifc.h new file +168 -0
@@ -0,0 +1,168 @@ +/* SPDX-License-Identifier: GPL-2.0 OR Linux-OpenIB */ +/* Copyright (c) 2020 Mellanox Technologies Ltd. */ + +#ifndef __MLX5_VDPA_IFC_H_ +#define __MLX5_VDPA_IFC_H_ + +#include <linux/mlx5/mlx5_ifc.h> + +enum { +MLX5_VIRTIO_Q_EVENT_MODE_NO_MSIX_MODE =0x0 , +MLX5_VIRTIO_Q_EVENT_MODE_QP_MODE =0x1 , +MLX5_VIRTIO_Q_EVENT_MODE_MSIX_MODE =0x2 , +}; + +enum { +MLX5_VIRTIO_EMULATION_CAP_VIRTIO_QUEUE_TYPE_SPLIT =0x1 ,// do I check this caps? +MLX5_VIRTIO_EMULATION_CAP_VIRTIO_QUEUE_TYPE_PACKED =0x2 , +}; + +enum { +MLX5_VIRTIO_EMULATION_VIRTIO_QUEUE_TYPE_SPLIT =0 , +MLX5_VIRTIO_EMULATION_VIRTIO_QUEUE_TYPE_PACKED =1 , +}; + +struct mlx5_ifc_virtio_q_bits { +u8 virtio_q_type[0x8 ]; +u8 reserved_at_8[0x5 ]; +u8 event_mode[0x3 ]; +u8 queue_index[0x10 ]; + +u8 full_emulation[0x1 ]; +u8 virtio_version_1_0[0x1 ]; +u8 reserved_at_22[0x2 ]; +u8 offload_type[0x4 ]; +u8 event_qpn_or_msix[0x18 ]; + +u8 doorbell_stride_index[0x10 ]; +u8 queue_size[0x10 ]; + +u8 device_emulation_id[0x20 ]; + +u8 desc_addr[0x40 ]; + +u8 used_addr[0x40 ]; + +u8 available_addr[0x40 ]; + +u8 virtio_q_mkey[0x20 ]; + +u8 max_tunnel_desc[0x10 ]; +u8 reserved_at_170[0x8 ]; +u8 error_type[0x8 ]; + +u8 umem_1_id[0x20 ]; + +u8 umem_1_size[0x20 ]; + +u8 umem_1_offset[0x40 ]; + +u8 umem_2_id[0x20 ]; + +u8 umem_2_size[0x20 ]; + +u8 umem_2_offset[0x40 ]; + +u8 umem_3_id[0x20 ]; + +u8 umem_3_size[0x20 ]; + +u8 umem_3_offset[0x40 ]; + +u8 counter_set_id[0x20 ]; + +u8 reserved_at_320[0x8 ]; +u8 pd[0x18 ]; + +u8 reserved_at_340[0xc0 ]; +}; + +struct mlx5_ifc_virtio_net_q_object_bits { +u8 modify_field_select[0x40 ]; + +u8 reserved_at_40[0x20 ]; + +u8 vhca_id[0x10 ]; +u8 reserved_at_70[0x10 ]; + +u8 queue_feature_bit_mask_12_3[0xa ]; +u8 dirty_bitmap_dump_enable[0x1 ]; +u8 vhost_log_page[0x5 ]; +u8 reserved_at_90[0xc ]; +u8 state[0x4 ]; + +u8 reserved_at_a0[0x5 ]; +u8 queue_feature_bit_mask_2_0[0x3 ]; +u8 tisn_or_qpn[0x18 ]; + +u8 dirty_bitmap_mkey[0x20 ]; + +u8 dirty_bitmap_size[0x20 ]; + +u8 dirty_bitmap_addr[0x40 ]; + +u8 hw_available_index[0x10 ]; +u8 hw_used_index[0x10 ]; + +u8 reserved_at_160[0xa0 ]; + +struct mlx5_ifc_virtio_q_bits virtio_q_context; +}; + +struct mlx5_ifc_create_virtio_net_q_in_bits { +struct mlx5_ifc_general_obj_in_cmd_hdr_bits general_obj_in_cmd_hdr; + +struct mlx5_ifc_virtio_net_q_object_bits obj_context; +}; + +struct mlx5_ifc_create_virtio_net_q_out_bits { +struct mlx5_ifc_general_obj_out_cmd_hdr_bits general_obj_out_cmd_hdr; +}; + +struct mlx5_ifc_destroy_virtio_net_q_in_bits { +struct mlx5_ifc_general_obj_in_cmd_hdr_bits general_obj_out_cmd_hdr; +}; + +struct mlx5_ifc_destroy_virtio_net_q_out_bits { +struct mlx5_ifc_general_obj_out_cmd_hdr_bits general_obj_out_cmd_hdr; +}; + +struct mlx5_ifc_query_virtio_net_q_in_bits { +struct mlx5_ifc_general_obj_in_cmd_hdr_bits general_obj_in_cmd_hdr; +}; + +struct mlx5_ifc_query_virtio_net_q_out_bits { +struct mlx5_ifc_general_obj_out_cmd_hdr_bits general_obj_out_cmd_hdr; + +struct mlx5_ifc_virtio_net_q_object_bits obj_context; +}; + +enum { +MLX5_VIRTQ_MODIFY_MASK_STATE = (u64 )1 <<0 , +MLX5_VIRTQ_MODIFY_MASK_DIRTY_BITMAP_PARAMS = (u64 )1 <<3 , +MLX5_VIRTQ_MODIFY_MASK_DIRTY_BITMAP_DUMP_ENABLE = (u64 )1 <<4 , +}; + +enum { +MLX5_VIRTIO_NET_Q_OBJECT_STATE_INIT =0x0 , +MLX5_VIRTIO_NET_Q_OBJECT_STATE_RDY =0x1 , +MLX5_VIRTIO_NET_Q_OBJECT_STATE_SUSPEND =0x2 , +MLX5_VIRTIO_NET_Q_OBJECT_STATE_ERR =0x3 , +}; + +enum { +MLX5_RQTC_LIST_Q_TYPE_RQ =0x0 , +MLX5_RQTC_LIST_Q_TYPE_VIRTIO_NET_Q =0x1 , +}; + +struct mlx5_ifc_modify_virtio_net_q_in_bits { +struct mlx5_ifc_general_obj_in_cmd_hdr_bits general_obj_in_cmd_hdr; + +struct mlx5_ifc_virtio_net_q_object_bits obj_context; +}; + +struct mlx5_ifc_modify_virtio_net_q_out_bits { +struct mlx5_ifc_general_obj_out_cmd_hdr_bits general_obj_out_cmd_hdr; +}; + +#endif /* __MLX5_VDPA_IFC_H_ */
Signature · 1 line
2.26.0
PATCH v4 10/12 vdpa/mlx5: Add support library for mlx5 VDPA implementation
In reply to Eli CohenFollowing patches introduce VDPA network driver for Mellanox Connectx6 devices. This patch provides functionality that will be used by those patches.
--- drivers/vdpa/Kconfig | 9 + drivers/vdpa/Makefile | 1 + drivers/vdpa/mlx5/Makefile | 1 + drivers/vdpa/mlx5/core/Makefile | 1 + drivers/vdpa/mlx5/core/mlx5_vdpa.h | 57 ++++++ drivers/vdpa/mlx5/core/resources.c | 281 +++++++++++++++++++++++++++++ 6 files changed, 350 insertions(+) create mode 100644 drivers/vdpa/mlx5/Makefile create mode 100644 drivers/vdpa/mlx5/core/Makefile create mode 100644 drivers/vdpa/mlx5/core/mlx5_vdpa.h create mode 100644 drivers/vdpa/mlx5/core/resources.c
drivers/vdpa/Kconfig +9 -0
@@ -28,4 +28,13 @@ config IFCVF To compile this driver as a module, choose M here: the module will be called ifcvf. +config MLX5_VDPA + bool "MLX5 VDPA support library for ConnectX devices" + depends on MLX5_CORE + default n + help + Support library for Mellanox VDPA drivers. Provides code that is + common for all types of VDPA drivers. The following drivers are planned: + net, block. + endif # VDPA
drivers/vdpa/Makefile +1 -0
@@ -2,3 +2,4 @@ obj-$(CONFIG_VDPA) += vdpa.o obj-$(CONFIG_VDPA_SIM) += vdpa_sim/ obj-$(CONFIG_IFCVF) += ifcvf/ +obj-$(CONFIG_MLX5_VDPA) += mlx5/
drivers/vdpa/mlx5/Makefile new file +1 -0
@@ -0,0 +1 @@ +obj-$(CONFIG_MLX5_VDPA) += core/resources.o
drivers/vdpa/mlx5/core/Makefile new file +1 -0
@@ -0,0 +1 @@ +obj-y += resources.o
drivers/vdpa/mlx5/core/mlx5_vdpa.h new file +57 -0
57 lines not on this page · raw message
drivers/vdpa/mlx5/core/resources.c new file +281 -0
281 lines not on this page · raw message
Signature · 1 line
2.26.0
PATCH v4 11/12 vdpa/mlx5: Add shared memory registration code
In reply to Eli CohenAdd code to support registering address space region for the device. The virtio driver can run as either:
1. Guest virtio driver
2. Userspace virtio driver on the host
3. Kernel virtio driver on the host
In any case a memory key object is required to provide access to memory for the device.
This code will be shared by network or block driver implementations.
--- drivers/vdpa/mlx5/Makefile | 2 +- drivers/vdpa/mlx5/core/Makefile | 1 - drivers/vdpa/mlx5/core/mlx5_vdpa.h | 34 ++ drivers/vdpa/mlx5/core/mr.c | 484 +++++++++++++++++++++++++++++ drivers/vdpa/mlx5/core/resources.c | 3 + 5 files changed, 522 insertions(+), 2 deletions(-) delete mode 100644 drivers/vdpa/mlx5/core/Makefile create mode 100644 drivers/vdpa/mlx5/core/mr.c
drivers/vdpa/mlx5/Makefile +1 -1
@@ -1 +1 @@ -obj-$(CONFIG_MLX5_VDPA) += core/resources.o +obj-$(CONFIG_MLX5_VDPA) += core/resources.o core/mr.o
drivers/vdpa/mlx5/core/Makefile deleted +0 -1
@@ -1 +0,0 @@ -obj-y += resources.o
drivers/vdpa/mlx5/core/mlx5_vdpa.h +34 -0
@@ -7,6 +7,31 @@#include <linux/vdpa.h> #include <linux/mlx5/driver.h> +struct mlx5_vdpa_direct_mr { +u64 start; +u64 end; +u32 perm; +struct mlx5_core_mkey mr; +struct sg_table sg_head; +int log_size; +int nsg; +struct list_head list; +u64 offset; +}; + +struct mlx5_vdpa_mr { +struct mlx5_core_mkey mkey; + +/* list of direct MRs descendants of this indirect mr */ +struct list_head head; +unsigned long num_directs; +unsigned long num_klms; +bool initialized; + +/* serialize mkey creation and destruction */ +struct mutex mkey_mtx; +}; +struct mlx5_vdpa_resources {u32 pdn;struct mlx5_uars_page *uar;
@@ -26,6 +51,8 @@ struct mlx5_vdpa_dev {u8 status;u32 max_vqs;u32 generation; + +struct mlx5_vdpa_mr mr; };int mlx5_vdpa_alloc_pd (struct mlx5_vdpa_dev *dev,u32 *pdn,u16 uid);
@@ -41,6 +68,13 @@ int mlx5_vdpa_alloc_transport_domain(struct mlx5_vdpa_dev *mvdev, u32 *tdn);void mlx5_vdpa_dealloc_transport_domain (struct mlx5_vdpa_dev *mvdev,u32 tdn);int mlx5_vdpa_alloc_resources (struct mlx5_vdpa_dev *mvdev);void mlx5_vdpa_free_resources (struct mlx5_vdpa_dev *mvdev); +int mlx5_vdpa_create_mkey (struct mlx5_vdpa_dev *mvdev,struct mlx5_core_mkey *mkey,u32 *in, +int inlen); +int mlx5_vdpa_destroy_mkey (struct mlx5_vdpa_dev *mvdev,struct mlx5_core_mkey *mkey); +int mlx5_vdpa_handle_set_map (struct mlx5_vdpa_dev *mvdev,struct vhost_iotlb *iotlb, +bool *change_map); +int mlx5_vdpa_create_mr (struct mlx5_vdpa_dev *mvdev,struct vhost_iotlb *iotlb); +void mlx5_vdpa_destroy_mr (struct mlx5_vdpa_dev *mvdev);#define mlx5_vdpa_warn (__dev, format, ...) \ dev_warn((__dev)->mdev->device, "%s:%d:(pid %d) warning: " format, __func__, __LINE__, \
drivers/vdpa/mlx5/core/mr.c new file +484 -0
484 lines not on this page · raw message
drivers/vdpa/mlx5/core/resources.c +3 -0
@@ -227,6 +227,7 @@ int mlx5_vdpa_alloc_resources(struct mlx5_vdpa_dev *mvdev)mlx5_vdpa_warn (mvdev,"resources already allocated\n" );return -EINVAL ; } +mutex_init (&mvdev->mr.mkey_mtx); res->uar =mlx5_get_uars_page (mdev);if (IS_ERR (res->uar)) { err =PTR_ERR (res->uar);
@@ -262,6 +263,7 @@ int mlx5_vdpa_alloc_resources(struct mlx5_vdpa_dev *mvdev) err_uctx:mlx5_put_uars_page (mdev, res->uar); err_uars: +mutex_destroy (&mvdev->mr.mkey_mtx);return err; }
@@ -277,5 +279,6 @@ void mlx5_vdpa_free_resources(struct mlx5_vdpa_dev *mvdev)dealloc_pd (mvdev, res->pdn, res->uid);destroy_uctx (mvdev, res->uid);mlx5_put_uars_page (mvdev->mdev, res->uar); +mutex_destroy (&mvdev->mr.mkey_mtx); res->valid = false; }
Signature · 1 line
2.26.0
PATCH v4 12/12 vdpa/mlx5: Add VDPA driver for supported mlx5 devices
In reply to Eli CohenAdd a front end VDPA driver that registers in the VDPA bus and provides networking to a guest. The VDPA driver creates the necessary resources on the VF it is driving such that data path will be offloaded.
Notifications are being communicated through the driver.
Currently, only VFs are supported. In subsequent patches we will have devlink support to control which VF is used for VDPA and which function is used for regular networking.
--- drivers/vdpa/Kconfig | 10 + drivers/vdpa/mlx5/Makefile | 5 +- drivers/vdpa/mlx5/core/mr.c | 2 +- drivers/vdpa/mlx5/net/main.c | 76 ++ drivers/vdpa/mlx5/net/mlx5_vnet.c | 1965 +++++++++++++++++++++++++++++ drivers/vdpa/mlx5/net/mlx5_vnet.h | 24 + 6 files changed, 2080 insertions(+), 2 deletions(-) create mode 100644 drivers/vdpa/mlx5/net/main.c create mode 100644 drivers/vdpa/mlx5/net/mlx5_vnet.c create mode 100644 drivers/vdpa/mlx5/net/mlx5_vnet.h
drivers/vdpa/Kconfig +10 -0
@@ -37,4 +37,14 @@ config MLX5_VDPA common for all types of VDPA drivers. The following drivers are planned: net, block. +config MLX5_VDPA_NET + tristate "vDPA driver for ConnectX devices" + depends on MLX5_VDPA + default n + help + VDPA network driver for ConnectX6 and newer. Provides offloading + of virtio net datapath such that descriptors put on the ring will + be executed by the hardware. It also supports a variety of stateless + offloads depending on the actual device used and firmware version. + endif # VDPA
drivers/vdpa/mlx5/Makefile +4 -1
@@ -1 +1,4 @@ -obj-$(CONFIG_MLX5_VDPA) += core/resources.o core/mr.o +subdir-ccflags-y += -I$(srctree)/drivers/vdpa/mlx5/core + +obj-$(CONFIG_MLX5_VDPA_NET) += mlx5_vdpa.o +mlx5_vdpa-$(CONFIG_MLX5_VDPA_NET) += net/main.o net/mlx5_vnet.o core/resources.o core/mr.o
drivers/vdpa/mlx5/core/mr.c +1 -1
@@ -464,7 +464,7 @@ int mlx5_vdpa_handle_set_map(struct mlx5_vdpa_dev *mvdev, struct vhost_iotlb *io bool *change_map) {struct mlx5_vdpa_mr *mr = &mvdev->mr; -int err; +int err =0 ; *change_map = false;if (map_empty (iotlb)) {
drivers/vdpa/mlx5/net/main.c new file +76 -0
76 lines not on this page · raw message
drivers/vdpa/mlx5/net/mlx5_vnet.c new file +1965 -0
1,965 lines not on this page · raw message
drivers/vdpa/mlx5/net/mlx5_vnet.h new file +24 -0
@@ -0,0 +1,24 @@ +/* SPDX-License-Identifier: GPL-2.0 OR Linux-OpenIB */ +/* Copyright (c) 2020 Mellanox Technologies Ltd. */ + +#ifndef __MLX5_VNET_H_ +#define __MLX5_VNET_H_ + +#include <linux/vdpa.h> +#include <linux/virtio_net.h> +#include <linux/vringh.h> +#include <linux/mlx5/driver.h> +#include <linux/mlx5/cq.h> +#include <linux/mlx5/qp.h> +#include "mlx5_vdpa.h" + +static inline u32 mlx5_vdpa_max_qps (int max_vqs) +{ +return max_vqs /2 ; +} + +#define to_mlx5_vdpa_ndev (__mvdev) container_of(__mvdev, struct mlx5_vdpa_net, mvdev) +void *mlx5_vdpa_add_dev (struct mlx5_core_dev *mdev); +void mlx5_vdpa_remove_dev (struct mlx5_vdpa_dev *mvdev); + +#endif /* __MLX5_VNET_H_ */
Signature · 1 line
2.26.0
Re: [PATCH V4 linux-next 12/12] vdpa/mlx5: Add VDPA driver for supported mlx5 devices
Quoted 6 lines · Eli Cohen
Add a front end VDPA driver that registers in the VDPA bus and provides networking to a guest. The VDPA driver creates the necessary resources on the VF it is driving such that data path will be offloaded.
Notifications are being communicated through the driver.
Quoted 7 lines
Currently, only VFs are supported. In subsequent patches we will have devlink support to control which VF is used for VDPA and which function is used for regular networking.
Reviewed-by Parav Pandit<[email protected]>Signed-off-by Eli Cohen<[email protected]>---
_______________________________________________
Virtualization mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/virtualization
Michael S. Tsirkin
maintainerQuoted 4 lines · Eli Cohen
Hi Michael,
please note that this series depends on mlx5 core device driver patches in mlx5-next branch ingit://git.kernel.org/pub/scm/linux/kernel/git/mellanox/linux.git.
Thanks! OK so what's the plan for merging this?
Do patches at least build well enough that I can push them upstream? Or do they have to go on top of the mellanox tree?
Quoted 3 lines
git pull
git://git.kernel.org/pub/scm/linux/kernel/git/mellanox/linux.gitmlx5-nextThey also depend Jason Wang's patches: https://lkml.org/lkml/2020/7/1/301
The ones you included, right?
Quoted 102 lines
Jason, I had to resolve some conflicts so I would appreciate of you can verify that it is ok.
The following series of patches provide VDPA support for Mellanox devices. The supported devices are ConnectX6 DX and newer.
Currently, only a network driver is implemented; future patches will introduce a block device driver. iperf performance on a single queue is around 12 Gbps. Future patches will introduce multi queue support.
The files are organized in such a way that code that can be used by different VDPA implementations will be placed in a common are resides in
drivers/vdpa/mlx5/core.Only virtual functions are currently supported. Also, certain firmware capabilities must be set to enable the driver. Physical functions (PFs) are skipped by the driver.
To make use of the VDPA net driver, one must load
mlx5_vdpa. In such case, VFs will be operated by the VDPA driver. Although one can see a regular instance of a network driver on the VF, the VDPA driver takes precedence over the NIC driver, steering-wize.Currently, the device/interface infrastructure in
mlx5_coreis used to probe drivers. Future patches will introduce virtbus as a means to register devices and drivers and VDPA will be adapted to it.The mlx5 mode of operation required to support VDPA is switchdev mode. Once can use Linux or OVS bridge to take care of layer 2 switching.
In order to provide virtio networking to a guest, an updated version of qemu is required. This version has been tested by the following quemu version:
url: https://github.com/jasowang/qemu.git
branch: vdpa
Commit ID:6f4e59b807db
V2->V3
Fix makefile to use include path relative to the root of the kernelV3-V4
Rebase Jason's patches on linux-next branch
Fix krobot error on mips arch
Make use of the free callback to destroy resoruces on unload UseVIRTIO_F_ACCESS_PLATFORMinstead of legacyVIRTIO_F_IOMMU_PLATFORMAdd empty implementations forget_vq_notification()andget_vq_irq()Eli Cohen (6):
net/vdpa: Use struct for set/get vq state
vdpa: Modifyget_vq_state()to return error code vdpa/mlx5: Add hardware descriptive header file vdpa/mlx5: Add support library for mlx5 VDPA implementation vdpa/mlx5: Add shared memory registration code vdpa/mlx5: Add VDPA driver for supported mlx5 devicesJason Wang (5):
vhost-vdpa: refine ioctl pre-processing
vhost: generialize backend features setting/getting
vhost-vdpa: support get/set backend features
vhost-vdpa: support IOTLB batching hints
vdpasim: support batch updatingMax Gurtovoy (1):
vdpa: remove hard coded virtq numdrivers/vdpa/Kconfig | 19 + drivers/vdpa/Makefile | 1 + drivers/vdpa/ifcvf/ifcvf_base.c | 4 +- drivers/vdpa/ifcvf/ifcvf_base.h | 4 +- drivers/vdpa/ifcvf/ifcvf_main.c | 13 +- drivers/vdpa/mlx5/Makefile | 4 + drivers/vdpa/mlx5/core/mlx5_vdpa.h | 91 ++ drivers/vdpa/mlx5/core/mlx5_vdpa_ifc.h | 168 ++ drivers/vdpa/mlx5/core/mr.c | 484 ++++++ drivers/vdpa/mlx5/core/resources.c | 284 ++++ drivers/vdpa/mlx5/net/main.c | 76 + drivers/vdpa/mlx5/net/mlx5_vnet.c | 1965 ++++++++++++++++++++++++ drivers/vdpa/mlx5/net/mlx5_vnet.h | 24 + drivers/vdpa/vdpa.c | 3 + drivers/vdpa/vdpa_sim/vdpa_sim.c | 53 +- drivers/vhost/net.c | 18 +- drivers/vhost/vdpa.c | 76 +- drivers/vhost/vhost.c | 15 + drivers/vhost/vhost.h | 2 + include/linux/vdpa.h | 24 +- include/uapi/linux/vhost.h | 2 + include/uapi/linux/vhost_types.h | 11 + 22 files changed, 3284 insertions(+), 57 deletions(-) create mode 100644 drivers/vdpa/mlx5/Makefile create mode 100644 drivers/vdpa/mlx5/core/mlx5_vdpa.h create mode 100644 drivers/vdpa/mlx5/core/mlx5_vdpa_ifc.h create mode 100644 drivers/vdpa/mlx5/core/mr.c create mode 100644 drivers/vdpa/mlx5/core/resources.c create mode 100644 drivers/vdpa/mlx5/net/main.c create mode 100644 drivers/vdpa/mlx5/net/mlx5_vnet.c create mode 100644 drivers/vdpa/mlx5/net/mlx5_vnet.hSignature · 1 line
2.26.0
Quoted 10 lines · Michael S. Tsirkin
Quoted 4 lines · Eli Cohen
Hi Michael,
please note that this series depends on mlx5 core device driver patches in mlx5-next branch ingit://git.kernel.org/pub/scm/linux/kernel/git/mellanox/linux.git.Thanks! OK so what's the plan for merging this?
Do patches at least build well enough that I can push them upstream? Or do they have to go on top of the mellanox tree?
The patches are built on your linux-next branch which I updated yesterday.
I am based on this commit:776b7b25f10b (origin/linux-next) vhost: add an RPMsg API
On top of that I mergedgit://git.kernel.org/pub/scm/linux/kernel/git/mellanox/linux.git
and after that I have Jason's patches (five patches), than one patch from Max and then my patches (seven).
It builds fine on x84_64.
I fixed some conflicts on Jason's patches.
I also tested it to verify it's working.
BTW, for some reason I did not get all the patches into my mailbox and I suspect they were not all sent. Did you get all the series 0-13?
Please let me know, and if needed I'll resend.
Quoted 7 lines
Quoted 4 lines
git pull
git://git.kernel.org/pub/scm/linux/kernel/git/mellanox/linux.gitmlx5-nextThey also depend Jason Wang's patches: https://lkml.org/lkml/2020/7/1/301
The ones you included, right?
Right.
Quoted 103 lines
Quoted 102 lines
Jason, I had to resolve some conflicts so I would appreciate of you can verify that it is ok.
The following series of patches provide VDPA support for Mellanox devices. The supported devices are ConnectX6 DX and newer.
Currently, only a network driver is implemented; future patches will introduce a block device driver. iperf performance on a single queue is around 12 Gbps. Future patches will introduce multi queue support.
The files are organized in such a way that code that can be used by different VDPA implementations will be placed in a common are resides in
drivers/vdpa/mlx5/core.Only virtual functions are currently supported. Also, certain firmware capabilities must be set to enable the driver. Physical functions (PFs) are skipped by the driver.
To make use of the VDPA net driver, one must load
mlx5_vdpa. In such case, VFs will be operated by the VDPA driver. Although one can see a regular instance of a network driver on the VF, the VDPA driver takes precedence over the NIC driver, steering-wize.Currently, the device/interface infrastructure in
mlx5_coreis used to probe drivers. Future patches will introduce virtbus as a means to register devices and drivers and VDPA will be adapted to it.The mlx5 mode of operation required to support VDPA is switchdev mode. Once can use Linux or OVS bridge to take care of layer 2 switching.
In order to provide virtio networking to a guest, an updated version of qemu is required. This version has been tested by the following quemu version:
url: https://github.com/jasowang/qemu.git
branch: vdpa
Commit ID:6f4e59b807db
V2->V3
Fix makefile to use include path relative to the root of the kernelV3-V4
Rebase Jason's patches on linux-next branch
Fix krobot error on mips arch
Make use of the free callback to destroy resoruces on unload UseVIRTIO_F_ACCESS_PLATFORMinstead of legacyVIRTIO_F_IOMMU_PLATFORMAdd empty implementations forget_vq_notification()andget_vq_irq()Eli Cohen (6):
net/vdpa: Use struct for set/get vq state
vdpa: Modifyget_vq_state()to return error code vdpa/mlx5: Add hardware descriptive header file vdpa/mlx5: Add support library for mlx5 VDPA implementation vdpa/mlx5: Add shared memory registration code vdpa/mlx5: Add VDPA driver for supported mlx5 devicesJason Wang (5):
vhost-vdpa: refine ioctl pre-processing
vhost: generialize backend features setting/getting
vhost-vdpa: support get/set backend features
vhost-vdpa: support IOTLB batching hints
vdpasim: support batch updatingMax Gurtovoy (1):
vdpa: remove hard coded virtq numdrivers/vdpa/Kconfig | 19 + drivers/vdpa/Makefile | 1 + drivers/vdpa/ifcvf/ifcvf_base.c | 4 +- drivers/vdpa/ifcvf/ifcvf_base.h | 4 +- drivers/vdpa/ifcvf/ifcvf_main.c | 13 +- drivers/vdpa/mlx5/Makefile | 4 + drivers/vdpa/mlx5/core/mlx5_vdpa.h | 91 ++ drivers/vdpa/mlx5/core/mlx5_vdpa_ifc.h | 168 ++ drivers/vdpa/mlx5/core/mr.c | 484 ++++++ drivers/vdpa/mlx5/core/resources.c | 284 ++++ drivers/vdpa/mlx5/net/main.c | 76 + drivers/vdpa/mlx5/net/mlx5_vnet.c | 1965 ++++++++++++++++++++++++ drivers/vdpa/mlx5/net/mlx5_vnet.h | 24 + drivers/vdpa/vdpa.c | 3 + drivers/vdpa/vdpa_sim/vdpa_sim.c | 53 +- drivers/vhost/net.c | 18 +- drivers/vhost/vdpa.c | 76 +- drivers/vhost/vhost.c | 15 + drivers/vhost/vhost.h | 2 + include/linux/vdpa.h | 24 +- include/uapi/linux/vhost.h | 2 + include/uapi/linux/vhost_types.h | 11 + 22 files changed, 3284 insertions(+), 57 deletions(-) create mode 100644 drivers/vdpa/mlx5/Makefile create mode 100644 drivers/vdpa/mlx5/core/mlx5_vdpa.h create mode 100644 drivers/vdpa/mlx5/core/mlx5_vdpa_ifc.h create mode 100644 drivers/vdpa/mlx5/core/mr.c create mode 100644 drivers/vdpa/mlx5/core/resources.c create mode 100644 drivers/vdpa/mlx5/net/main.c create mode 100644 drivers/vdpa/mlx5/net/mlx5_vnet.c create mode 100644 drivers/vdpa/mlx5/net/mlx5_vnet.hSignature · 1 line
2.26.0
Quoted 22 lines · Eli Cohen
Quoted 9 lines · Michael S. Tsirkin
Quoted 4 lines · Eli Cohen
Hi Michael,
please note that this series depends on mlx5 core device driver patches in mlx5-next branch ingit://git.kernel.org/pub/scm/linux/kernel/git/mellanox/linux.git.Thanks! OK so what's the plan for merging this?
Do patches at least build well enough that I can push them upstream? Or do they have to go on top of the mellanox tree?The patches are built on your linux-next branch which I updated yesterday.
I am based on this commit:
776b7b25f10b(origin/linux-next) vhost: add an RPMsg APIOn top of that I merged
git://git.kernel.org/pub/scm/linux/kernel/git/mellanox/linux.gitand after that I have Jason's patches (five patches), than one patch from Max and then my patches (seven).
Quoted 7 lines
It builds fine on
x84_64.
I fixed some conflicts on Jason's patches.I also tested it to verify it's working.
BTW, for some reason I did not get all the patches into my mailbox and I suspect they were not all sent. Did you get all the series 0-13?
I can see patch 0 to patch 12 but not patch 13 (I guess 12 is all).
Thanks
Quoted 112 lines
Please let me know, and if needed I'll resend.
Quoted 5 lines
Quoted 3 lines
git pull
git://git.kernel.org/pub/scm/linux/kernel/git/mellanox/linux.gitmlx5-nextThey also depend Jason Wang's patches: https://lkml.org/lkml/2020/7/1/301
The ones you included, right?
Right.
Quoted 102 lines
Quoted 102 lines
Jason, I had to resolve some conflicts so I would appreciate of you can verify that it is ok.
The following series of patches provide VDPA support for Mellanox devices. The supported devices are ConnectX6 DX and newer.
Currently, only a network driver is implemented; future patches will introduce a block device driver. iperf performance on a single queue is around 12 Gbps. Future patches will introduce multi queue support.
The files are organized in such a way that code that can be used by different VDPA implementations will be placed in a common are resides in
drivers/vdpa/mlx5/core.Only virtual functions are currently supported. Also, certain firmware capabilities must be set to enable the driver. Physical functions (PFs) are skipped by the driver.
To make use of the VDPA net driver, one must load
mlx5_vdpa. In such case, VFs will be operated by the VDPA driver. Although one can see a regular instance of a network driver on the VF, the VDPA driver takes precedence over the NIC driver, steering-wize.Currently, the device/interface infrastructure in
mlx5_coreis used to probe drivers. Future patches will introduce virtbus as a means to register devices and drivers and VDPA will be adapted to it.The mlx5 mode of operation required to support VDPA is switchdev mode. Once can use Linux or OVS bridge to take care of layer 2 switching.
In order to provide virtio networking to a guest, an updated version of qemu is required. This version has been tested by the following quemu version:
url: https://github.com/jasowang/qemu.git
branch: vdpa
Commit ID:6f4e59b807db
V2->V3
Fix makefile to use include path relative to the root of the kernelV3-V4
Rebase Jason's patches on linux-next branch
Fix krobot error on mips arch
Make use of the free callback to destroy resoruces on unload UseVIRTIO_F_ACCESS_PLATFORMinstead of legacyVIRTIO_F_IOMMU_PLATFORMAdd empty implementations forget_vq_notification()andget_vq_irq()Eli Cohen (6):
net/vdpa: Use struct for set/get vq state
vdpa: Modifyget_vq_state()to return error code vdpa/mlx5: Add hardware descriptive header file vdpa/mlx5: Add support library for mlx5 VDPA implementation vdpa/mlx5: Add shared memory registration code vdpa/mlx5: Add VDPA driver for supported mlx5 devicesJason Wang (5):
vhost-vdpa: refine ioctl pre-processing
vhost: generialize backend features setting/getting
vhost-vdpa: support get/set backend features
vhost-vdpa: support IOTLB batching hints
vdpasim: support batch updatingMax Gurtovoy (1):
vdpa: remove hard coded virtq numdrivers/vdpa/Kconfig | 19 + drivers/vdpa/Makefile | 1 + drivers/vdpa/ifcvf/ifcvf_base.c | 4 +- drivers/vdpa/ifcvf/ifcvf_base.h | 4 +- drivers/vdpa/ifcvf/ifcvf_main.c | 13 +- drivers/vdpa/mlx5/Makefile | 4 + drivers/vdpa/mlx5/core/mlx5_vdpa.h | 91 ++ drivers/vdpa/mlx5/core/mlx5_vdpa_ifc.h | 168 ++ drivers/vdpa/mlx5/core/mr.c | 484 ++++++ drivers/vdpa/mlx5/core/resources.c | 284 ++++ drivers/vdpa/mlx5/net/main.c | 76 + drivers/vdpa/mlx5/net/mlx5_vnet.c | 1965 ++++++++++++++++++++++++ drivers/vdpa/mlx5/net/mlx5_vnet.h | 24 + drivers/vdpa/vdpa.c | 3 + drivers/vdpa/vdpa_sim/vdpa_sim.c | 53 +- drivers/vhost/net.c | 18 +- drivers/vhost/vdpa.c | 76 +- drivers/vhost/vhost.c | 15 + drivers/vhost/vhost.h | 2 + include/linux/vdpa.h | 24 +- include/uapi/linux/vhost.h | 2 + include/uapi/linux/vhost_types.h | 11 + 22 files changed, 3284 insertions(+), 57 deletions(-)create mode 100644
drivers/vdpa/mlx5/Makefilecreate mode 100644drivers/vdpa/mlx5/core/mlx5_vdpa.hcreate mode 100644drivers/vdpa/mlx5/core/mlx5_vdpa_ifc.hcreate mode 100644drivers/vdpa/mlx5/core/mr.ccreate mode 100644drivers/vdpa/mlx5/core/resources.ccreate mode 100644drivers/vdpa/mlx5/net/main.ccreate mode 100644drivers/vdpa/mlx5/net/mlx5_vnet.ccreate mode 100644drivers/vdpa/mlx5/net/mlx5_vnet.hSignature · 1 line
2.26.0
_______________________________________________
Virtualization mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/virtualization
Quoted 9 lines · Jason Wang
Quoted 3 lines
BTW, for some reason I did not get all the patches into my mailbox and I suspect they were not all sent. Did you get all the series 0-13?
I can see patch 0 to patch 12 but not patch 13 (I guess 12 is all).
Thanks
Yes, I meant 13 patches 0-12.
Thanks
Quoted 11 lines · Eli Cohen
Hi Michael,
please note that this series depends on mlx5 core device driver patches in mlx5-next branch ingit://git.kernel.org/pub/scm/linux/kernel/git/mellanox/linux.git.git pull
git://git.kernel.org/pub/scm/linux/kernel/git/mellanox/linux.gitmlx5-nextThey also depend Jason Wang's patches:https://lkml.org/lkml/2020/7/1/301
Jason, I had to resolve some conflicts so I would appreciate of you can verify that it is ok.
Looks good to me.
Thanks
_______________________________________________
Virtualization mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/virtualization
Michael S. Tsirkin
maintainerQuoted 8 lines · Eli Cohen
Hi Michael,
please note that this series depends on mlx5 core device driver patches in mlx5-next branch ingit://git.kernel.org/pub/scm/linux/kernel/git/mellanox/linux.git.git pull
git://git.kernel.org/pub/scm/linux/kernel/git/mellanox/linux.gitmlx5-nextThey also depend Jason Wang's patches: https://lkml.org/lkml/2020/7/1/301
So if I apply this to linux-next branch of my tree, I get:
CALL scripts/checksyscalls.sh
CALL scripts/atomic/check-atomics.sh
DESCEND objtool
CHK include/generated/compile.h
CC [M] drivers/vdpa/mlx5/net/main.o
In file included from ./include/linux/swab.h:5,
from ./include/uapi/linux/byteorder/little_endian.h:13,
from ./include/linux/byteorder/little_endian.h:5,
from ./arch/x86/include/uapi/asm/byteorder.h:5,
from ./include/asm-generic/bitops/le.h:6,
from ./arch/x86/include/asm/bitops.h:395,
from ./include/linux/bitops.h:29,
from ./include/linux/kernel.h:12,
from ./include/linux/list.h:9,
from ./include/linux/module.h:12,
from drivers/vdpa/mlx5/net/main.c:4:
drivers/vdpa/mlx5/net/main.c: In function ‘required_caps_supported’:
././include/linux/compiler_types.h:129:35: error: ‘struct mlx5_ifc_device_virtio_emulation_cap_bits’ has no member named ‘event_mode’
129 | #define __compiler_offsetof(a, b) __builtin_offsetof(a, b)
| ^~~~~~~~~~~~~~~~~~./include/uapi/linux/swab.h:115:54: note: in definition of macro ‘__swab32’
115 | #define __swab32(x) (__u32)__builtin_bswap32((__u32)(x))
| ^./include/linux/byteorder/generic.h:95:21: note: in expansion of macro ‘__be32_to_cpu’
95 | #define be32_to_cpu __be32_to_cpu
| ^~~~~~~~~~~~~
./include/linux/stddef.h:17:32: note: in expansion of macro ‘__compiler_offsetof’
17 | #define offsetof(TYPE, MEMBER) __compiler_offsetof(TYPE, MEMBER)
| ^~~~~~~~~~~~~~~~~~~./include/linux/mlx5/device.h:51:35: note: in expansion of macro ‘offsetof’
51 | #define __mlx5_bit_off(typ, fld) (offsetof(struct mlx5_ifc_##typ##_bits, fld))
| ^~~~~~~~./include/linux/mlx5/device.h:53:34: note: in expansion of macro ‘__mlx5_bit_off’
53 | #define __mlx5_dw_off(typ, fld) (__mlx5_bit_off(typ, fld) / 32)
| ^~~~~~~~~~~~~~
./include/linux/mlx5/device.h:96:1: note: in expansion of macro ‘__mlx5_dw_off’
96 | __mlx5_dw_off(typ, fld))) >> __mlx5_dw_bit_off(typ, fld)) & \
| ^~~~~~~~~~~~~
./include/linux/mlx5/device.h:1355:2: note: in expansion of macro ‘MLX5_GET’
1355 | MLX5_GET(device_virtio_emulation_cap, \
| ^~~~~~~~
drivers/vdpa/mlx5/net/main.c:24:15: note: in expansion of macro ‘MLX5_CAP_DEV_VDPA_EMULATION’
24 | event_mode = MLX5_CAP_DEV_VDPA_EMULATION(mdev, event_mode);
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~In file included from ./include/linux/mlx5/driver.h:52,
from drivers/vdpa/mlx5/net/main.c:5:./include/linux/mlx5/device.h:50:57: error: ‘struct mlx5_ifc_device_virtio_emulation_cap_bits’ has no member named ‘event_mode’
50 | #define __mlx5_bit_sz(typ, fld) sizeof(__mlx5_nullp(typ)->fld)
| ^~./include/linux/mlx5/device.h:56:43: note: in expansion of macro ‘__mlx5_bit_sz’
56 | #define __mlx5_dw_bit_off(typ, fld) (32 - __mlx5_bit_sz(typ, fld) - (__mlx5_bit_off(typ, fld) & 0x1f))
| ^~~~~~~~~~~~~./include/linux/mlx5/device.h:96:30: note: in expansion of macro ‘__mlx5_dw_bit_off’
96 | __mlx5_dw_off(typ, fld))) >> __mlx5_dw_bit_off(typ, fld)) & \
| ^~~~~~~~~~~~~~~~~
./include/linux/mlx5/device.h:1355:2: note: in expansion of macro ‘MLX5_GET’
1355 | MLX5_GET(device_virtio_emulation_cap, \
| ^~~~~~~~
drivers/vdpa/mlx5/net/main.c:24:15: note: in expansion of macro ‘MLX5_CAP_DEV_VDPA_EMULATION’
24 | event_mode = MLX5_CAP_DEV_VDPA_EMULATION(mdev, event_mode);
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~
In file included from <command-line>:
././include/linux/compiler_types.h:129:35: error: ‘struct mlx5_ifc_device_virtio_emulation_cap_bits’ has no member named ‘event_mode’
129 | #define __compiler_offsetof(a, b) __builtin_offsetof(a, b)
| ^~~~~~~~~~~~~~~~~~
./include/linux/stddef.h:17:32: note: in expansion of macro ‘__compiler_offsetof’
17 | #define offsetof(TYPE, MEMBER) __compiler_offsetof(TYPE, MEMBER)
| ^~~~~~~~~~~~~~~~~~~./include/linux/mlx5/device.h:51:35: note: in expansion of macro ‘offsetof’
51 | #define __mlx5_bit_off(typ, fld) (offsetof(struct mlx5_ifc_##typ##_bits, fld))
| ^~~~~~~~./include/linux/mlx5/device.h:56:70: note: in expansion of macro ‘__mlx5_bit_off’
56 | #define __mlx5_dw_bit_off(typ, fld) (32 - __mlx5_bit_sz(typ, fld) - (__mlx5_bit_off(typ, fld) & 0x1f))
| ^~~~~~~~~~~~~~./include/linux/mlx5/device.h:96:30: note: in expansion of macro ‘__mlx5_dw_bit_off’
96 | __mlx5_dw_off(typ, fld))) >> __mlx5_dw_bit_off(typ, fld)) & \
| ^~~~~~~~~~~~~~~~~
./include/linux/mlx5/device.h:1355:2: note: in expansion of macro ‘MLX5_GET’
1355 | MLX5_GET(device_virtio_emulation_cap, \
| ^~~~~~~~
drivers/vdpa/mlx5/net/main.c:24:15: note: in expansion of macro ‘MLX5_CAP_DEV_VDPA_EMULATION’
24 | event_mode = MLX5_CAP_DEV_VDPA_EMULATION(mdev, event_mode);
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~In file included from ./include/linux/mlx5/driver.h:52,
from drivers/vdpa/mlx5/net/main.c:5:./include/linux/mlx5/device.h:50:57: error: ‘struct mlx5_ifc_device_virtio_emulation_cap_bits’ has no member named ‘event_mode’
50 | #define __mlx5_bit_sz(typ, fld) sizeof(__mlx5_nullp(typ)->fld)
| ^~./include/linux/mlx5/device.h:57:47: note: in expansion of macro ‘__mlx5_bit_sz’
57 | #define __mlx5_mask(typ, fld) ((u32)((1ull << __mlx5_bit_sz(typ, fld)) - 1))
| ^~~~~~~~~~~~~./include/linux/mlx5/device.h:97:1: note: in expansion of macro ‘__mlx5_mask’
97 | __mlx5_mask(typ, fld))
| ^~~~~~~~~~~
./include/linux/mlx5/device.h:1355:2: note: in expansion of macro ‘MLX5_GET’
1355 | MLX5_GET(device_virtio_emulation_cap, \
| ^~~~~~~~
drivers/vdpa/mlx5/net/main.c:24:15: note: in expansion of macro ‘MLX5_CAP_DEV_VDPA_EMULATION’
24 | event_mode = MLX5_CAP_DEV_VDPA_EMULATION(mdev, event_mode);
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~
In file included from ./include/linux/swab.h:5,
from ./include/uapi/linux/byteorder/little_endian.h:13,
from ./include/linux/byteorder/little_endian.h:5,
from ./arch/x86/include/uapi/asm/byteorder.h:5,
from ./include/asm-generic/bitops/le.h:6,
from ./arch/x86/include/asm/bitops.h:395,
from ./include/linux/bitops.h:29,
from ./include/linux/kernel.h:12,
from ./include/linux/list.h:9,
from ./include/linux/module.h:12,
from drivers/vdpa/mlx5/net/main.c:4:
././include/linux/compiler_types.h:129:35: error: ‘struct mlx5_ifc_device_virtio_emulation_cap_bits’ has no member named ‘eth_frame_offload_type’
129 | #define __compiler_offsetof(a, b) __builtin_offsetof(a, b)
| ^~~~~~~~~~~~~~~~~~./include/uapi/linux/swab.h:115:54: note: in definition of macro ‘__swab32’
115 | #define __swab32(x) (__u32)__builtin_bswap32((__u32)(x))
| ^./include/linux/byteorder/generic.h:95:21: note: in expansion of macro ‘__be32_to_cpu’
95 | #define be32_to_cpu __be32_to_cpu
| ^~~~~~~~~~~~~
./include/linux/stddef.h:17:32: note: in expansion of macro ‘__compiler_offsetof’
17 | #define offsetof(TYPE, MEMBER) __compiler_offsetof(TYPE, MEMBER)
| ^~~~~~~~~~~~~~~~~~~./include/linux/mlx5/device.h:51:35: note: in expansion of macro ‘offsetof’
51 | #define __mlx5_bit_off(typ, fld) (offsetof(struct mlx5_ifc_##typ##_bits, fld))
| ^~~~~~~~./include/linux/mlx5/device.h:53:34: note: in expansion of macro ‘__mlx5_bit_off’
53 | #define __mlx5_dw_off(typ, fld) (__mlx5_bit_off(typ, fld) / 32)
| ^~~~~~~~~~~~~~
./include/linux/mlx5/device.h:96:1: note: in expansion of macro ‘__mlx5_dw_off’
96 | __mlx5_dw_off(typ, fld))) >> __mlx5_dw_bit_off(typ, fld)) & \
| ^~~~~~~~~~~~~
./include/linux/mlx5/device.h:1355:2: note: in expansion of macro ‘MLX5_GET’
1355 | MLX5_GET(device_virtio_emulation_cap, \
| ^~~~~~~~
drivers/vdpa/mlx5/net/main.c:28:7: note: in expansion of macro ‘MLX5_CAP_DEV_VDPA_EMULATION’
28 | if (!MLX5_CAP_DEV_VDPA_EMULATION(mdev, eth_frame_offload_type))
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~In file included from ./include/linux/mlx5/driver.h:52,
from drivers/vdpa/mlx5/net/main.c:5:./include/linux/mlx5/device.h:50:57: error: ‘struct mlx5_ifc_device_virtio_emulation_cap_bits’ has no member named ‘eth_frame_offload_type’
50 | #define __mlx5_bit_sz(typ, fld) sizeof(__mlx5_nullp(typ)->fld)
| ^~./include/linux/mlx5/device.h:56:43: note: in expansion of macro ‘__mlx5_bit_sz’
56 | #define __mlx5_dw_bit_off(typ, fld) (32 - __mlx5_bit_sz(typ, fld) - (__mlx5_bit_off(typ, fld) & 0x1f))
| ^~~~~~~~~~~~~./include/linux/mlx5/device.h:96:30: note: in expansion of macro ‘__mlx5_dw_bit_off’
96 | __mlx5_dw_off(typ, fld))) >> __mlx5_dw_bit_off(typ, fld)) & \
| ^~~~~~~~~~~~~~~~~
./include/linux/mlx5/device.h:1355:2: note: in expansion of macro ‘MLX5_GET’
1355 | MLX5_GET(device_virtio_emulation_cap, \
| ^~~~~~~~
drivers/vdpa/mlx5/net/main.c:28:7: note: in expansion of macro ‘MLX5_CAP_DEV_VDPA_EMULATION’
28 | if (!MLX5_CAP_DEV_VDPA_EMULATION(mdev, eth_frame_offload_type))
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~
In file included from <command-line>:
././include/linux/compiler_types.h:129:35: error: ‘struct mlx5_ifc_device_virtio_emulation_cap_bits’ has no member named ‘eth_frame_offload_type’
129 | #define __compiler_offsetof(a, b) __builtin_offsetof(a, b)
| ^~~~~~~~~~~~~~~~~~
./include/linux/stddef.h:17:32: note: in expansion of macro ‘__compiler_offsetof’
17 | #define offsetof(TYPE, MEMBER) __compiler_offsetof(TYPE, MEMBER)
| ^~~~~~~~~~~~~~~~~~~./include/linux/mlx5/device.h:51:35: note: in expansion of macro ‘offsetof’
51 | #define __mlx5_bit_off(typ, fld) (offsetof(struct mlx5_ifc_##typ##_bits, fld))
| ^~~~~~~~./include/linux/mlx5/device.h:56:70: note: in expansion of macro ‘__mlx5_bit_off’
56 | #define __mlx5_dw_bit_off(typ, fld) (32 - __mlx5_bit_sz(typ, fld) - (__mlx5_bit_off(typ, fld) & 0x1f))
| ^~~~~~~~~~~~~~./include/linux/mlx5/device.h:96:30: note: in expansion of macro ‘__mlx5_dw_bit_off’
96 | __mlx5_dw_off(typ, fld))) >> __mlx5_dw_bit_off(typ, fld)) & \
| ^~~~~~~~~~~~~~~~~
./include/linux/mlx5/device.h:1355:2: note: in expansion of macro ‘MLX5_GET’
1355 | MLX5_GET(device_virtio_emulation_cap, \
| ^~~~~~~~
drivers/vdpa/mlx5/net/main.c:28:7: note: in expansion of macro ‘MLX5_CAP_DEV_VDPA_EMULATION’
28 | if (!MLX5_CAP_DEV_VDPA_EMULATION(mdev, eth_frame_offload_type))
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~In file included from ./include/linux/mlx5/driver.h:52,
from drivers/vdpa/mlx5/net/main.c:5:./include/linux/mlx5/device.h:50:57: error: ‘struct mlx5_ifc_device_virtio_emulation_cap_bits’ has no member named ‘eth_frame_offload_type’
50 | #define __mlx5_bit_sz(typ, fld) sizeof(__mlx5_nullp(typ)->fld)
| ^~./include/linux/mlx5/device.h:57:47: note: in expansion of macro ‘__mlx5_bit_sz’
57 | #define __mlx5_mask(typ, fld) ((u32)((1ull << __mlx5_bit_sz(typ, fld)) - 1))
| ^~~~~~~~~~~~~./include/linux/mlx5/device.h:97:1: note: in expansion of macro ‘__mlx5_mask’
97 | __mlx5_mask(typ, fld))
| ^~~~~~~~~~~
./include/linux/mlx5/device.h:1355:2: note: in expansion of macro ‘MLX5_GET’
1355 | MLX5_GET(device_virtio_emulation_cap, \
| ^~~~~~~~
drivers/vdpa/mlx5/net/main.c:28:7: note: in expansion of macro ‘MLX5_CAP_DEV_VDPA_EMULATION’
28 | if (!MLX5_CAP_DEV_VDPA_EMULATION(mdev, eth_frame_offload_type))
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~
drivers/vdpa/mlx5/net/main.c: At top level:
drivers/vdpa/mlx5/net/main.c:62:14: error: ‘MLX5_INTERFACE_PROTOCOL_VDPA’ undeclared here (not in a function); did you mean ‘MLX5_INTERFACE_PROTOCOL_ETH’?
62 | .protocol = MLX5_INTERFACE_PROTOCOL_VDPA,
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~
| MLX5_INTERFACE_PROTOCOL_ETHmake[3]: *** [scripts/Makefile.build:281: drivers/vdpa/mlx5/net/main.o] Error 1 make[2]: *** [scripts/Makefile.build:497: drivers/vdpa/mlx5] Error 2 make[1]: *** [scripts/Makefile.build:497: drivers/vdpa] Error 2
make: *** [Makefile:1756: drivers] Error 2
I am guessing this is because of the missing dependency, right? So what's the plan for merging this?
Signature · 6 lines
MST _______________________________________________ Virtualization mailing list [email protected] https://lists.linuxfoundation.org/mailman/listinfo/virtualization
Quoted 6 lines · Michael S. Tsirkin
Quoted 5 lines · Eli Cohen
Hi Michael,
please note that this series depends on mlx5 core device driver patches in mlx5-next branch ingit://git.kernel.org/pub/scm/linux/kernel/git/mellanox/linux.git.
Quoted 6 lines
Quoted 3 lines
git pull
git://git.kernel.org/pub/scm/linux/kernel/git/mellanox/linux.gitmlx5-nextThey also depend Jason Wang's patches: https://lkml.org/lkml/2020/7/1/301
So if I apply this to linux-next branch of my tree, I get:
Did you merge this?:
git pull git://git.kernel.org/pub/scm/linux/kernel/git/mellanox/linux.git mlx5-next
Quoted 213 lines
CALL
scripts/checksyscalls.sh
CALLscripts/atomic/check-atomics.sh
DESCEND objtool
CHKinclude/generated/compile.h
CC [M]drivers/vdpa/mlx5/net/main.o
In file included from./include/linux/swab.h:5,from ./include/uapi/linux/byteorder/little_endian.h:13, from ./include/linux/byteorder/little_endian.h:5, from ./arch/x86/include/uapi/asm/byteorder.h:5, from ./include/asm-generic/bitops/le.h:6, from ./arch/x86/include/asm/bitops.h:395, from ./include/linux/bitops.h:29, from ./include/linux/kernel.h:12, from ./include/linux/list.h:9, from ./include/linux/module.h:12, from drivers/vdpa/mlx5/net/main.c:4: drivers/vdpa/mlx5/net/main.c: In function ‘required_caps_supported’: ././include/linux/compiler_types.h:129:35: error: ‘struct mlx5_ifc_device_virtio_emulation_cap_bits’ has no member named ‘event_mode’ 129 | #define __compiler_offsetof(a, b) __builtin_offsetof(a, b) | ^~~~~~~~~~~~~~~~~~
./include/uapi/linux/swab.h:115:54: note: in definition of macro ‘__swab32’
115 | #define__swab32(x)(__u32)__builtin_bswap32((__u32)(x))
| ^./include/linux/byteorder/generic.h:95:21: note: in expansion of macro ‘__be32_to_cpu’
95 | #definebe32_to_cpu__be32_to_cpu| ^~~~~~~~~~~~~ ./include/linux/stddef.h:17:32: note: in expansion of macro ‘__compiler_offsetof’ 17 | #define offsetof(TYPE, MEMBER) __compiler_offsetof(TYPE, MEMBER) | ^~~~~~~~~~~~~~~~~~~
./include/linux/mlx5/device.h:51:35: note: in expansion of macro ‘offsetof’
51 | #define__mlx5_bit_off(typ, fld)(offsetof(struct mlx5_ifc_##typ##_bits, fld))
| ^~~~~~~~./include/linux/mlx5/device.h:53:34: note: in expansion of macro ‘__mlx5_bit_off’
53 | #define__mlx5_dw_off(typ, fld)(__mlx5_bit_off(typ, fld)/ 32)| ^~~~~~~~~~~~~~ ./include/linux/mlx5/device.h:96:1: note: in expansion of macro ‘__mlx5_dw_off’ 96 | __mlx5_dw_off(typ, fld))) >> __mlx5_dw_bit_off(typ, fld)) & \ | ^~~~~~~~~~~~~ ./include/linux/mlx5/device.h:1355:2: note: in expansion of macro ‘MLX5_GET’ 1355 | MLX5_GET(device_virtio_emulation_cap, \ | ^~~~~~~~ drivers/vdpa/mlx5/net/main.c:24:15: note: in expansion of macro ‘MLX5_CAP_DEV_VDPA_EMULATION’ 24 | event_mode = MLX5_CAP_DEV_VDPA_EMULATION(mdev, event_mode); | ^~~~~~~~~~~~~~~~~~~~~~~~~~~In file included from
./include/linux/mlx5/driver.h:52,
fromdrivers/vdpa/mlx5/net/main.c:5:./include/linux/mlx5/device.h:50:57: error: ‘struct mlx5_ifc_device_virtio_emulation_cap_bits’ has no member named ‘event_mode’
50 | #define__mlx5_bit_sz(typ, fld)sizeof(__mlx5_nullp(typ)->fld)
| ^~./include/linux/mlx5/device.h:56:43: note: in expansion of macro ‘__mlx5_bit_sz’
56 | #define__mlx5_dw_bit_off(typ, fld)(32 -__mlx5_bit_sz(typ, fld)- (__mlx5_bit_off(typ, fld)&0x1f))
| ^~~~~~~~~~~~~./include/linux/mlx5/device.h:96:30: note: in expansion of macro ‘__mlx5_dw_bit_off’96 | __mlx5_dw_off(typ, fld))) >> __mlx5_dw_bit_off(typ, fld)) & \ | ^~~~~~~~~~~~~~~~~ ./include/linux/mlx5/device.h:1355:2: note: in expansion of macro ‘MLX5_GET’ 1355 | MLX5_GET(device_virtio_emulation_cap, \ | ^~~~~~~~ drivers/vdpa/mlx5/net/main.c:24:15: note: in expansion of macro ‘MLX5_CAP_DEV_VDPA_EMULATION’ 24 | event_mode = MLX5_CAP_DEV_VDPA_EMULATION(mdev, event_mode); | ^~~~~~~~~~~~~~~~~~~~~~~~~~~ In file included from <command-line>: ././include/linux/compiler_types.h:129:35: error: ‘struct mlx5_ifc_device_virtio_emulation_cap_bits’ has no member named ‘event_mode’ 129 | #define __compiler_offsetof(a, b) __builtin_offsetof(a, b) | ^~~~~~~~~~~~~~~~~~ ./include/linux/stddef.h:17:32: note: in expansion of macro ‘__compiler_offsetof’ 17 | #define offsetof(TYPE, MEMBER) __compiler_offsetof(TYPE, MEMBER) | ^~~~~~~~~~~~~~~~~~~
./include/linux/mlx5/device.h:51:35: note: in expansion of macro ‘offsetof’
51 | #define__mlx5_bit_off(typ, fld)(offsetof(struct mlx5_ifc_##typ##_bits, fld))
| ^~~~~~~~./include/linux/mlx5/device.h:56:70: note: in expansion of macro ‘__mlx5_bit_off’
56 | #define__mlx5_dw_bit_off(typ, fld)(32 -__mlx5_bit_sz(typ, fld)- (__mlx5_bit_off(typ, fld)&0x1f))
| ^~~~~~~~~~~~~~./include/linux/mlx5/device.h:96:30: note: in expansion of macro ‘__mlx5_dw_bit_off’96 | __mlx5_dw_off(typ, fld))) >> __mlx5_dw_bit_off(typ, fld)) & \ | ^~~~~~~~~~~~~~~~~ ./include/linux/mlx5/device.h:1355:2: note: in expansion of macro ‘MLX5_GET’ 1355 | MLX5_GET(device_virtio_emulation_cap, \ | ^~~~~~~~ drivers/vdpa/mlx5/net/main.c:24:15: note: in expansion of macro ‘MLX5_CAP_DEV_VDPA_EMULATION’ 24 | event_mode = MLX5_CAP_DEV_VDPA_EMULATION(mdev, event_mode); | ^~~~~~~~~~~~~~~~~~~~~~~~~~~In file included from
./include/linux/mlx5/driver.h:52,
fromdrivers/vdpa/mlx5/net/main.c:5:./include/linux/mlx5/device.h:50:57: error: ‘struct mlx5_ifc_device_virtio_emulation_cap_bits’ has no member named ‘event_mode’
50 | #define__mlx5_bit_sz(typ, fld)sizeof(__mlx5_nullp(typ)->fld)
| ^~./include/linux/mlx5/device.h:57:47: note: in expansion of macro ‘__mlx5_bit_sz’
57 | #define__mlx5_mask(typ, fld)((u32)((1ull <<__mlx5_bit_sz(typ, fld)) - 1))
| ^~~~~~~~~~~~~./include/linux/mlx5/device.h:97:1: note: in expansion of macro ‘__mlx5_mask’
97 |__mlx5_mask(typ, fld))| ^~~~~~~~~~~ ./include/linux/mlx5/device.h:1355:2: note: in expansion of macro ‘MLX5_GET’ 1355 | MLX5_GET(device_virtio_emulation_cap, \ | ^~~~~~~~ drivers/vdpa/mlx5/net/main.c:24:15: note: in expansion of macro ‘MLX5_CAP_DEV_VDPA_EMULATION’ 24 | event_mode = MLX5_CAP_DEV_VDPA_EMULATION(mdev, event_mode); | ^~~~~~~~~~~~~~~~~~~~~~~~~~~ In file included from ./include/linux/swab.h:5, from ./include/uapi/linux/byteorder/little_endian.h:13, from ./include/linux/byteorder/little_endian.h:5, from ./arch/x86/include/uapi/asm/byteorder.h:5, from ./include/asm-generic/bitops/le.h:6, from ./arch/x86/include/asm/bitops.h:395, from ./include/linux/bitops.h:29, from ./include/linux/kernel.h:12, from ./include/linux/list.h:9, from ./include/linux/module.h:12, from drivers/vdpa/mlx5/net/main.c:4: ././include/linux/compiler_types.h:129:35: error: ‘struct mlx5_ifc_device_virtio_emulation_cap_bits’ has no member named ‘eth_frame_offload_type’ 129 | #define __compiler_offsetof(a, b) __builtin_offsetof(a, b) | ^~~~~~~~~~~~~~~~~~
./include/uapi/linux/swab.h:115:54: note: in definition of macro ‘__swab32’
115 | #define__swab32(x)(__u32)__builtin_bswap32((__u32)(x))
| ^./include/linux/byteorder/generic.h:95:21: note: in expansion of macro ‘__be32_to_cpu’
95 | #definebe32_to_cpu__be32_to_cpu| ^~~~~~~~~~~~~ ./include/linux/stddef.h:17:32: note: in expansion of macro ‘__compiler_offsetof’ 17 | #define offsetof(TYPE, MEMBER) __compiler_offsetof(TYPE, MEMBER) | ^~~~~~~~~~~~~~~~~~~
./include/linux/mlx5/device.h:51:35: note: in expansion of macro ‘offsetof’
51 | #define__mlx5_bit_off(typ, fld)(offsetof(struct mlx5_ifc_##typ##_bits, fld))
| ^~~~~~~~./include/linux/mlx5/device.h:53:34: note: in expansion of macro ‘__mlx5_bit_off’
53 | #define__mlx5_dw_off(typ, fld)(__mlx5_bit_off(typ, fld)/ 32)| ^~~~~~~~~~~~~~ ./include/linux/mlx5/device.h:96:1: note: in expansion of macro ‘__mlx5_dw_off’ 96 | __mlx5_dw_off(typ, fld))) >> __mlx5_dw_bit_off(typ, fld)) & \ | ^~~~~~~~~~~~~ ./include/linux/mlx5/device.h:1355:2: note: in expansion of macro ‘MLX5_GET’ 1355 | MLX5_GET(device_virtio_emulation_cap, \ | ^~~~~~~~ drivers/vdpa/mlx5/net/main.c:28:7: note: in expansion of macro ‘MLX5_CAP_DEV_VDPA_EMULATION’ 28 | if (!MLX5_CAP_DEV_VDPA_EMULATION(mdev, eth_frame_offload_type)) | ^~~~~~~~~~~~~~~~~~~~~~~~~~~In file included from
./include/linux/mlx5/driver.h:52,
fromdrivers/vdpa/mlx5/net/main.c:5:./include/linux/mlx5/device.h:50:57: error: ‘struct mlx5_ifc_device_virtio_emulation_cap_bits’ has no member named ‘eth_frame_offload_type’
50 | #define__mlx5_bit_sz(typ, fld)sizeof(__mlx5_nullp(typ)->fld)
| ^~./include/linux/mlx5/device.h:56:43: note: in expansion of macro ‘__mlx5_bit_sz’
56 | #define__mlx5_dw_bit_off(typ, fld)(32 -__mlx5_bit_sz(typ, fld)- (__mlx5_bit_off(typ, fld)&0x1f))
| ^~~~~~~~~~~~~./include/linux/mlx5/device.h:96:30: note: in expansion of macro ‘__mlx5_dw_bit_off’96 | __mlx5_dw_off(typ, fld))) >> __mlx5_dw_bit_off(typ, fld)) & \ | ^~~~~~~~~~~~~~~~~ ./include/linux/mlx5/device.h:1355:2: note: in expansion of macro ‘MLX5_GET’ 1355 | MLX5_GET(device_virtio_emulation_cap, \ | ^~~~~~~~ drivers/vdpa/mlx5/net/main.c:28:7: note: in expansion of macro ‘MLX5_CAP_DEV_VDPA_EMULATION’ 28 | if (!MLX5_CAP_DEV_VDPA_EMULATION(mdev, eth_frame_offload_type)) | ^~~~~~~~~~~~~~~~~~~~~~~~~~~ In file included from <command-line>: ././include/linux/compiler_types.h:129:35: error: ‘struct mlx5_ifc_device_virtio_emulation_cap_bits’ has no member named ‘eth_frame_offload_type’ 129 | #define __compiler_offsetof(a, b) __builtin_offsetof(a, b) | ^~~~~~~~~~~~~~~~~~ ./include/linux/stddef.h:17:32: note: in expansion of macro ‘__compiler_offsetof’ 17 | #define offsetof(TYPE, MEMBER) __compiler_offsetof(TYPE, MEMBER) | ^~~~~~~~~~~~~~~~~~~
./include/linux/mlx5/device.h:51:35: note: in expansion of macro ‘offsetof’
51 | #define__mlx5_bit_off(typ, fld)(offsetof(struct mlx5_ifc_##typ##_bits, fld))
| ^~~~~~~~./include/linux/mlx5/device.h:56:70: note: in expansion of macro ‘__mlx5_bit_off’
56 | #define__mlx5_dw_bit_off(typ, fld)(32 -__mlx5_bit_sz(typ, fld)- (__mlx5_bit_off(typ, fld)&0x1f))
| ^~~~~~~~~~~~~~./include/linux/mlx5/device.h:96:30: note: in expansion of macro ‘__mlx5_dw_bit_off’96 | __mlx5_dw_off(typ, fld))) >> __mlx5_dw_bit_off(typ, fld)) & \ | ^~~~~~~~~~~~~~~~~ ./include/linux/mlx5/device.h:1355:2: note: in expansion of macro ‘MLX5_GET’ 1355 | MLX5_GET(device_virtio_emulation_cap, \ | ^~~~~~~~ drivers/vdpa/mlx5/net/main.c:28:7: note: in expansion of macro ‘MLX5_CAP_DEV_VDPA_EMULATION’ 28 | if (!MLX5_CAP_DEV_VDPA_EMULATION(mdev, eth_frame_offload_type)) | ^~~~~~~~~~~~~~~~~~~~~~~~~~~In file included from
./include/linux/mlx5/driver.h:52,
fromdrivers/vdpa/mlx5/net/main.c:5:./include/linux/mlx5/device.h:50:57: error: ‘struct mlx5_ifc_device_virtio_emulation_cap_bits’ has no member named ‘eth_frame_offload_type’
50 | #define__mlx5_bit_sz(typ, fld)sizeof(__mlx5_nullp(typ)->fld)
| ^~./include/linux/mlx5/device.h:57:47: note: in expansion of macro ‘__mlx5_bit_sz’
57 | #define__mlx5_mask(typ, fld)((u32)((1ull <<__mlx5_bit_sz(typ, fld)) - 1))
| ^~~~~~~~~~~~~./include/linux/mlx5/device.h:97:1: note: in expansion of macro ‘__mlx5_mask’
97 |__mlx5_mask(typ, fld))| ^~~~~~~~~~~ ./include/linux/mlx5/device.h:1355:2: note: in expansion of macro ‘MLX5_GET’ 1355 | MLX5_GET(device_virtio_emulation_cap, \ | ^~~~~~~~ drivers/vdpa/mlx5/net/main.c:28:7: note: in expansion of macro ‘MLX5_CAP_DEV_VDPA_EMULATION’ 28 | if (!MLX5_CAP_DEV_VDPA_EMULATION(mdev, eth_frame_offload_type)) | ^~~~~~~~~~~~~~~~~~~~~~~~~~~ drivers/vdpa/mlx5/net/main.c: At top level: drivers/vdpa/mlx5/net/main.c:62:14: error: ‘MLX5_INTERFACE_PROTOCOL_VDPA’ undeclared here (not in a function); did you mean ‘MLX5_INTERFACE_PROTOCOL_ETH’? 62 | .protocol = MLX5_INTERFACE_PROTOCOL_VDPA, | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~ | MLX5_INTERFACE_PROTOCOL_ETHmake[3]: *** [
scripts/Makefile.build:281:drivers/vdpa/mlx5/net/main.o] Error 1 make[2]: *** [scripts/Makefile.build:497:drivers/vdpa/mlx5] Error 2 make[1]: *** [scripts/Makefile.build:497:drivers/vdpa] Error 2
make: *** [Makefile:1756: drivers] Error 2I am guessing this is because of the missing dependency, right? So what's the plan for merging this?
Signature · 1 line
MST
Michael S. Tsirkin
maintainerQuoted 9 lines · Eli Cohen
Quoted 8 lines · Michael S. Tsirkin
Quoted 7 lines · Eli Cohen
Hi Michael,
please note that this series depends on mlx5 core device driver patches in mlx5-next branch ingit://git.kernel.org/pub/scm/linux/kernel/git/mellanox/linux.git.git pull
git://git.kernel.org/pub/scm/linux/kernel/git/mellanox/linux.gitmlx5-next
Quoted 7 lines
Quoted 4 lines
Quoted 1 line
They also depend Jason Wang's patches: https://lkml.org/lkml/2020/7/1/301
So if I apply this to linux-next branch of my tree, I get:
Did you merge this?:
git pullgit://git.kernel.org/pub/scm/linux/kernel/git/mellanox/linux.gitmlx5-next
I can only merge this tree if no one else will. Linus does not like getting same patches through two trees.
Is this the case? Is mlx5-next going to be merged through my tree in this cycle?
Quoted 206 lines
Quoted 206 lines
CALL
scripts/checksyscalls.sh
CALLscripts/atomic/check-atomics.sh
DESCEND objtool
CHKinclude/generated/compile.h
CC [M]drivers/vdpa/mlx5/net/main.o
In file included from./include/linux/swab.h:5,from ./include/uapi/linux/byteorder/little_endian.h:13, from ./include/linux/byteorder/little_endian.h:5, from ./arch/x86/include/uapi/asm/byteorder.h:5, from ./include/asm-generic/bitops/le.h:6, from ./arch/x86/include/asm/bitops.h:395, from ./include/linux/bitops.h:29, from ./include/linux/kernel.h:12, from ./include/linux/list.h:9, from ./include/linux/module.h:12, from drivers/vdpa/mlx5/net/main.c:4: drivers/vdpa/mlx5/net/main.c: In function ‘required_caps_supported’: ././include/linux/compiler_types.h:129:35: error: ‘struct mlx5_ifc_device_virtio_emulation_cap_bits’ has no member named ‘event_mode’ 129 | #define __compiler_offsetof(a, b) __builtin_offsetof(a, b) | ^~~~~~~~~~~~~~~~~~
./include/uapi/linux/swab.h:115:54: note: in definition of macro ‘__swab32’
115 | #define__swab32(x)(__u32)__builtin_bswap32((__u32)(x))
| ^./include/linux/byteorder/generic.h:95:21: note: in expansion of macro ‘__be32_to_cpu’
95 | #definebe32_to_cpu__be32_to_cpu| ^~~~~~~~~~~~~ ./include/linux/stddef.h:17:32: note: in expansion of macro ‘__compiler_offsetof’ 17 | #define offsetof(TYPE, MEMBER) __compiler_offsetof(TYPE, MEMBER) | ^~~~~~~~~~~~~~~~~~~
./include/linux/mlx5/device.h:51:35: note: in expansion of macro ‘offsetof’
51 | #define__mlx5_bit_off(typ, fld)(offsetof(struct mlx5_ifc_##typ##_bits, fld))
| ^~~~~~~~./include/linux/mlx5/device.h:53:34: note: in expansion of macro ‘__mlx5_bit_off’
53 | #define__mlx5_dw_off(typ, fld)(__mlx5_bit_off(typ, fld)/ 32)| ^~~~~~~~~~~~~~ ./include/linux/mlx5/device.h:96:1: note: in expansion of macro ‘__mlx5_dw_off’ 96 | __mlx5_dw_off(typ, fld))) >> __mlx5_dw_bit_off(typ, fld)) & \ | ^~~~~~~~~~~~~ ./include/linux/mlx5/device.h:1355:2: note: in expansion of macro ‘MLX5_GET’ 1355 | MLX5_GET(device_virtio_emulation_cap, \ | ^~~~~~~~ drivers/vdpa/mlx5/net/main.c:24:15: note: in expansion of macro ‘MLX5_CAP_DEV_VDPA_EMULATION’ 24 | event_mode = MLX5_CAP_DEV_VDPA_EMULATION(mdev, event_mode); | ^~~~~~~~~~~~~~~~~~~~~~~~~~~In file included from
./include/linux/mlx5/driver.h:52,
fromdrivers/vdpa/mlx5/net/main.c:5:./include/linux/mlx5/device.h:50:57: error: ‘struct mlx5_ifc_device_virtio_emulation_cap_bits’ has no member named ‘event_mode’
50 | #define__mlx5_bit_sz(typ, fld)sizeof(__mlx5_nullp(typ)->fld)
| ^~./include/linux/mlx5/device.h:56:43: note: in expansion of macro ‘__mlx5_bit_sz’
56 | #define__mlx5_dw_bit_off(typ, fld)(32 -__mlx5_bit_sz(typ, fld)- (__mlx5_bit_off(typ, fld)&0x1f))
| ^~~~~~~~~~~~~./include/linux/mlx5/device.h:96:30: note: in expansion of macro ‘__mlx5_dw_bit_off’96 | __mlx5_dw_off(typ, fld))) >> __mlx5_dw_bit_off(typ, fld)) & \ | ^~~~~~~~~~~~~~~~~ ./include/linux/mlx5/device.h:1355:2: note: in expansion of macro ‘MLX5_GET’ 1355 | MLX5_GET(device_virtio_emulation_cap, \ | ^~~~~~~~ drivers/vdpa/mlx5/net/main.c:24:15: note: in expansion of macro ‘MLX5_CAP_DEV_VDPA_EMULATION’ 24 | event_mode = MLX5_CAP_DEV_VDPA_EMULATION(mdev, event_mode); | ^~~~~~~~~~~~~~~~~~~~~~~~~~~ In file included from <command-line>: ././include/linux/compiler_types.h:129:35: error: ‘struct mlx5_ifc_device_virtio_emulation_cap_bits’ has no member named ‘event_mode’ 129 | #define __compiler_offsetof(a, b) __builtin_offsetof(a, b) | ^~~~~~~~~~~~~~~~~~ ./include/linux/stddef.h:17:32: note: in expansion of macro ‘__compiler_offsetof’ 17 | #define offsetof(TYPE, MEMBER) __compiler_offsetof(TYPE, MEMBER) | ^~~~~~~~~~~~~~~~~~~
./include/linux/mlx5/device.h:51:35: note: in expansion of macro ‘offsetof’
51 | #define__mlx5_bit_off(typ, fld)(offsetof(struct mlx5_ifc_##typ##_bits, fld))
| ^~~~~~~~./include/linux/mlx5/device.h:56:70: note: in expansion of macro ‘__mlx5_bit_off’
56 | #define__mlx5_dw_bit_off(typ, fld)(32 -__mlx5_bit_sz(typ, fld)- (__mlx5_bit_off(typ, fld)&0x1f))
| ^~~~~~~~~~~~~~./include/linux/mlx5/device.h:96:30: note: in expansion of macro ‘__mlx5_dw_bit_off’96 | __mlx5_dw_off(typ, fld))) >> __mlx5_dw_bit_off(typ, fld)) & \ | ^~~~~~~~~~~~~~~~~ ./include/linux/mlx5/device.h:1355:2: note: in expansion of macro ‘MLX5_GET’ 1355 | MLX5_GET(device_virtio_emulation_cap, \ | ^~~~~~~~ drivers/vdpa/mlx5/net/main.c:24:15: note: in expansion of macro ‘MLX5_CAP_DEV_VDPA_EMULATION’ 24 | event_mode = MLX5_CAP_DEV_VDPA_EMULATION(mdev, event_mode); | ^~~~~~~~~~~~~~~~~~~~~~~~~~~In file included from
./include/linux/mlx5/driver.h:52,
fromdrivers/vdpa/mlx5/net/main.c:5:./include/linux/mlx5/device.h:50:57: error: ‘struct mlx5_ifc_device_virtio_emulation_cap_bits’ has no member named ‘event_mode’
50 | #define__mlx5_bit_sz(typ, fld)sizeof(__mlx5_nullp(typ)->fld)
| ^~./include/linux/mlx5/device.h:57:47: note: in expansion of macro ‘__mlx5_bit_sz’
57 | #define__mlx5_mask(typ, fld)((u32)((1ull <<__mlx5_bit_sz(typ, fld)) - 1))
| ^~~~~~~~~~~~~./include/linux/mlx5/device.h:97:1: note: in expansion of macro ‘__mlx5_mask’
97 |__mlx5_mask(typ, fld))| ^~~~~~~~~~~ ./include/linux/mlx5/device.h:1355:2: note: in expansion of macro ‘MLX5_GET’ 1355 | MLX5_GET(device_virtio_emulation_cap, \ | ^~~~~~~~ drivers/vdpa/mlx5/net/main.c:24:15: note: in expansion of macro ‘MLX5_CAP_DEV_VDPA_EMULATION’ 24 | event_mode = MLX5_CAP_DEV_VDPA_EMULATION(mdev, event_mode); | ^~~~~~~~~~~~~~~~~~~~~~~~~~~ In file included from ./include/linux/swab.h:5, from ./include/uapi/linux/byteorder/little_endian.h:13, from ./include/linux/byteorder/little_endian.h:5, from ./arch/x86/include/uapi/asm/byteorder.h:5, from ./include/asm-generic/bitops/le.h:6, from ./arch/x86/include/asm/bitops.h:395, from ./include/linux/bitops.h:29, from ./include/linux/kernel.h:12, from ./include/linux/list.h:9, from ./include/linux/module.h:12, from drivers/vdpa/mlx5/net/main.c:4: ././include/linux/compiler_types.h:129:35: error: ‘struct mlx5_ifc_device_virtio_emulation_cap_bits’ has no member named ‘eth_frame_offload_type’ 129 | #define __compiler_offsetof(a, b) __builtin_offsetof(a, b) | ^~~~~~~~~~~~~~~~~~
./include/uapi/linux/swab.h:115:54: note: in definition of macro ‘__swab32’
115 | #define__swab32(x)(__u32)__builtin_bswap32((__u32)(x))
| ^./include/linux/byteorder/generic.h:95:21: note: in expansion of macro ‘__be32_to_cpu’
95 | #definebe32_to_cpu__be32_to_cpu| ^~~~~~~~~~~~~ ./include/linux/stddef.h:17:32: note: in expansion of macro ‘__compiler_offsetof’ 17 | #define offsetof(TYPE, MEMBER) __compiler_offsetof(TYPE, MEMBER) | ^~~~~~~~~~~~~~~~~~~
./include/linux/mlx5/device.h:51:35: note: in expansion of macro ‘offsetof’
51 | #define__mlx5_bit_off(typ, fld)(offsetof(struct mlx5_ifc_##typ##_bits, fld))
| ^~~~~~~~./include/linux/mlx5/device.h:53:34: note: in expansion of macro ‘__mlx5_bit_off’
53 | #define__mlx5_dw_off(typ, fld)(__mlx5_bit_off(typ, fld)/ 32)| ^~~~~~~~~~~~~~ ./include/linux/mlx5/device.h:96:1: note: in expansion of macro ‘__mlx5_dw_off’ 96 | __mlx5_dw_off(typ, fld))) >> __mlx5_dw_bit_off(typ, fld)) & \ | ^~~~~~~~~~~~~ ./include/linux/mlx5/device.h:1355:2: note: in expansion of macro ‘MLX5_GET’ 1355 | MLX5_GET(device_virtio_emulation_cap, \ | ^~~~~~~~ drivers/vdpa/mlx5/net/main.c:28:7: note: in expansion of macro ‘MLX5_CAP_DEV_VDPA_EMULATION’ 28 | if (!MLX5_CAP_DEV_VDPA_EMULATION(mdev, eth_frame_offload_type)) | ^~~~~~~~~~~~~~~~~~~~~~~~~~~In file included from
./include/linux/mlx5/driver.h:52,
fromdrivers/vdpa/mlx5/net/main.c:5:./include/linux/mlx5/device.h:50:57: error: ‘struct mlx5_ifc_device_virtio_emulation_cap_bits’ has no member named ‘eth_frame_offload_type’
50 | #define__mlx5_bit_sz(typ, fld)sizeof(__mlx5_nullp(typ)->fld)
| ^~./include/linux/mlx5/device.h:56:43: note: in expansion of macro ‘__mlx5_bit_sz’
56 | #define__mlx5_dw_bit_off(typ, fld)(32 -__mlx5_bit_sz(typ, fld)- (__mlx5_bit_off(typ, fld)&0x1f))
| ^~~~~~~~~~~~~./include/linux/mlx5/device.h:96:30: note: in expansion of macro ‘__mlx5_dw_bit_off’96 | __mlx5_dw_off(typ, fld))) >> __mlx5_dw_bit_off(typ, fld)) & \ | ^~~~~~~~~~~~~~~~~ ./include/linux/mlx5/device.h:1355:2: note: in expansion of macro ‘MLX5_GET’ 1355 | MLX5_GET(device_virtio_emulation_cap, \ | ^~~~~~~~ drivers/vdpa/mlx5/net/main.c:28:7: note: in expansion of macro ‘MLX5_CAP_DEV_VDPA_EMULATION’ 28 | if (!MLX5_CAP_DEV_VDPA_EMULATION(mdev, eth_frame_offload_type)) | ^~~~~~~~~~~~~~~~~~~~~~~~~~~ In file included from <command-line>: ././include/linux/compiler_types.h:129:35: error: ‘struct mlx5_ifc_device_virtio_emulation_cap_bits’ has no member named ‘eth_frame_offload_type’ 129 | #define __compiler_offsetof(a, b) __builtin_offsetof(a, b) | ^~~~~~~~~~~~~~~~~~ ./include/linux/stddef.h:17:32: note: in expansion of macro ‘__compiler_offsetof’ 17 | #define offsetof(TYPE, MEMBER) __compiler_offsetof(TYPE, MEMBER) | ^~~~~~~~~~~~~~~~~~~
./include/linux/mlx5/device.h:51:35: note: in expansion of macro ‘offsetof’
51 | #define__mlx5_bit_off(typ, fld)(offsetof(struct mlx5_ifc_##typ##_bits, fld))
| ^~~~~~~~./include/linux/mlx5/device.h:56:70: note: in expansion of macro ‘__mlx5_bit_off’
56 | #define__mlx5_dw_bit_off(typ, fld)(32 -__mlx5_bit_sz(typ, fld)- (__mlx5_bit_off(typ, fld)&0x1f))
| ^~~~~~~~~~~~~~./include/linux/mlx5/device.h:96:30: note: in expansion of macro ‘__mlx5_dw_bit_off’96 | __mlx5_dw_off(typ, fld))) >> __mlx5_dw_bit_off(typ, fld)) & \ | ^~~~~~~~~~~~~~~~~ ./include/linux/mlx5/device.h:1355:2: note: in expansion of macro ‘MLX5_GET’ 1355 | MLX5_GET(device_virtio_emulation_cap, \ | ^~~~~~~~ drivers/vdpa/mlx5/net/main.c:28:7: note: in expansion of macro ‘MLX5_CAP_DEV_VDPA_EMULATION’ 28 | if (!MLX5_CAP_DEV_VDPA_EMULATION(mdev, eth_frame_offload_type)) | ^~~~~~~~~~~~~~~~~~~~~~~~~~~In file included from
./include/linux/mlx5/driver.h:52,
fromdrivers/vdpa/mlx5/net/main.c:5:./include/linux/mlx5/device.h:50:57: error: ‘struct mlx5_ifc_device_virtio_emulation_cap_bits’ has no member named ‘eth_frame_offload_type’
50 | #define__mlx5_bit_sz(typ, fld)sizeof(__mlx5_nullp(typ)->fld)
| ^~./include/linux/mlx5/device.h:57:47: note: in expansion of macro ‘__mlx5_bit_sz’
57 | #define__mlx5_mask(typ, fld)((u32)((1ull <<__mlx5_bit_sz(typ, fld)) - 1))
| ^~~~~~~~~~~~~./include/linux/mlx5/device.h:97:1: note: in expansion of macro ‘__mlx5_mask’
97 |__mlx5_mask(typ, fld))| ^~~~~~~~~~~ ./include/linux/mlx5/device.h:1355:2: note: in expansion of macro ‘MLX5_GET’ 1355 | MLX5_GET(device_virtio_emulation_cap, \ | ^~~~~~~~ drivers/vdpa/mlx5/net/main.c:28:7: note: in expansion of macro ‘MLX5_CAP_DEV_VDPA_EMULATION’ 28 | if (!MLX5_CAP_DEV_VDPA_EMULATION(mdev, eth_frame_offload_type)) | ^~~~~~~~~~~~~~~~~~~~~~~~~~~ drivers/vdpa/mlx5/net/main.c: At top level: drivers/vdpa/mlx5/net/main.c:62:14: error: ‘MLX5_INTERFACE_PROTOCOL_VDPA’ undeclared here (not in a function); did you mean ‘MLX5_INTERFACE_PROTOCOL_ETH’? 62 | .protocol = MLX5_INTERFACE_PROTOCOL_VDPA, | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~ | MLX5_INTERFACE_PROTOCOL_ETHmake[3]: *** [
scripts/Makefile.build:281:drivers/vdpa/mlx5/net/main.o] Error 1 make[2]: *** [scripts/Makefile.build:497:drivers/vdpa/mlx5] Error 2 make[1]: *** [scripts/Makefile.build:497:drivers/vdpa] Error 2
make: *** [Makefile:1756: drivers] Error 2
Quoted 7 lines
Quoted 7 lines
I am guessing this is because of the missing dependency, right? So what's the plan for merging this?
Signature · 1 line
MST
_______________________________________________
Virtualization mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/virtualization
Quoted 11 lines · Michael S. Tsirkin
Quoted 3 lines
Did you merge this?:
git pullgit://git.kernel.org/pub/scm/linux/kernel/git/mellanox/linux.gitmlx5-nextI can only merge this tree if no one else will. Linus does not like getting same patches through two trees.
Is this the case? Is mlx5-next going to be merged through my tree in this cycle?
Saeed Mahameed from Mellanox (located in California) usuaally sends out net patches. So he's supposed to send that to Dave Miller.
I think Saeed should answer this. Let's wait a few more hours till he wakes up.
Michael S. Tsirkin
maintainerQuoted 13 lines · Eli Cohen
Quoted 11 lines · Michael S. Tsirkin
Quoted 3 lines
Did you merge this?:
git pullgit://git.kernel.org/pub/scm/linux/kernel/git/mellanox/linux.gitmlx5-nextI can only merge this tree if no one else will. Linus does not like getting same patches through two trees.
Is this the case? Is mlx5-next going to be merged through my tree in this cycle?
Quoted 5 lines
Saeed Mahameed from Mellanox (located in California) usuaally sends out net patches. So he's supposed to send that to Dave Miller.
I think Saeed should answer this. Let's wait a few more hours till he wakes up.
Alternatives:
- merge vdpa through Saeed's tree. I can ack that, we'll need to resolve any conflicts by merging the two trees and show the result to Linus so he can resolve the merge in the same way.
- extract just the necessary patches that are needed for vdpa and merge through my tree.
- if Saeed sends his pull today, it's likely it will be merged early next week. Then I can rebase and send a pull with your patches on top. A bit risky.
- do some tricks with build. E.g. disable build of your code, and enable in Saeed's tree when everything is merged together. Can be somewhat hard.
_______________________________________________
Virtualization mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/virtualization
Quoted 23 lines · Michael S. Tsirkin
Quoted 14 lines · Eli Cohen
Quoted 5 lines · Michael S. Tsirkin
Quoted 4 lines
Did you merge this?:
git pullgit://git.kernel.org/pub/scm/linux/kernel/git/mellanox/linux.gi
t mlx5-nextQuoted 7 lines
I can only merge this tree if no one else will. Linus does not like
getting same patches through two trees.Is this the case? Is mlx5-next going to be merged through my tree in this cycle?
Quoted 7 lines
Saeed Mahameed from Mellanox (located in California) usuaally sends out
net patches. So he's supposed to send that to Dave Miller.I think Saeed should answer this. Let's wait a few more hours till he
wakes up.
Quoted 14 lines
Alternatives:
- merge vdpa through Saeed's tree. I can ack that, we'll need to resolve any conflicts by merging the two trees and show the result to Linus so he can resolve the merge in the same way.
- extract just the necessary patches that are needed for vdpa and merge through my tree.
- if Saeed sends his pull today, it's likely it will be merged early next week. Then I can rebase and send a pull with your
patches
on top. A bit risky.
- do some tricks with build. E.g. disable build of your code, and enable in Saeed's tree when everything is merged together. Can be somewhat hard.
Hi Michael,
We do this all the time with net-next and rdma,
mlx5-next is a very small branch based on a very early rc that includes mlx5 shared stuff between rdma and net-next, and now virtio as well.
we send pull requests of mlx5-next to both rdma and net-next with the respective features, exactly as we did here, and it works nicely, since we reduce the number of conflicts to 0 between different subsystems that rely on mlx5 core.
all the alternative you suggested have never been tried before :), net-next is Closed, so i can't do further submissions.
_______________________________________________
Virtualization mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/virtualization
Quoted 10 lines · Saeed Mahameed
Quoted 9 lines · Michael S. Tsirkin
Quoted 8 lines · Eli Cohen
Quoted 7 lines · Michael S. Tsirkin
Quoted 4 lines
Did you merge this?:
git pullgit://git.kernel.org/pub/scm/linux/kernel/git/mellanox/linux.gi
t mlx5-nextI can only merge this tree if no one else will. Linus does not like getting same patches through two trees.
This is not quite the case
Linus does not like multiple *copies* of the same patches. The same actual git commits can be OK.
Linus also does not like unnecessarily cross linking trees, mlx5-next is designed to be small enough and approved enough that it is not controversial.
Linus really doesn't like it when people jams stuff together in rc7 or the weeks of the merge window. He wants to see stuff be in linux-next for at least a bit. So it may be too late regardless.
Quoted 3 lines
We do this all the time with net-next and rdma,
mlx5-next is a very small branch based on a very early rc that includes mlx5 shared stuff between rdma and net-next, and now virtio as well.
Yes, going on two years now? Been working well
Jason
_______________________________________________
Virtualization mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/virtualization
Michael S. Tsirkin
maintainerQuoted 17 lines · Jason Gunthorpe
Quoted 10 lines · Saeed Mahameed
Quoted 9 lines · Michael S. Tsirkin
Quoted 8 lines · Eli Cohen
Quoted 7 lines · Michael S. Tsirkin
Quoted 4 lines
Did you merge this?:
git pullgit://git.kernel.org/pub/scm/linux/kernel/git/mellanox/linux.gi
t mlx5-nextI can only merge this tree if no one else will. Linus does not like getting same patches through two trees.
This is not quite the case
Linus does not like multiple *copies* of the same patches. The same actual git commits can be OK.
Quoted 7 lines
Linus also does not like unnecessarily cross linking trees, mlx5-next is designed to be small enough and approved enough that it is not controversial.
Linus really doesn't like it when people jams stuff together in rc7 or the weeks of the merge window. He wants to see stuff be in linux-next for at least a bit. So it may be too late regardless.
I'll try, let's see what happens.
Quoted 7 lines
Quoted 3 lines
We do this all the time with net-next and rdma,
mlx5-next is a very small branch based on a very early rc that includes mlx5 shared stuff between rdma and net-next, and now virtio as well.Yes, going on two years now? Been working well
Jason
OK, I'll merge it then. Thanks!
Signature · 1 line
MST