VDPA support for Mellanox ConnectX devices
Subsystems (4)
v2 cover VDPA support for Mellanox ConnectX devices
Hi Michael,
please note that this series depends on mlx5 core device driver patches in mlx5-next branch ingit://git.kernel.org/pub/scm/linux/kernel/git/mellanox/linux.git.
git pull git://git.kernel.org/pub/scm/linux/kernel/git/mellanox/linux.git mlx5-next
They also depend Jason Wang's patches submitted a couple of weeks ago.
vdpa_sim: use the batching API
vhost-vdpa: support batch updating
The following series of patches provide VDPA support for Mellanox devices. The supported devices are ConnectX6 DX and newer.
Currently, only a network driver is implemented; future patches will introduce a block device driver. iperf performance on a single queue is around 12 Gbps. Future patches will introduce multi queue support.
The files are organized in such a way that code that can be used by different VDPA implementations will be placed in a common are resides in drivers/vdpa/mlx5/core.
Only virtual functions are currently supported. Also, certain firmware capabilities must be set to enable the driver. Physical functions (PFs) are skipped by the driver.
To make use of the VDPA net driver, one must load mlx5_vdpa. In such case, VFs will be operated by the VDPA driver. Although one can see a regular instance of a network driver on the VF, the VDPA driver takes precedence over the NIC driver, steering-wize.
Currently, the device/interface infrastructure in mlx5_core is used to probe drivers. Future patches will introduce virtbus as a means to register devices and drivers and VDPA will be adapted to it.
The mlx5 mode of operation required to support VDPA is switchdev mode. Once can use Linux or OVS bridge to take care of layer 2 switching.
In order to provide virtio networking to a guest, an updated version of qemu is required. This version has been tested by the following quemu version:
url: https://github.com/jasowang/qemu.git
branch: vdpa
Commit ID: 6f4e59b807db
Eli Cohen (7):
net/vdpa: Use struct for set/get vq state
vhost: Fix documentation
vdpa: Modify get_vq_state() to return error code vdpa/mlx5: Add hardware descriptive header file vdpa/mlx5: Add support library for mlx5 VDPA implementation vdpa/mlx5: Add shared memory registration code vdpa/mlx5: Add VDPA driver for supported mlx5 devices
Jason Wang (2):
vhost-vdpa: support batch updating
vdpa_sim: use the batching API
Max Gurtovoy (1):
vdpa: remove hard coded virtq num
drivers/vdpa/Kconfig | 18 + drivers/vdpa/Makefile | 1 + drivers/vdpa/ifcvf/ifcvf_base.c | 4 +- drivers/vdpa/ifcvf/ifcvf_base.h | 4 +- drivers/vdpa/ifcvf/ifcvf_main.c | 13 +- drivers/vdpa/mlx5/Makefile | 4 + drivers/vdpa/mlx5/core/mlx5_vdpa.h | 91 ++ drivers/vdpa/mlx5/core/mlx5_vdpa_ifc.h | 168 ++ drivers/vdpa/mlx5/core/mr.c | 473 ++++++ drivers/vdpa/mlx5/core/resources.c | 284 ++++ drivers/vdpa/mlx5/net/main.c | 76 + drivers/vdpa/mlx5/net/mlx5_vnet.c | 1950 ++++++++++++++++++++++++ drivers/vdpa/mlx5/net/mlx5_vnet.h | 24 + drivers/vdpa/vdpa.c | 3 + drivers/vdpa/vdpa_sim/vdpa_sim.c | 35 +- drivers/vhost/iotlb.c | 4 +- drivers/vhost/vdpa.c | 46 +- include/linux/vdpa.h | 24 +- include/uapi/linux/vhost_types.h | 2 + 19 files changed, 3165 insertions(+), 59 deletions(-) create mode 100644 drivers/vdpa/mlx5/Makefile create mode 100644 drivers/vdpa/mlx5/core/mlx5_vdpa.h create mode 100644 drivers/vdpa/mlx5/core/mlx5_vdpa_ifc.h create mode 100644 drivers/vdpa/mlx5/core/mr.c create mode 100644 drivers/vdpa/mlx5/core/resources.c create mode 100644 drivers/vdpa/mlx5/net/main.c create mode 100644 drivers/vdpa/mlx5/net/mlx5_vnet.c create mode 100644 drivers/vdpa/mlx5/net/mlx5_vnet.h
Signature · 1 line
2.26.0
PATCH v2 1/10 vhost-vdpa: support batch updating
From: Jason Wang <[email protected]>
--- drivers/vhost/vdpa.c | 23 +++++++++++++++++------ include/uapi/linux/vhost_types.h | 2 ++ 2 files changed, 19 insertions(+), 6 deletions(-)
drivers/vhost/vdpa.c +17 -6
@@ -73,6 +73,7 @@ struct vhost_vdpa {int virtio_id;int minor;struct eventfd_ctx *config_ctx; +bool in_batch; };static DEFINE_IDA (vhost_vdpa_ida);
@@ -521,9 +522,9 @@ static int vhost_vdpa_map(struct vhost_vdpa *v,if (ops->dma_map) r = ops->dma_map (vdpa, iova, size, pa, perm); -else if (ops->set_map) - r = ops->set_map (vdpa, dev->iotlb); -else +else if (ops->set_map) { + + }else r =iommu_map (v->domain, iova, pa, size,perm_to_iommu_flags (perm));
@@ -540,9 +541,8 @@ static void vhost_vdpa_unmap(struct vhost_vdpa *v, u64 iova, u64 size)if (ops->dma_map) ops->dma_unmap (vdpa, iova, size); -else if (ops->set_map) - ops->set_map (vdpa, dev->iotlb); -else +else if (ops->set_map) { + }else iommu_unmap (v->domain, iova, size); }
@@ -636,6 +636,8 @@ static int vhost_vdpa_process_iotlb_msg(struct vhost_dev *dev, struct vhost_iotlb_msg *msg) { struct vhost_vdpa *v = container_of(dev, struct vhost_vdpa, vdev); +struct vdpa_device *vdpa = v->vdpa; +const struct vdpa_config_ops *ops = vdpa->config;int r =0 ; r =vhost_dev_check_owner (dev);
@@ -649,6 +651,15 @@ static int vhost_vdpa_process_iotlb_msg(struct vhost_dev *dev,case VHOST_IOTLB_INVALIDATE :vhost_vdpa_unmap (v, msg->iova, msg->size);break ; +case VHOST_IOTLB_BATCH_BEGIN : + v->in_batch = true; +break ; +case VHOST_IOTLB_BATCH_END : +if (v->in_batch && ops->set_map) { + ops->set_map (vdpa, dev->iotlb); + } + v->in_batch = false; +break ;default : r = -EINVAL ;break ;
include/uapi/linux/vhost_types.h +2 -0
@@ -60,6 +60,8 @@ struct vhost_iotlb_msg {#define VHOST_IOTLB_UPDATE 2#define VHOST_IOTLB_INVALIDATE 3#define VHOST_IOTLB_ACCESS_FAIL 4 +#define VHOST_IOTLB_BATCH_BEGIN 5 +#define VHOST_IOTLB_BATCH_END 6__u8 type; };
Signature · 1 line
2.26.0
PATCH v2 2/10 vdpa_sim: use the batching API
In reply to Eli CohenFrom: Jason Wang <[email protected]>
--- drivers/vdpa/vdpa_sim/vdpa_sim.c | 20 -------------------- drivers/vhost/vdpa.c | 1 - 2 files changed, 21 deletions(-)
drivers/vdpa/vdpa_sim/vdpa_sim.c +0 -20
@@ -548,24 +548,6 @@ static int vdpasim_set_map(struct vdpa_device *vdpa,return ret; } -static int vdpasim_dma_map (struct vdpa_device *vdpa,u64 iova,u64 size, -u64 pa,u32 perm) -{ -struct vdpasim *vdpasim =vdpa_to_sim (vdpa); - -return vhost_iotlb_add_range (vdpasim->iommu, iova, - iova + size -1 , pa, perm); -} - -static int vdpasim_dma_unmap (struct vdpa_device *vdpa,u64 iova,u64 size) -{ -struct vdpasim *vdpasim =vdpa_to_sim (vdpa); - -vhost_iotlb_del_range (vdpasim->iommu, iova, iova + size -1 ); - -return 0 ; -} -static void vdpasim_free (struct vdpa_device *vdpa) {struct vdpasim *vdpasim =vdpa_to_sim (vdpa);
@@ -598,8 +580,6 @@ static const struct vdpa_config_ops vdpasim_net_config_ops = { .set_config = vdpasim_set_config, .get_generation = vdpasim_get_generation, .set_map = vdpasim_set_map, - .dma_map = vdpasim_dma_map, - .dma_unmap = vdpasim_dma_unmap, .free = vdpasim_free, };
drivers/vhost/vdpa.c +0 -1
@@ -533,7 +533,6 @@ static int vhost_vdpa_map(struct vhost_vdpa *v,static void vhost_vdpa_unmap (struct vhost_vdpa *v,u64 iova,u64 size) { -struct vhost_dev *dev = &v->vdev;struct vdpa_device *vdpa = v->vdpa;const struct vdpa_config_ops *ops = vdpa->config;
Signature · 1 line
2.26.0
PATCH v2 3/10 vdpa: remove hard coded virtq num
In reply to Eli CohenFrom: Max Gurtovoy <[email protected]>
This will enable vdpa providers to add support for multi queue feature and publish it to upper layers (vhost and virtio).
--- drivers/vdpa/ifcvf/ifcvf_main.c | 3 ++- drivers/vdpa/vdpa.c | 3 +++ drivers/vdpa/vdpa_sim/vdpa_sim.c | 4 ++-- drivers/vhost/vdpa.c | 9 +++------ include/linux/vdpa.h | 6 ++++-- 5 files changed, 14 insertions(+), 11 deletions(-)
drivers/vdpa/ifcvf/ifcvf_main.c +2 -1
@@ -420,7 +420,8 @@ static int ifcvf_probe(struct pci_dev *pdev, const struct pci_device_id *id) } adapter = vdpa_alloc_device(struct ifcvf_adapter, vdpa, - dev, &ifc_vdpa_ops); + dev, &ifc_vdpa_ops, +IFCVF_MAX_QUEUE_PAIRS *2 );if (adapter ==NULL ) {IFCVF_ERR (pdev,"Failed to allocate vDPA structure" );return -ENOMEM ;
drivers/vdpa/vdpa.c +3 -0
@@ -61,6 +61,7 @@ static void vdpa_release_dev(struct device *d)* initialized but before registered. * @parent: the parent device * @config: the bus operations that is supported by this device +* @nvqs: number of virtqueues supported by this device * @size: size of the parent structure that contains private data * * Driver should use vdpa_alloc_device() wrapper macro instead of
@@ -71,6 +72,7 @@ static void vdpa_release_dev(struct device *d)*/ struct vdpa_device *__vdpa_alloc_device (struct device *parent,const struct vdpa_config_ops *config, +int nvqs,size_t size) {struct vdpa_device *vdev;
@@ -96,6 +98,7 @@ struct vdpa_device *__vdpa_alloc_device(struct device *parent, vdev->dev.release = vdpa_release_dev; vdev->index = err; vdev->config = config; + vdev->nvqs = nvqs; err =dev_set_name (&vdev->dev,"vdpa%u" , vdev->index);if (err)
drivers/vdpa/vdpa_sim/vdpa_sim.c +2 -2
@@ -60,7 +60,7 @@ static u64 vdpasim_features = (1ULL << VIRTIO_F_ANY_LAYOUT) |/* State of each vdpasim device */ struct vdpasim {struct vdpa_device vdpa; -struct vdpasim_virtqueue vqs[2 ]; +struct vdpasim_virtqueue vqs[VDPASIM_VQ_NUM ];struct work_struct work;/* spinlock to synchronize virtqueue state */ spinlock_t lock;
@@ -312,7 +312,7 @@ static struct vdpasim *vdpasim_create(void)int ret = -ENOMEM ; vdpasim = vdpa_alloc_device(struct vdpasim, vdpa, NULL, - &vdpasim_net_config_ops); + &vdpasim_net_config_ops,VDPASIM_VQ_NUM );if (!vdpasim) goto err_alloc;
drivers/vhost/vdpa.c +3 -6
@@ -55,9 +55,6 @@ enum { (1ULL << VIRTIO_NET_F_SPEED_DUPLEX), }; -/* Currently, only network backend w/o multiqueue is supported. */ -#define VHOST_VDPA_VQ_MAX 2 -#define VHOST_VDPA_DEV_MAX (1U << MINORBITS)struct vhost_vdpa {
@@ -882,7 +879,7 @@ static int vhost_vdpa_probe(struct vdpa_device *vdpa) {const struct vdpa_config_ops *ops = vdpa->config;struct vhost_vdpa *v; -int minor, nvqs =VHOST_VDPA_VQ_MAX ; +int minor;int r;/* Currently, we only accept the network devices. */
@@ -903,14 +900,14 @@ static int vhost_vdpa_probe(struct vdpa_device *vdpa)atomic_set (&v->opened,0 ); v->minor = minor; v->vdpa = vdpa; - v->nvqs = nvqs; + v->nvqs = vdpa->nvqs; v->virtio_id = ops->get_device_id (vdpa);device_initialize (&v->dev); v->dev.release = vhost_vdpa_release_dev; v->dev.parent = &vdpa->dev; v->dev.devt =MKDEV (MAJOR (vhost_vdpa_major), minor); - v->vqs =kmalloc_array (nvqs,sizeof (struct vhost_virtqueue ), + v->vqs =kmalloc_array (v->nvqs,sizeof (struct vhost_virtqueue ),GFP_KERNEL );if (!v->vqs) { r = -ENOMEM ;
include/linux/vdpa.h +4 -2
@@ -39,6 +39,7 @@ struct vdpa_device {struct device *dma_dev;const struct vdpa_config_ops *config;unsigned int index; +int nvqs; };/**
@@ -208,11 +209,12 @@ struct vdpa_config_ops {struct vdpa_device *__vdpa_alloc_device (struct device *parent,const struct vdpa_config_ops *config, +int nvqs,size_t size); -#define vdpa_alloc_device (dev_struct, member, parent, config) \ +#define vdpa_alloc_device (dev_struct, member, parent, config, nvqs) \ container_of(__vdpa_alloc_device( \ - parent, config, \ + parent, config, nvqs, \ sizeof(dev_struct) + \ BUILD_BUG_ON_ZERO(offsetof( \ dev_struct, member))), \
Signature · 1 line
2.26.0
PATCH v2 4/10 net/vdpa: Use struct for set/get vq state
In reply to Eli CohenFor now VQ state involves 16 bit available index value encoded in u64 variable. In the future it will be extended to contain more fields. Use struct to contain the state, now containing only a single u16 for the available index. In the future we can add fields to this struct.
--- drivers/vdpa/ifcvf/ifcvf_base.c | 4 ++-- drivers/vdpa/ifcvf/ifcvf_base.h | 4 ++-- drivers/vdpa/ifcvf/ifcvf_main.c | 9 +++++---- drivers/vdpa/vdpa_sim/vdpa_sim.c | 10 ++++++---- drivers/vhost/vdpa.c | 10 +++++++--- include/linux/vdpa.h | 18 ++++++++++++++---- 6 files changed, 36 insertions(+), 19 deletions(-)
drivers/vdpa/ifcvf/ifcvf_base.c +2 -2
@@ -272,7 +272,7 @@ static int ifcvf_config_features(struct ifcvf_hw *hw)return 0 ; } -u64 ifcvf_get_vq_state (struct ifcvf_hw *hw,u16 qid) +u16 ifcvf_get_vq_state (struct ifcvf_hw *hw,u16 qid) {struct ifcvf_lm_cfg __iomem *ifcvf_lm;void __iomem *avail_idx_addr;
@@ -287,7 +287,7 @@ u64 ifcvf_get_vq_state(struct ifcvf_hw *hw, u16 qid)return last_avail_idx; } -int ifcvf_set_vq_state (struct ifcvf_hw *hw,u16 qid,u64 num) +int ifcvf_set_vq_state (struct ifcvf_hw *hw,u16 qid,u16 num) {struct ifcvf_lm_cfg __iomem *ifcvf_lm;void __iomem *avail_idx_addr;
drivers/vdpa/ifcvf/ifcvf_base.h +2 -2
@@ -116,7 +116,7 @@ void ifcvf_set_status(struct ifcvf_hw *hw, u8 status);void io_write64_twopart (u64 val,u32 *lo,u32 *hi);void ifcvf_reset (struct ifcvf_hw *hw);u64 ifcvf_get_features (struct ifcvf_hw *hw); -u64 ifcvf_get_vq_state (struct ifcvf_hw *hw,u16 qid); -int ifcvf_set_vq_state (struct ifcvf_hw *hw,u16 qid,u64 num); +u16 ifcvf_get_vq_state (struct ifcvf_hw *hw,u16 qid); +int ifcvf_set_vq_state (struct ifcvf_hw *hw,u16 qid,u16 num);struct ifcvf_adapter *vf_to_adapter (struct ifcvf_hw *hw);#endif /* _IFCVF_H_ */
drivers/vdpa/ifcvf/ifcvf_main.c +5 -4
@@ -235,19 +235,20 @@ static u16 ifcvf_vdpa_get_vq_num_max(struct vdpa_device *vdpa_dev)return IFCVF_QUEUE_MAX ; } -static u64 ifcvf_vdpa_get_vq_state (struct vdpa_device *vdpa_dev,u16 qid) +static void ifcvf_vdpa_get_vq_state (struct vdpa_device *vdpa_dev,u16 qid, +struct vdpa_vq_state *state) {struct ifcvf_hw *vf =vdpa_to_vf (vdpa_dev); -return ifcvf_get_vq_state (vf, qid); + state->avail_index =ifcvf_get_vq_state (vf, qid); }static int ifcvf_vdpa_set_vq_state (struct vdpa_device *vdpa_dev,u16 qid, -u64 num) +const struct vdpa_vq_state *state) {struct ifcvf_hw *vf =vdpa_to_vf (vdpa_dev); -return ifcvf_set_vq_state (vf, qid, num); +return ifcvf_set_vq_state (vf, qid, state->avail_index); } static void ifcvf_vdpa_set_vq_cb(struct vdpa_device *vdpa_dev, u16 qid,
drivers/vdpa/vdpa_sim/vdpa_sim.c +6 -4
@@ -413,26 +413,28 @@ static bool vdpasim_get_vq_ready(struct vdpa_device *vdpa, u16 idx)return vq->ready; } -static int vdpasim_set_vq_state (struct vdpa_device *vdpa,u16 idx,u64 state) +static int vdpasim_set_vq_state (struct vdpa_device *vdpa,u16 idx, +const struct vdpa_vq_state *state) {struct vdpasim *vdpasim =vdpa_to_sim (vdpa);struct vdpasim_virtqueue *vq = &vdpasim->vqs[idx];struct vringh *vrh = &vq->vring;spin_lock (&vdpasim->lock); - vrh->last_avail_idx = state; + vrh->last_avail_idx = state->avail_index;spin_unlock (&vdpasim->lock);return 0 ; } -static u64 vdpasim_get_vq_state (struct vdpa_device *vdpa,u16 idx) +static void vdpasim_get_vq_state (struct vdpa_device *vdpa,u16 idx, +struct vdpa_vq_state *state) {struct vdpasim *vdpasim =vdpa_to_sim (vdpa);struct vdpasim_virtqueue *vq = &vdpasim->vqs[idx];struct vringh *vrh = &vq->vring; -return vrh->last_avail_idx; + state->avail_index = vrh->last_avail_idx; }static u32 vdpasim_get_vq_align (struct vdpa_device *vdpa)
drivers/vhost/vdpa.c +7 -3
@@ -335,6 +335,7 @@ static long vhost_vdpa_vring_ioctl(struct vhost_vdpa *v, unsigned int cmd, {struct vdpa_device *vdpa = v->vdpa;const struct vdpa_config_ops *ops = vdpa->config; +struct vdpa_vq_state vq_state;struct vdpa_callback cb;struct vhost_virtqueue *vq;struct vhost_vring_state s;
@@ -358,8 +359,10 @@ static long vhost_vdpa_vring_ioctl(struct vhost_vdpa *v, unsigned int cmd,return 0 ; } -if (cmd ==VHOST_GET_VRING_BASE ) - vq->last_avail_idx = ops->get_vq_state (v->vdpa, idx); +if (cmd ==VHOST_GET_VRING_BASE ) { + ops->get_vq_state (v->vdpa, idx, &vq_state); + vq->last_avail_idx = vq_state.avail_index; + } r =vhost_vring_ioctl (&v->vdev, cmd, argp);if (r)
@@ -375,7 +378,8 @@ static long vhost_vdpa_vring_ioctl(struct vhost_vdpa *v, unsigned int cmd,break ;case VHOST_SET_VRING_BASE : -if (ops->set_vq_state (vdpa, idx, vq->last_avail_idx)) + vq_state.avail_index = vq->last_avail_idx; +if (ops->set_vq_state (vdpa, idx, &vq_state)) r = -EINVAL ;break ;
include/linux/vdpa.h +14 -4
@@ -27,6 +27,14 @@ struct vdpa_notification_area {resource_size_t size; }; +/** +* vDPA vq_state definition +* @avail_index: available index +*/ +struct vdpa_vq_state { +u16 avail_index; +}; +/** * vDPA device - representation of a vDPA device * @dev: underlying device
@@ -78,12 +86,12 @@ struct vdpa_device {* @set_vq_state: Set the state for a virtqueue * @vdev: vdpa device * @idx: virtqueue index -* @state: virtqueue state (last_avail_idx) +* @state: pointer to set virtqueue state (last_avail_idx) * Returns integer: success (0) or error (< 0) * @get_vq_state: Get the state for a virtqueue * @vdev: vdpa device * @idx: virtqueue index -* Returns virtqueue state (last_avail_idx) +* @state: pointer to returned state (last_avail_idx) * @get_vq_notification: Get the notification area for a virtqueue * @vdev: vdpa device * @idx: virtqueue index
@@ -175,8 +183,10 @@ struct vdpa_config_ops { struct vdpa_callback *cb);void (*set_vq_ready)(struct vdpa_device *vdev,u16 idx,bool ready);bool (*get_vq_ready)(struct vdpa_device *vdev,u16 idx); -int (*set_vq_state)(struct vdpa_device *vdev,u16 idx,u64 state); -u64 (*get_vq_state)(struct vdpa_device *vdev,u16 idx); +int (*set_vq_state)(struct vdpa_device *vdev,u16 idx, +const struct vdpa_vq_state *state); +void (*get_vq_state)(struct vdpa_device *vdev,u16 idx, +struct vdpa_vq_state *state);struct vdpa_notification_area (*get_vq_notification)(struct vdpa_device *vdev,u16 idx);
Signature · 1 line
2.26.0
PATCH v2 5/10 vhost: Fix documentation
In reply to Eli CohenFix documentation to match actual function prototypes.
--- drivers/vhost/iotlb.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
drivers/vhost/iotlb.c +2 -2
@@ -149,7 +149,7 @@ EXPORT_SYMBOL_GPL(vhost_iotlb_free);* vhost_iotlb_itree_first - return the first overlapped range * @iotlb: the IOTLB * @start: start of IOVA range -* @end: end of IOVA range +* @last: last byte in IOVA range */ struct vhost_iotlb_map *vhost_iotlb_itree_first (struct vhost_iotlb *iotlb,u64 start,u64 last)
@@ -162,7 +162,7 @@ EXPORT_SYMBOL_GPL(vhost_iotlb_itree_first);* vhost_iotlb_itree_first - return the next overlapped range * @iotlb: the IOTLB * @start: start of IOVA range -* @end: end of IOVA range +* @last: last byte IOVA range */ struct vhost_iotlb_map *vhost_iotlb_itree_next (struct vhost_iotlb_map *map,u64 start,u64 last)
Signature · 1 line
2.26.0
PATCH v2 6/10 vdpa: Modify get_vq_state() to return error code
In reply to Eli CohenModify get_vq_state() so it returns an error code. In case of hardware acceleration, the available index may be retrieved from the device, an operation that can possibly fail.
--- drivers/vdpa/ifcvf/ifcvf_main.c | 5 +++-- drivers/vdpa/vdpa_sim/vdpa_sim.c | 5 +++-- drivers/vhost/vdpa.c | 5 ++++- include/linux/vdpa.h | 4 ++-- 4 files changed, 12 insertions(+), 7 deletions(-)
drivers/vdpa/ifcvf/ifcvf_main.c +3 -2
@@ -235,12 +235,13 @@ static u16 ifcvf_vdpa_get_vq_num_max(struct vdpa_device *vdpa_dev)return IFCVF_QUEUE_MAX ; } -static void ifcvf_vdpa_get_vq_state (struct vdpa_device *vdpa_dev,u16 qid, -struct vdpa_vq_state *state) +static int ifcvf_vdpa_get_vq_state (struct vdpa_device *vdpa_dev,u16 qid, +struct vdpa_vq_state *state) {struct ifcvf_hw *vf =vdpa_to_vf (vdpa_dev); state->avail_index =ifcvf_get_vq_state (vf, qid); +return 0 ; } static int ifcvf_vdpa_set_vq_state(struct vdpa_device *vdpa_dev, u16 qid,
drivers/vdpa/vdpa_sim/vdpa_sim.c +3 -2
@@ -427,14 +427,15 @@ static int vdpasim_set_vq_state(struct vdpa_device *vdpa, u16 idx,return 0 ; } -static void vdpasim_get_vq_state (struct vdpa_device *vdpa,u16 idx, -struct vdpa_vq_state *state) +static int vdpasim_get_vq_state (struct vdpa_device *vdpa,u16 idx, +struct vdpa_vq_state *state) {struct vdpasim *vdpasim =vdpa_to_sim (vdpa);struct vdpasim_virtqueue *vq = &vdpasim->vqs[idx];struct vringh *vrh = &vq->vring; state->avail_index = vrh->last_avail_idx; +return 0 ; }static u32 vdpasim_get_vq_align (struct vdpa_device *vdpa)
drivers/vhost/vdpa.c +4 -1
@@ -360,7 +360,10 @@ static long vhost_vdpa_vring_ioctl(struct vhost_vdpa *v, unsigned int cmd, }if (cmd ==VHOST_GET_VRING_BASE ) { - ops->get_vq_state (v->vdpa, idx, &vq_state); + r = ops->get_vq_state (v->vdpa, idx, &vq_state); +if (r) +return r; + vq->last_avail_idx = vq_state.avail_index; }
include/linux/vdpa.h +2 -2
@@ -185,8 +185,8 @@ struct vdpa_config_ops {bool (*get_vq_ready)(struct vdpa_device *vdev,u16 idx);int (*set_vq_state)(struct vdpa_device *vdev,u16 idx,const struct vdpa_vq_state *state); -void (*get_vq_state)(struct vdpa_device *vdev,u16 idx, -struct vdpa_vq_state *state); +int (*get_vq_state)(struct vdpa_device *vdev,u16 idx, +struct vdpa_vq_state *state);struct vdpa_notification_area (*get_vq_notification)(struct vdpa_device *vdev,u16 idx);
Signature · 1 line
2.26.0
PATCH v2 7/10 vdpa/mlx5: Add hardware descriptive header file
In reply to Eli CohenKeep all vdpa related hardware definitions in this file.
--- drivers/vdpa/mlx5/core/mlx5_vdpa_ifc.h | 168 +++++++++++++++++++++++++ 1 file changed, 168 insertions(+) create mode 100644 drivers/vdpa/mlx5/core/mlx5_vdpa_ifc.h
drivers/vdpa/mlx5/core/mlx5_vdpa_ifc.h new file +168 -0
@@ -0,0 +1,168 @@ +/* SPDX-License-Identifier: GPL-2.0 OR Linux-OpenIB */ +/* Copyright (c) 2020 Mellanox Technologies Ltd. */ + +#ifndef __MLX5_VDPA_IFC_H_ +#define __MLX5_VDPA_IFC_H_ + +#include <linux/mlx5/mlx5_ifc.h> + +enum { +MLX5_VIRTIO_Q_EVENT_MODE_NO_MSIX_MODE =0x0 , +MLX5_VIRTIO_Q_EVENT_MODE_QP_MODE =0x1 , +MLX5_VIRTIO_Q_EVENT_MODE_MSIX_MODE =0x2 , +}; + +enum { +MLX5_VIRTIO_EMULATION_CAP_VIRTIO_QUEUE_TYPE_SPLIT =0x1 ,// do I check this caps? +MLX5_VIRTIO_EMULATION_CAP_VIRTIO_QUEUE_TYPE_PACKED =0x2 , +}; + +enum { +MLX5_VIRTIO_EMULATION_VIRTIO_QUEUE_TYPE_SPLIT =0 , +MLX5_VIRTIO_EMULATION_VIRTIO_QUEUE_TYPE_PACKED =1 , +}; + +struct mlx5_ifc_virtio_q_bits { +u8 virtio_q_type[0x8 ]; +u8 reserved_at_8[0x5 ]; +u8 event_mode[0x3 ]; +u8 queue_index[0x10 ]; + +u8 full_emulation[0x1 ]; +u8 virtio_version_1_0[0x1 ]; +u8 reserved_at_22[0x2 ]; +u8 offload_type[0x4 ]; +u8 event_qpn_or_msix[0x18 ]; + +u8 doorbell_stride_index[0x10 ]; +u8 queue_size[0x10 ]; + +u8 device_emulation_id[0x20 ]; + +u8 desc_addr[0x40 ]; + +u8 used_addr[0x40 ]; + +u8 available_addr[0x40 ]; + +u8 virtio_q_mkey[0x20 ]; + +u8 max_tunnel_desc[0x10 ]; +u8 reserved_at_170[0x8 ]; +u8 error_type[0x8 ]; + +u8 umem_1_id[0x20 ]; + +u8 umem_1_size[0x20 ]; + +u8 umem_1_offset[0x40 ]; + +u8 umem_2_id[0x20 ]; + +u8 umem_2_size[0x20 ]; + +u8 umem_2_offset[0x40 ]; + +u8 umem_3_id[0x20 ]; + +u8 umem_3_size[0x20 ]; + +u8 umem_3_offset[0x40 ]; + +u8 counter_set_id[0x20 ]; + +u8 reserved_at_320[0x8 ]; +u8 pd[0x18 ]; + +u8 reserved_at_340[0xc0 ]; +}; + +struct mlx5_ifc_virtio_net_q_object_bits { +u8 modify_field_select[0x40 ]; + +u8 reserved_at_40[0x20 ]; + +u8 vhca_id[0x10 ]; +u8 reserved_at_70[0x10 ]; + +u8 queue_feature_bit_mask_12_3[0xa ]; +u8 dirty_bitmap_dump_enable[0x1 ]; +u8 vhost_log_page[0x5 ]; +u8 reserved_at_90[0xc ]; +u8 state[0x4 ]; + +u8 reserved_at_a0[0x5 ]; +u8 queue_feature_bit_mask_2_0[0x3 ]; +u8 tisn_or_qpn[0x18 ]; + +u8 dirty_bitmap_mkey[0x20 ]; + +u8 dirty_bitmap_size[0x20 ]; + +u8 dirty_bitmap_addr[0x40 ]; + +u8 hw_available_index[0x10 ]; +u8 hw_used_index[0x10 ]; + +u8 reserved_at_160[0xa0 ]; + +struct mlx5_ifc_virtio_q_bits virtio_q_context; +}; + +struct mlx5_ifc_create_virtio_net_q_in_bits { +struct mlx5_ifc_general_obj_in_cmd_hdr_bits general_obj_in_cmd_hdr; + +struct mlx5_ifc_virtio_net_q_object_bits obj_context; +}; + +struct mlx5_ifc_create_virtio_net_q_out_bits { +struct mlx5_ifc_general_obj_out_cmd_hdr_bits general_obj_out_cmd_hdr; +}; + +struct mlx5_ifc_destroy_virtio_net_q_in_bits { +struct mlx5_ifc_general_obj_in_cmd_hdr_bits general_obj_out_cmd_hdr; +}; + +struct mlx5_ifc_destroy_virtio_net_q_out_bits { +struct mlx5_ifc_general_obj_out_cmd_hdr_bits general_obj_out_cmd_hdr; +}; + +struct mlx5_ifc_query_virtio_net_q_in_bits { +struct mlx5_ifc_general_obj_in_cmd_hdr_bits general_obj_in_cmd_hdr; +}; + +struct mlx5_ifc_query_virtio_net_q_out_bits { +struct mlx5_ifc_general_obj_out_cmd_hdr_bits general_obj_out_cmd_hdr; + +struct mlx5_ifc_virtio_net_q_object_bits obj_context; +}; + +enum { +MLX5_VIRTQ_MODIFY_MASK_STATE = (u64 )1 <<0 , +MLX5_VIRTQ_MODIFY_MASK_DIRTY_BITMAP_PARAMS = (u64 )1 <<3 , +MLX5_VIRTQ_MODIFY_MASK_DIRTY_BITMAP_DUMP_ENABLE = (u64 )1 <<4 , +}; + +enum { +MLX5_VIRTIO_NET_Q_OBJECT_STATE_INIT =0x0 , +MLX5_VIRTIO_NET_Q_OBJECT_STATE_RDY =0x1 , +MLX5_VIRTIO_NET_Q_OBJECT_STATE_SUSPEND =0x2 , +MLX5_VIRTIO_NET_Q_OBJECT_STATE_ERR =0x3 , +}; + +enum { +MLX5_RQTC_LIST_Q_TYPE_RQ =0x0 , +MLX5_RQTC_LIST_Q_TYPE_VIRTIO_NET_Q =0x1 , +}; + +struct mlx5_ifc_modify_virtio_net_q_in_bits { +struct mlx5_ifc_general_obj_in_cmd_hdr_bits general_obj_in_cmd_hdr; + +struct mlx5_ifc_virtio_net_q_object_bits obj_context; +}; + +struct mlx5_ifc_modify_virtio_net_q_out_bits { +struct mlx5_ifc_general_obj_out_cmd_hdr_bits general_obj_out_cmd_hdr; +}; + +#endif /* __MLX5_VDPA_IFC_H_ */
Signature · 1 line
2.26.0
PATCH v2 8/10 vdpa/mlx5: Add support library for mlx5 VDPA implementation
In reply to Eli CohenFollowing patches introduce VDPA network driver for Mellanox Connectx6 devices. This patch provides functionality that will be used by those patches.
--- drivers/vdpa/Kconfig | 8 + drivers/vdpa/Makefile | 1 + drivers/vdpa/mlx5/Makefile | 1 + drivers/vdpa/mlx5/core/Makefile | 1 + drivers/vdpa/mlx5/core/mlx5_vdpa.h | 57 ++++++ drivers/vdpa/mlx5/core/resources.c | 281 +++++++++++++++++++++++++++++ 6 files changed, 349 insertions(+) create mode 100644 drivers/vdpa/mlx5/Makefile create mode 100644 drivers/vdpa/mlx5/core/Makefile create mode 100644 drivers/vdpa/mlx5/core/mlx5_vdpa.h create mode 100644 drivers/vdpa/mlx5/core/resources.c
drivers/vdpa/Kconfig +8 -0
@@ -28,4 +28,12 @@ config IFCVF To compile this driver as a module, choose M here: the module will be called ifcvf. +config MLX5_VDPA + bool "MLX5 VDPA support library for ConnectX devices" + depends on MLX5_CORE + default n + help + Support library for Mellanox VDPA drivers. Provides code that is + common for all types of VDPA drivers. + endif # VDPA
drivers/vdpa/Makefile +1 -0
@@ -2,3 +2,4 @@ obj-$(CONFIG_VDPA) += vdpa.o obj-$(CONFIG_VDPA_SIM) += vdpa_sim/ obj-$(CONFIG_IFCVF) += ifcvf/ +obj-$(CONFIG_MLX5_VDPA) += mlx5/
drivers/vdpa/mlx5/Makefile new file +1 -0
@@ -0,0 +1 @@ +obj-$(CONFIG_MLX5_VDPA) += core/resources.o
drivers/vdpa/mlx5/core/Makefile new file +1 -0
@@ -0,0 +1 @@ +obj-y += resources.o
drivers/vdpa/mlx5/core/mlx5_vdpa.h new file +57 -0
57 lines not on this page · raw message
drivers/vdpa/mlx5/core/resources.c new file +281 -0
281 lines not on this page · raw message
Signature · 1 line
2.26.0
PATCH v2 9/10 vdpa/mlx5: Add shared memory registration code
In reply to Eli CohenAdd code to support registering address space region for the device. The virtio driver can run as either:
1. Guest virtio driver
2. Userspace virtio driver on the host
3. Kernel virtio driver on the host
In any case a memory key object is required to provide access to memory for the device.
This code will be shared by network or block driver implementations.
---
V0 -> V2:
1. Improve changelog
2. Fix static checker warning
drivers/vdpa/mlx5/Makefile | 2 +- drivers/vdpa/mlx5/core/Makefile | 1 - drivers/vdpa/mlx5/core/mlx5_vdpa.h | 34 +++ drivers/vdpa/mlx5/core/mr.c | 473 +++++++++++++++++++++++++++++ drivers/vdpa/mlx5/core/resources.c | 3 + 5 files changed, 511 insertions(+), 2 deletions(-) delete mode 100644 drivers/vdpa/mlx5/core/Makefile create mode 100644 drivers/vdpa/mlx5/core/mr.c
drivers/vdpa/mlx5/Makefile +1 -1
@@ -1 +1 @@ -obj-$(CONFIG_MLX5_VDPA) += core/resources.o +obj-$(CONFIG_MLX5_VDPA) += core/resources.o core/mr.o
drivers/vdpa/mlx5/core/Makefile deleted +0 -1
@@ -1 +0,0 @@ -obj-y += resources.o
drivers/vdpa/mlx5/core/mlx5_vdpa.h +34 -0
@@ -7,6 +7,31 @@#include <linux/vdpa.h> #include <linux/mlx5/driver.h> +struct mlx5_vdpa_direct_mr { +u64 start; +u64 end; +u32 perm; +struct mlx5_core_mkey mr; +struct sg_table sg_head; +int log_size; +int nsg; +struct list_head list; +u64 offset; +}; + +struct mlx5_vdpa_mr { +struct mlx5_core_mkey mkey; + +/* list of direct MRs descendants of this indirect mr */ +struct list_head head; +unsigned long num_directs; +unsigned long num_klms; +bool initialized; + +/* serialize mkey creation and destruction */ +struct mutex mkey_mtx; +}; +struct mlx5_vdpa_resources {u32 pdn;struct mlx5_uars_page *uar;
@@ -26,6 +51,8 @@ struct mlx5_vdpa_dev {u8 status;u32 max_vqs;u32 generation; + +struct mlx5_vdpa_mr mr; };int mlx5_vdpa_alloc_pd (struct mlx5_vdpa_dev *dev,u32 *pdn,u16 uid);
@@ -41,6 +68,13 @@ int mlx5_vdpa_alloc_transport_domain(struct mlx5_vdpa_dev *mvdev, u32 *tdn);void mlx5_vdpa_dealloc_transport_domain (struct mlx5_vdpa_dev *mvdev,u32 tdn);int mlx5_vdpa_alloc_resources (struct mlx5_vdpa_dev *mvdev);void mlx5_vdpa_free_resources (struct mlx5_vdpa_dev *mvdev); +int mlx5_vdpa_create_mkey (struct mlx5_vdpa_dev *mvdev,struct mlx5_core_mkey *mkey,u32 *in, +int inlen); +int mlx5_vdpa_destroy_mkey (struct mlx5_vdpa_dev *mvdev,struct mlx5_core_mkey *mkey); +int mlx5_vdpa_handle_set_map (struct mlx5_vdpa_dev *mvdev,struct vhost_iotlb *iotlb, +bool *change_map); +int mlx5_vdpa_create_mr (struct mlx5_vdpa_dev *mvdev,struct vhost_iotlb *iotlb); +void mlx5_vdpa_destroy_mr (struct mlx5_vdpa_dev *mvdev);#define mlx5_vdpa_warn (__dev, format, ...) \ dev_warn((__dev)->mdev->device, "%s:%d:(pid %d) warning: " format, __func__, __LINE__, \
drivers/vdpa/mlx5/core/mr.c new file +473 -0
473 lines not on this page · raw message
drivers/vdpa/mlx5/core/resources.c +3 -0
@@ -227,6 +227,7 @@ int mlx5_vdpa_alloc_resources(struct mlx5_vdpa_dev *mvdev)mlx5_vdpa_warn (mvdev,"resources already allocated\n" );return -EINVAL ; } +mutex_init (&mvdev->mr.mkey_mtx); res->uar =mlx5_get_uars_page (mdev);if (IS_ERR (res->uar)) { err =PTR_ERR (res->uar);
@@ -262,6 +263,7 @@ int mlx5_vdpa_alloc_resources(struct mlx5_vdpa_dev *mvdev) err_uctx:mlx5_put_uars_page (mdev, res->uar); err_uars: +mutex_destroy (&mvdev->mr.mkey_mtx);return err; }
@@ -277,5 +279,6 @@ void mlx5_vdpa_free_resources(struct mlx5_vdpa_dev *mvdev)dealloc_pd (mvdev, res->pdn, res->uid);destroy_uctx (mvdev, res->uid);mlx5_put_uars_page (mvdev->mdev, res->uar); +mutex_destroy (&mvdev->mr.mkey_mtx); res->valid = false; }
Signature · 1 line
2.26.0
PATCH v2 10/10 vdpa/mlx5: Add VDPA driver for supported mlx5 devices
In reply to Eli CohenAdd a front end VDPA driver that registers in the VDPA bus and provides networking to a guest. The VDPA driver creates the necessary resources on the VF it is driving such that data path will be offloaded.
Notifications are being communicated through the driver.
Currently, only VFs are supported. In subsequent patches we will have devlink support to control which VF is used for VDPA and which function is used for regular networking.
---
Changes from V0:
1. Fix include path usage
2. Fix use after free in qp_create()
3. Consistently use mvq->initialized to check if a vq was initialized.
4. Remove unused local variable.
5. Defer modifyig vq to ready to driver ok
6. suspend hardware vq in set_vq_ready(0)
7. Remove reservation for control VQ since it multi queue is not supported in this version
8. Avoid call put_device() since this is not a pci device driver.
drivers/vdpa/Kconfig | 10 + drivers/vdpa/mlx5/Makefile | 5 +- drivers/vdpa/mlx5/core/mr.c | 2 +- drivers/vdpa/mlx5/net/main.c | 76 ++ drivers/vdpa/mlx5/net/mlx5_vnet.c | 1950 +++++++++++++++++++++++++++++ drivers/vdpa/mlx5/net/mlx5_vnet.h | 24 + 6 files changed, 2065 insertions(+), 2 deletions(-) create mode 100644 drivers/vdpa/mlx5/net/main.c create mode 100644 drivers/vdpa/mlx5/net/mlx5_vnet.c create mode 100644 drivers/vdpa/mlx5/net/mlx5_vnet.h
drivers/vdpa/Kconfig +10 -0
@@ -36,4 +36,14 @@ config MLX5_VDPA Support library for Mellanox VDPA drivers. Provides code that is common for all types of VDPA drivers. +config MLX5_VDPA_NET + tristate "vDPA driver for ConnectX devices" + depends on MLX5_VDPA + default n + help + VDPA network driver for ConnectX6 and newer. Provides offloading + of virtio net datapath such that descriptors put on the ring will + be executed by the hardware. It also supports a variety of stateless + offloads depending on the actual device used and firmware version. + endif # VDPA
drivers/vdpa/mlx5/Makefile +4 -1
@@ -1 +1,4 @@ -obj-$(CONFIG_MLX5_VDPA) += core/resources.o core/mr.o +subdir-ccflags-y += -I$(src)/core + +obj-$(CONFIG_MLX5_VDPA_NET) += mlx5_vdpa.o +mlx5_vdpa-$(CONFIG_MLX5_VDPA_NET) += net/main.o net/mlx5_vnet.o core/resources.o core/mr.o
drivers/vdpa/mlx5/core/mr.c +1 -1
@@ -453,7 +453,7 @@ int mlx5_vdpa_handle_set_map(struct mlx5_vdpa_dev *mvdev, struct vhost_iotlb *io bool *change_map) {struct mlx5_vdpa_mr *mr = &mvdev->mr; -int err; +int err =0 ; *change_map = false;if (map_empty (iotlb)) {
drivers/vdpa/mlx5/net/main.c new file +76 -0
76 lines not on this page · raw message
drivers/vdpa/mlx5/net/mlx5_vnet.c new file +1950 -0
1,950 lines not on this page · raw message
drivers/vdpa/mlx5/net/mlx5_vnet.h new file +24 -0
@@ -0,0 +1,24 @@ +/* SPDX-License-Identifier: GPL-2.0 OR Linux-OpenIB */ +/* Copyright (c) 2020 Mellanox Technologies Ltd. */ + +#ifndef __MLX5_VNET_H_ +#define __MLX5_VNET_H_ + +#include <linux/vdpa.h> +#include <linux/virtio_net.h> +#include <linux/vringh.h> +#include <linux/mlx5/driver.h> +#include <linux/mlx5/cq.h> +#include <linux/mlx5/qp.h> +#include "mlx5_vdpa.h" + +static inline u32 mlx5_vdpa_max_qps (int max_vqs) +{ +return max_vqs /2 ; +} + +#define to_mlx5_vdpa_ndev (__mvdev) container_of(__mvdev, struct mlx5_vdpa_net, mvdev) +void *mlx5_vdpa_add_dev (struct mlx5_core_dev *mdev); +void mlx5_vdpa_remove_dev (struct mlx5_vdpa_dev *mvdev); + +#endif /* __MLX5_VNET_H_ */
Signature · 1 line
2.26.0