VIRTIO_NET Emulation Offload
Subsystems (5)
cover VIRTIO_NET Emulation Offload
From: Leon Romanovsky <[email protected]>
Hi,
In this series, we introduce VIRTIO_NET_Q HW offload capability, so SW will be able to create special general object with relevant virtqueue properties.
This series is based on -rc patches:
https://lore.kernel.org/linux-rdma/[email protected]
Thanks
Yishai Hadas (5):
net/mlx5: Add Virtio Emulation related device capabilities net/mlx5: Expose vDPA emulation device capabilities IB/mlx5: Extend caps stage to handle VAR capabilities IB/mlx5: Introduce VAR object and its alloc/destroy methods IB/mlx5: Add mmap support for VAR
drivers/infiniband/hw/mlx5/main.c | 202 ++++++++++++++++++- drivers/infiniband/hw/mlx5/mlx5_ib.h | 17 ++ drivers/net/ethernet/mellanox/mlx5/core/fw.c | 7 + include/linux/mlx5/device.h | 9 + include/linux/mlx5/mlx5_ifc.h | 15 ++ include/uapi/rdma/mlx5_user_ioctl_cmds.h | 17 ++ 6 files changed, 264 insertions(+), 3 deletions(-)
--
2.20.1
PATCH 1/5 net/mlx5: Add Virtio Emulation related device capabilities
From: Yishai Hadas <[email protected]>
Add Virtio Emulation related fields to the device capabilities.
It includes a general bit to indicate whether Virtio Emulation is supported and the capabilities structure itself.
--- include/linux/mlx5/mlx5_ifc.h | 15 +++++++++++++++ 1 file changed, 15 insertions(+)
include/linux/mlx5/mlx5_ifc.h +15 -0
@@ -87,6 +87,7 @@ enum {enum {MLX5_GENERAL_OBJ_TYPES_CAP_SW_ICM = (1ULL <<MLX5_OBJ_TYPE_SW_ICM ),MLX5_GENERAL_OBJ_TYPES_CAP_GENEVE_TLV_OPT = (1ULL <<11 ), +MLX5_GENERAL_OBJ_TYPES_CAP_VIRTIO_NET_Q = (1ULL <<13 ), };enum {
@@ -953,6 +954,19 @@ struct mlx5_ifc_device_event_cap_bits {u8 user_unaffiliated_events[4 ][0x40 ]; }; +struct mlx5_ifc_device_virtio_emulation_cap_bits { +u8 reserved_at_0[0x20 ]; + +u8 reserved_at_20[0x13 ]; +u8 log_doorbell_stride[0x5 ]; +u8 reserved_at_38[0x3 ]; +u8 log_doorbell_bar_size[0x5 ]; + +u8 doorbell_bar_offset[0x40 ]; + +u8 reserved_at_80[0x780 ]; +}; +enum {MLX5_ATOMIC_CAPS_ATOMIC_SIZE_QP_1_BYTE =0x0 ,MLX5_ATOMIC_CAPS_ATOMIC_SIZE_QP_2_BYTES =0x2 ,
@@ -2751,6 +2765,7 @@ union mlx5_ifc_hca_cap_union_bits {struct mlx5_ifc_fpga_cap_bits fpga_cap;struct mlx5_ifc_tls_cap_bits tls_cap;struct mlx5_ifc_device_mem_cap_bits device_mem_cap; +struct mlx5_ifc_device_virtio_emulation_cap_bits virtio_emulation_cap;u8 reserved_at_0[0x8000 ]; };
Signature · 1 line
2.20.1
PATCH 3/5 IB/mlx5: Extend caps stage to handle VAR capabilities
In reply to Leon RomanovskyFrom: Yishai Hadas <[email protected]>
Extend caps stage to handle VAR capabilities.
--- drivers/infiniband/hw/mlx5/main.c | 40 ++++++++++++++++++++++++++-- drivers/infiniband/hw/mlx5/mlx5_ib.h | 10 +++++++ 2 files changed, 48 insertions(+), 2 deletions(-)
drivers/infiniband/hw/mlx5/main.c +38 -2
@@ -6335,6 +6335,35 @@ static const struct ib_device_ops mlx5_ib_dev_dm_ops = { .reg_dm_mr = mlx5_ib_reg_dm_mr, }; +static int mlx5_ib_init_var_table (struct mlx5_ib_dev *dev) +{ +struct mlx5_core_dev *mdev = dev->mdev; +struct mlx5_var_table *var_table = &dev->var_table; +u8 log_doorbell_bar_size; +u8 log_doorbell_stride; +u64 bar_size; + + log_doorbell_bar_size =MLX5_CAP_DEV_VDPA_EMULATION (mdev, + log_doorbell_bar_size); + log_doorbell_stride =MLX5_CAP_DEV_VDPA_EMULATION (mdev, + log_doorbell_stride); + var_table->hw_start_addr = dev->mdev->bar_addr + +MLX5_CAP64_DEV_VDPA_EMULATION (mdev, + doorbell_bar_offset); + bar_size = (1ULL << log_doorbell_bar_size) *4096 ; + var_table->stride_size =1ULL << log_doorbell_stride; + var_table->num_var_hw_entries = bar_size / var_table->stride_size; +mutex_init (&var_table->bitmap_lock); + var_table->bitmap =bitmap_zalloc (var_table->num_var_hw_entries, +GFP_KERNEL ); +return (var_table->bitmap) ?0 : -ENOMEM ; +} + +static void mlx5_ib_stage_caps_cleanup (struct mlx5_ib_dev *dev) +{ +bitmap_free (dev->var_table.bitmap); +} +static int mlx5_ib_stage_caps_init (struct mlx5_ib_dev *dev) {struct mlx5_core_dev *mdev = dev->mdev;
@@ -6422,6 +6451,13 @@ static int mlx5_ib_stage_caps_init(struct mlx5_ib_dev *dev) MLX5_CAP_GEN(dev->mdev, disable_local_lb_mc)))mutex_init (&dev->lb.mutex); +if (MLX5_CAP_GEN_64 (dev->mdev, general_obj_types) & +MLX5_GENERAL_OBJ_TYPES_CAP_VIRTIO_NET_Q ) { + err =mlx5_ib_init_var_table (dev); +if (err) +return err; + } + dev->ib_dev.use_cq_dim = true;return 0 ;
@@ -6770,7 +6806,7 @@ static const struct mlx5_ib_profile pf_profile = { mlx5_ib_stage_flow_db_cleanup),STAGE_CREATE (MLX5_IB_STAGE_CAPS , mlx5_ib_stage_caps_init, -NULL ), + mlx5_ib_stage_caps_cleanup),STAGE_CREATE (MLX5_IB_STAGE_NON_DEFAULT_CB , mlx5_ib_stage_non_default_cb, NULL),
@@ -6827,7 +6863,7 @@ const struct mlx5_ib_profile raw_eth_profile = { mlx5_ib_stage_flow_db_cleanup),STAGE_CREATE (MLX5_IB_STAGE_CAPS , mlx5_ib_stage_caps_init, -NULL ), + mlx5_ib_stage_caps_cleanup),STAGE_CREATE (MLX5_IB_STAGE_NON_DEFAULT_CB , mlx5_ib_stage_raw_eth_non_default_cb, NULL),
drivers/infiniband/hw/mlx5/mlx5_ib.h +10 -0
@@ -959,6 +959,15 @@ struct mlx5_devx_event_table {struct xarray event_xa; }; +struct mlx5_var_table { +/* serialize updating the bitmap */ +struct mutex bitmap_lock; +unsigned long *bitmap; +u64 hw_start_addr; +u32 stride_size; +u64 num_var_hw_entries; +}; +struct mlx5_ib_dev {struct ib_device ib_dev;struct mlx5_core_dev *mdev;
@@ -1013,6 +1022,7 @@ struct mlx5_ib_dev {struct mlx5_srq_table srq_table;struct mlx5_async_ctx async_ctx;struct mlx5_devx_event_table devx_event_table; +struct mlx5_var_table var_table;struct xarray sig_mrs; };
Signature · 1 line
2.20.1
PATCH 4/5 IB/mlx5: Introduce VAR object and its alloc/destroy methods
In reply to Leon RomanovskyFrom: Yishai Hadas <[email protected]>
Introduce VAR object and its alloc/destroy KABI methods. The internal implementation uses the IB core API to manage mmap/munamp calls.
--- drivers/infiniband/hw/mlx5/main.c | 157 +++++++++++++++++++++++ drivers/infiniband/hw/mlx5/mlx5_ib.h | 7 + include/uapi/rdma/mlx5_user_ioctl_cmds.h | 17 +++ 3 files changed, 181 insertions(+)
drivers/infiniband/hw/mlx5/main.c +157 -0
@@ -2078,6 +2078,7 @@ static void mlx5_ib_mmap_free(struct rdma_user_mmap_entry *entry) {struct mlx5_user_mmap_entry *mentry =to_mmmap (entry);struct mlx5_ib_dev *dev =to_mdev (entry->ucontext->device); +struct mlx5_var_table *var_table = &dev->var_table;struct mlx5_ib_dm *mdm;switch (mentry->mmap_flag) {
@@ -2087,6 +2088,12 @@ static void mlx5_ib_mmap_free(struct rdma_user_mmap_entry *entry) mdm->size);kfree (mdm);break ; +case MLX5_IB_MMAP_TYPE_VAR : +mutex_lock (&var_table->bitmap_lock); +clear_bit (mentry->page_idx, var_table->bitmap); +mutex_unlock (&var_table->bitmap_lock); +kfree (mentry); +break ;default :WARN_ON (true); }
@@ -2255,6 +2262,15 @@ static int mlx5_ib_mmap_offset(struct mlx5_ib_dev *dev,return ret; } +static u64 mlx5_entry_to_mmap_offset (struct mlx5_user_mmap_entry *entry) +{ +u16 cmd = entry->rdma_entry.start_pgoff >>16 ; +u16 index = entry->rdma_entry.start_pgoff &0xFFFF ; + +return (((index >>8 ) <<16 ) | (cmd <<MLX5_IB_MMAP_CMD_SHIFT ) | + (index &0xFF )) <<PAGE_SHIFT ; +} +static int mlx5_ib_mmap (struct ib_ucontext *ibcontext,struct vm_area_struct *vma) {struct mlx5_ib_ucontext *context =to_mucontext (ibcontext);
@@ -6034,6 +6050,145 @@ static void mlx5_ib_cleanup_multiport_master(struct mlx5_ib_dev *dev)mlx5_nic_vport_disable_roce (dev->mdev); } +static int var_obj_cleanup (struct ib_uobject *uobject, +enum rdma_remove_reason why, +struct uverbs_attr_bundle *attrs) +{ +struct mlx5_user_mmap_entry *obj = uobject->object; + +rdma_user_mmap_entry_remove (&obj->rdma_entry); +return 0 ; +} + +static struct mlx5_user_mmap_entry * +alloc_var_entry (struct mlx5_ib_ucontext *c) +{ +struct mlx5_user_mmap_entry *entry; +struct mlx5_var_table *var_table; +u32 page_idx; +int err; + + var_table = &to_mdev (c->ibucontext.device)->var_table; + entry =kzalloc (sizeof (*entry),GFP_KERNEL ); +if (!entry) +return ERR_PTR (-ENOMEM ); + +mutex_lock (&var_table->bitmap_lock); + page_idx =find_first_zero_bit (var_table->bitmap, + var_table->num_var_hw_entries); +if (page_idx >= var_table->num_var_hw_entries) { + err = -ENOSPC ; +mutex_unlock (&var_table->bitmap_lock); + goto end; + } + +set_bit (page_idx, var_table->bitmap); +mutex_unlock (&var_table->bitmap_lock); + + entry->address = var_table->hw_start_addr + + (page_idx * var_table->stride_size); + entry->page_idx = page_idx; + entry->mmap_flag =MLX5_IB_MMAP_TYPE_VAR ; + + err =rdma_user_mmap_entry_insert_range ( + &c->ibucontext, &entry->rdma_entry, var_table->stride_size, +MLX5_IB_MMAP_OFFSET_START <<16 , + (MLX5_IB_MMAP_OFFSET_END <<16 ) + (1UL <<16 ) -1 ); +if (err) + goto err_insert; + +return entry; + +err_insert: +mutex_lock (&var_table->bitmap_lock); +clear_bit (page_idx, var_table->bitmap); +mutex_unlock (&var_table->bitmap_lock); +end: +kfree (entry); +return ERR_PTR (err); +} + +static int UVERBS_HANDLER (MLX5_IB_METHOD_VAR_OBJ_ALLOC )( +struct uverbs_attr_bundle *attrs) +{ +struct ib_uobject *uobj =uverbs_attr_get_uobject ( + attrs,MLX5_IB_ATTR_VAR_OBJ_ALLOC_HANDLE ); +struct mlx5_ib_ucontext *c; +struct mlx5_user_mmap_entry *entry; +u64 mmap_offset; +u32 length; +int err; + + c =to_mucontext (ib_uverbs_get_ucontext (attrs)); +if (IS_ERR (c)) +return PTR_ERR (c); + + entry =alloc_var_entry (c); +if (IS_ERR (entry)) +return PTR_ERR (entry); + + mmap_offset =mlx5_entry_to_mmap_offset (entry); + length = entry->rdma_entry.npages *PAGE_SIZE ; + uobj->object = entry; + + err =uverbs_copy_to (attrs,MLX5_IB_ATTR_VAR_OBJ_ALLOC_MMAP_OFFSET , + &mmap_offset,sizeof (mmap_offset)); +if (err) + goto err; + + err =uverbs_copy_to (attrs,MLX5_IB_ATTR_VAR_OBJ_ALLOC_PAGE_ID , + &entry->page_idx,sizeof (entry->page_idx)); +if (err) + goto err; + + err =uverbs_copy_to (attrs,MLX5_IB_ATTR_VAR_OBJ_ALLOC_MMAP_LENGTH , + &length,sizeof (length)); +if (err) + goto err; + +return 0 ; + +err: +rdma_user_mmap_entry_remove (&entry->rdma_entry); +return err; +} + +DECLARE_UVERBS_NAMED_METHOD ( +MLX5_IB_METHOD_VAR_OBJ_ALLOC , +UVERBS_ATTR_IDR (MLX5_IB_ATTR_VAR_OBJ_ALLOC_HANDLE , +MLX5_IB_OBJECT_VAR , +UVERBS_ACCESS_NEW , +UA_MANDATORY ), +UVERBS_ATTR_PTR_OUT (MLX5_IB_ATTR_VAR_OBJ_ALLOC_PAGE_ID , +UVERBS_ATTR_TYPE (u32), +UA_MANDATORY ), +UVERBS_ATTR_PTR_OUT (MLX5_IB_ATTR_VAR_OBJ_ALLOC_MMAP_LENGTH , +UVERBS_ATTR_TYPE (u32), +UA_MANDATORY ), +UVERBS_ATTR_PTR_OUT (MLX5_IB_ATTR_VAR_OBJ_ALLOC_MMAP_OFFSET , +UVERBS_ATTR_TYPE (u64), +UA_MANDATORY )); + +DECLARE_UVERBS_NAMED_METHOD_DESTROY ( +MLX5_IB_METHOD_VAR_OBJ_DESTROY , +UVERBS_ATTR_IDR (MLX5_IB_ATTR_VAR_OBJ_DESTROY_HANDLE , +MLX5_IB_OBJECT_VAR , +UVERBS_ACCESS_DESTROY , +UA_MANDATORY )); + +DECLARE_UVERBS_NAMED_OBJECT (MLX5_IB_OBJECT_VAR , +UVERBS_TYPE_ALLOC_IDR (var_obj_cleanup), + &UVERBS_METHOD (MLX5_IB_METHOD_VAR_OBJ_ALLOC ), + &UVERBS_METHOD (MLX5_IB_METHOD_VAR_OBJ_DESTROY )); + +static bool var_is_supported (struct ib_device *device) +{ +struct mlx5_ib_dev *dev =to_mdev (device); + +return (MLX5_CAP_GEN_64 (dev->mdev, general_obj_types) & +MLX5_GENERAL_OBJ_TYPES_CAP_VIRTIO_NET_Q ); +} + ADD_UVERBS_ATTRIBUTES_SIMPLE( mlx5_ib_dm, UVERBS_OBJECT_DM,
@@ -6064,6 +6219,8 @@ static const struct uapi_definition mlx5_ib_defs[] = {UAPI_DEF_CHAIN_OBJ_TREE (UVERBS_OBJECT_FLOW_ACTION , &mlx5_ib_flow_action),UAPI_DEF_CHAIN_OBJ_TREE (UVERBS_OBJECT_DM , &mlx5_ib_dm), +UAPI_DEF_CHAIN_OBJ_TREE_NAMED (MLX5_IB_OBJECT_VAR , + UAPI_DEF_IS_OBJ_SUPPORTED(var_is_supported)), {} };
drivers/infiniband/hw/mlx5/mlx5_ib.h +7 -0
@@ -71,6 +71,11 @@#define MLX5_MKEY_PAGE_SHIFT_MASK __mlx5_mask(mkc, log_page_size) +enum { +MLX5_IB_MMAP_OFFSET_START =9 , +MLX5_IB_MMAP_OFFSET_END =255 , +}; +enum {MLX5_IB_MMAP_CMD_SHIFT =8 ,MLX5_IB_MMAP_CMD_MASK =0xff ,
@@ -120,6 +125,7 @@ enum {enum mlx5_ib_mmap_type {MLX5_IB_MMAP_TYPE_MEMIC =1 , +MLX5_IB_MMAP_TYPE_VAR =2 , };#define MLX5_LOG_SW_ICM_BLOCK_SIZE (dev) \
@@ -563,6 +569,7 @@ struct mlx5_user_mmap_entry {struct rdma_user_mmap_entry rdma_entry;u8 mmap_flag;u64 address; +u32 page_idx; };struct mlx5_ib_dm {
include/uapi/rdma/mlx5_user_ioctl_cmds.h +17 -0
@@ -115,6 +115,22 @@ enum mlx5_ib_devx_obj_methods { MLX5_IB_METHOD_DEVX_OBJ_ASYNC_QUERY, }; +enum mlx5_ib_var_alloc_attrs { +MLX5_IB_ATTR_VAR_OBJ_ALLOC_HANDLE = (1U <<UVERBS_ID_NS_SHIFT ), +MLX5_IB_ATTR_VAR_OBJ_ALLOC_MMAP_OFFSET , +MLX5_IB_ATTR_VAR_OBJ_ALLOC_MMAP_LENGTH , +MLX5_IB_ATTR_VAR_OBJ_ALLOC_PAGE_ID , +}; + +enum mlx5_ib_var_obj_destroy_attrs { +MLX5_IB_ATTR_VAR_OBJ_DESTROY_HANDLE = (1U <<UVERBS_ID_NS_SHIFT ), +}; + +enum mlx5_ib_var_obj_methods { +MLX5_IB_METHOD_VAR_OBJ_ALLOC = (1U <<UVERBS_ID_NS_SHIFT ), +MLX5_IB_METHOD_VAR_OBJ_DESTROY , +}; +enum mlx5_ib_devx_umem_reg_attrs {MLX5_IB_ATTR_DEVX_UMEM_REG_HANDLE = (1U <<UVERBS_ID_NS_SHIFT ),MLX5_IB_ATTR_DEVX_UMEM_REG_ADDR ,
@@ -156,6 +172,7 @@ enum mlx5_ib_objects {MLX5_IB_OBJECT_FLOW_MATCHER ,MLX5_IB_OBJECT_DEVX_ASYNC_CMD_FD ,MLX5_IB_OBJECT_DEVX_ASYNC_EVENT_FD , + MLX5_IB_OBJECT_VAR, }; enum mlx5_ib_flow_matcher_create_attrs {
Signature · 1 line
2.20.1
PATCH 5/5 IB/mlx5: Add mmap support for VAR
In reply to Leon RomanovskyFrom: Yishai Hadas <[email protected]>
Add mmap support for VAR, it uses the 'offset' command mode with involvement of IB core APIs to find the previously allocated mmap entry.
--- drivers/infiniband/hw/mlx5/main.c | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-)
drivers/infiniband/hw/mlx5/main.c +4 -1
@@ -2253,7 +2253,10 @@ static int mlx5_ib_mmap_offset(struct mlx5_ib_dev *dev, mentry =to_mmmap (entry); pfn = (mentry->address >>PAGE_SHIFT ); - prot =pgprot_writecombine (vma->vm_page_prot); +if (mentry->mmap_flag ==MLX5_IB_MMAP_TYPE_VAR ) + prot =pgprot_noncached (vma->vm_page_prot); +else + prot =pgprot_writecombine (vma->vm_page_prot); ret = rdma_user_mmap_io(ucontext, vma, pfn, entry->npages * PAGE_SIZE, prot,
Signature · 1 line
2.20.1
PATCH 2/5 net/mlx5: Expose vDPA emulation device capabilities
In reply to Leon RomanovskyFrom: Yishai Hadas <[email protected]>
Expose vDPA emulation device capabilities from the core layer. It includes reading the capabilities from the firmware and exposing helper functions to access the data.
--- drivers/net/ethernet/mellanox/mlx5/core/fw.c | 7 +++++++ include/linux/mlx5/device.h | 9 +++++++++ 2 files changed, 16 insertions(+)
drivers/net/ethernet/mellanox/mlx5/core/fw.c +7 -0
@@ -245,6 +245,13 @@ int mlx5_query_hca_caps(struct mlx5_core_dev *dev)return err; } +if (MLX5_CAP_GEN_64 (dev, general_obj_types) & +MLX5_GENERAL_OBJ_TYPES_CAP_VIRTIO_NET_Q ) { + err =mlx5_core_get_caps (dev,MLX5_CAP_VDPA_EMULATION ); +if (err) +return err; + } +return 0 ; }
include/linux/mlx5/device.h +9 -0
@@ -1105,6 +1105,7 @@ enum mlx5_cap_type {MLX5_CAP_DEV_MEM ,MLX5_CAP_RESERVED_16 ,MLX5_CAP_TLS , +MLX5_CAP_VDPA_EMULATION =0x13 ,MLX5_CAP_DEV_EVENT =0x14 ,/* NUM OF CAP Types */ MLX5_CAP_NUM
@@ -1297,6 +1298,14 @@ enum mlx5_qcam_feature_groups {#define MLX5_CAP_DEV_EVENT (mdev, cap)\ MLX5_ADDR_OF(device_event_cap, (mdev)->caps.hca_cur[MLX5_CAP_DEV_EVENT], cap) +#define MLX5_CAP_DEV_VDPA_EMULATION (mdev, cap)\ + MLX5_GET(device_virtio_emulation_cap, \ + (mdev)->caps.hca_cur[MLX5_CAP_VDPA_EMULATION], cap) + +#define MLX5_CAP64_DEV_VDPA_EMULATION (mdev, cap)\ + MLX5_GET64(device_virtio_emulation_cap, \ + (mdev)->caps.hca_cur[MLX5_CAP_VDPA_EMULATION], cap) +enum {MLX5_CMD_STAT_OK =0x0 ,MLX5_CMD_STAT_INT_ERR =0x1 ,
Signature · 1 line
2.20.1
Re: [PATCH rdma-next 4/5] IB/mlx5: Introduce VAR object and its alloc/destroy methods
In reply to Leon RomanovskyQuoted 231 lines · Leon Romanovsky
From: Yishai Hadas <[email protected]>
Introduce VAR object and its alloc/destroy KABI methods. The internal implementation uses the IB core API to manage mmap/munamp calls.
Signed-off-by Yishai Hadas <[email protected]>Signed-off-by Leon Romanovsky <[email protected]>drivers/infiniband/hw/mlx5/main.c | 157 +++++++++++++++++++++++ drivers/infiniband/hw/mlx5/mlx5_ib.h | 7 + include/uapi/rdma/mlx5_user_ioctl_cmds.h | 17 +++ 3 files changed, 181 insertions(+)drivers/infiniband/hw/mlx5/main.c +158
-0+++ b/drivers/infiniband/hw/mlx5/main.c@@ -2078,6 +2078,7 @@ static void mlx5_ib_mmap_free(struct rdma_user_mmap_entry *entry) {struct mlx5_user_mmap_entry *mentry =to_mmmap (entry);struct mlx5_ib_dev *dev =to_mdev (entry->ucontext->device); +struct mlx5_var_table *var_table = &dev->var_table;struct mlx5_ib_dm *mdm;switch (mentry->mmap_flag) {@@ -2087,6 +2088,12 @@ static void mlx5_ib_mmap_free(struct rdma_user_mmap_entry *entry) mdm->size);kfree (mdm);break ; +case MLX5_IB_MMAP_TYPE_VAR : +mutex_lock (&var_table->bitmap_lock); +clear_bit (mentry->page_idx, var_table->bitmap); +mutex_unlock (&var_table->bitmap_lock); +kfree (mentry); +break ;default :WARN_ON (true); }@@ -2255,6 +2262,15 @@ static int mlx5_ib_mmap_offset(struct mlx5_ib_dev *dev,return ret; } +static u64 mlx5_entry_to_mmap_offset (struct mlx5_user_mmap_entry *entry) +{ +u16 cmd = entry->rdma_entry.start_pgoff >>16 ; +u16 index = entry->rdma_entry.start_pgoff &0xFFFF ; + +return (((index >>8 ) <<16 ) | (cmd <<MLX5_IB_MMAP_CMD_SHIFT ) | + (index &0xFF )) <<PAGE_SHIFT ; +} +static int mlx5_ib_mmap (struct ib_ucontext *ibcontext,struct vm_area_struct *vma) {struct mlx5_ib_ucontext *context =to_mucontext (ibcontext);@@ -6034,6 +6050,145 @@ static void mlx5_ib_cleanup_multiport_master(struct mlx5_ib_dev *dev)mlx5_nic_vport_disable_roce (dev->mdev); } +static int var_obj_cleanup (struct ib_uobject *uobject, +enum rdma_remove_reason why, +struct uverbs_attr_bundle *attrs) +{ +struct mlx5_user_mmap_entry *obj = uobject->object; + +rdma_user_mmap_entry_remove (&obj->rdma_entry); +return 0 ; +} + +static struct mlx5_user_mmap_entry * +alloc_var_entry (struct mlx5_ib_ucontext *c) +{ +struct mlx5_user_mmap_entry *entry; +struct mlx5_var_table *var_table; +u32 page_idx; +int err; + + var_table = &to_mdev (c->ibucontext.device)->var_table; + entry =kzalloc (sizeof (*entry),GFP_KERNEL ); +if (!entry) +return ERR_PTR (-ENOMEM ); + +mutex_lock (&var_table->bitmap_lock); + page_idx =find_first_zero_bit (var_table->bitmap, + var_table->num_var_hw_entries); +if (page_idx >= var_table->num_var_hw_entries) { + err = -ENOSPC ; +mutex_unlock (&var_table->bitmap_lock); + goto end; + } + +set_bit (page_idx, var_table->bitmap); +mutex_unlock (&var_table->bitmap_lock); + + entry->address = var_table->hw_start_addr + + (page_idx * var_table->stride_size); + entry->page_idx = page_idx; + entry->mmap_flag =MLX5_IB_MMAP_TYPE_VAR ; + + err =rdma_user_mmap_entry_insert_range ( + &c->ibucontext, &entry->rdma_entry, var_table->stride_size, +MLX5_IB_MMAP_OFFSET_START <<16 , + (MLX5_IB_MMAP_OFFSET_END <<16 ) + (1UL <<16 ) -1 ); +if (err) + goto err_insert; + +return entry; + +err_insert: +mutex_lock (&var_table->bitmap_lock); +clear_bit (page_idx, var_table->bitmap); +mutex_unlock (&var_table->bitmap_lock); +end: +kfree (entry); +return ERR_PTR (err); +} + +static int UVERBS_HANDLER (MLX5_IB_METHOD_VAR_OBJ_ALLOC )( +struct uverbs_attr_bundle *attrs) +{ +struct ib_uobject *uobj =uverbs_attr_get_uobject ( + attrs,MLX5_IB_ATTR_VAR_OBJ_ALLOC_HANDLE ); +struct mlx5_ib_ucontext *c; +struct mlx5_user_mmap_entry *entry; +u64 mmap_offset; +u32 length; +int err; + + c =to_mucontext (ib_uverbs_get_ucontext (attrs)); +if (IS_ERR (c)) +return PTR_ERR (c); + + entry =alloc_var_entry (c); +if (IS_ERR (entry)) +return PTR_ERR (entry); + + mmap_offset =mlx5_entry_to_mmap_offset (entry); + length = entry->rdma_entry.npages *PAGE_SIZE ; + uobj->object = entry; + + err =uverbs_copy_to (attrs,MLX5_IB_ATTR_VAR_OBJ_ALLOC_MMAP_OFFSET , + &mmap_offset,sizeof (mmap_offset)); +if (err) + goto err; + + err =uverbs_copy_to (attrs,MLX5_IB_ATTR_VAR_OBJ_ALLOC_PAGE_ID , + &entry->page_idx,sizeof (entry->page_idx)); +if (err) + goto err; + + err =uverbs_copy_to (attrs,MLX5_IB_ATTR_VAR_OBJ_ALLOC_MMAP_LENGTH , + &length,sizeof (length)); +if (err) + goto err; + +return 0 ; + +err: +rdma_user_mmap_entry_remove (&entry->rdma_entry); +return err; +} + +DECLARE_UVERBS_NAMED_METHOD ( +MLX5_IB_METHOD_VAR_OBJ_ALLOC , +UVERBS_ATTR_IDR (MLX5_IB_ATTR_VAR_OBJ_ALLOC_HANDLE , +MLX5_IB_OBJECT_VAR , +UVERBS_ACCESS_NEW , +UA_MANDATORY ), +UVERBS_ATTR_PTR_OUT (MLX5_IB_ATTR_VAR_OBJ_ALLOC_PAGE_ID , +UVERBS_ATTR_TYPE (u32), +UA_MANDATORY ), +UVERBS_ATTR_PTR_OUT (MLX5_IB_ATTR_VAR_OBJ_ALLOC_MMAP_LENGTH , +UVERBS_ATTR_TYPE (u32), +UA_MANDATORY ), +UVERBS_ATTR_PTR_OUT (MLX5_IB_ATTR_VAR_OBJ_ALLOC_MMAP_OFFSET , +UVERBS_ATTR_TYPE (u64), +UA_MANDATORY )); + +DECLARE_UVERBS_NAMED_METHOD_DESTROY ( +MLX5_IB_METHOD_VAR_OBJ_DESTROY , +UVERBS_ATTR_IDR (MLX5_IB_ATTR_VAR_OBJ_DESTROY_HANDLE , +MLX5_IB_OBJECT_VAR , +UVERBS_ACCESS_DESTROY , +UA_MANDATORY )); + +DECLARE_UVERBS_NAMED_OBJECT (MLX5_IB_OBJECT_VAR , +UVERBS_TYPE_ALLOC_IDR (var_obj_cleanup), + &UVERBS_METHOD (MLX5_IB_METHOD_VAR_OBJ_ALLOC ), + &UVERBS_METHOD (MLX5_IB_METHOD_VAR_OBJ_DESTROY )); + +static bool var_is_supported (struct ib_device *device) +{ +struct mlx5_ib_dev *dev =to_mdev (device); + +return (MLX5_CAP_GEN_64 (dev->mdev, general_obj_types) & +MLX5_GENERAL_OBJ_TYPES_CAP_VIRTIO_NET_Q ); +} + ADD_UVERBS_ATTRIBUTES_SIMPLE( mlx5_ib_dm, UVERBS_OBJECT_DM,@@ -6064,6 +6219,8 @@ static const struct uapi_definition mlx5_ib_defs[] = {UAPI_DEF_CHAIN_OBJ_TREE (UVERBS_OBJECT_FLOW_ACTION , &mlx5_ib_flow_action),UAPI_DEF_CHAIN_OBJ_TREE (UVERBS_OBJECT_DM , &mlx5_ib_dm), +UAPI_DEF_CHAIN_OBJ_TREE_NAMED (MLX5_IB_OBJECT_VAR , + UAPI_DEF_IS_OBJ_SUPPORTED(var_is_supported)), {} };drivers/infiniband/hw/mlx5/mlx5_ib.h +7
-0+++ b/drivers/infiniband/hw/mlx5/mlx5_ib.h@@ -71,6 +71,11 @@#define MLX5_MKEY_PAGE_SHIFT_MASK __mlx5_mask(mkc, log_page_size) +enum { +MLX5_IB_MMAP_OFFSET_START =9 , +MLX5_IB_MMAP_OFFSET_END =255 , +}; +enum {MLX5_IB_MMAP_CMD_SHIFT =8 ,MLX5_IB_MMAP_CMD_MASK =0xff ,@@ -120,6 +125,7 @@ enum {enum mlx5_ib_mmap_type {MLX5_IB_MMAP_TYPE_MEMIC =1 , +MLX5_IB_MMAP_TYPE_VAR =2 , };#define MLX5_LOG_SW_ICM_BLOCK_SIZE (dev) \
drivers/infiniband/hw/mlx5/mlx5_ib.h struct mlx5_user_mmap_entry { · line 572
Quoted 4 lines
drivers/infiniband/hw/mlx5/mlx5_ib.h +1
-0@@ -563,6 +569,7 @@ struct mlx5_user_mmap_entry {struct rdma_user_mmap_entry rdma_entry;u8 mmap_flag;u64 address; +u32 page_idx;
Why are we storing this in the global struct when it is never read except by the caller of alloc_var_entry()? Return it from alloc_var_entry?
Also the final patch in the series should be here as at this point mmap will succeed but return the wrong cachability flags.
Since Leon is away I can fix this two things if you agree.
Jason
Quoted 7 lines · Leon Romanovsky
From: Leon Romanovsky <[email protected]>
Hi,
In this series, we introduce
VIRTIO_NET_QHW offload capability, so SW will be able to create special general object with relevant virtqueue properties.
Quoted 8 lines
This series is based on -rc patches:
https://lore.kernel.org/linux-rdma/[email protected]Thanks
Yishai Hadas (5):
net/mlx5: Add Virtio Emulation related device capabilities net/mlx5: Expose vDPA emulation device capabilities
This series looks OK enough to me. Saeed can you update the share branch with the two patches?
https://patchwork.kernel.org/patch/11287947/
https://patchwork.kernel.org/patch/11287955/
Thanks,
Jason
Re: [PATCH rdma-next 4/5] IB/mlx5: Introduce VAR object and its alloc/destroy methods
In reply to Jason GunthorpeQuoted 230 lines · Jason Gunthorpe
Quoted 229 lines · Leon Romanovsky
From: Yishai Hadas <[email protected]>
Introduce VAR object and its alloc/destroy KABI methods. The internal implementation uses the IB core API to manage mmap/munamp calls.
Signed-off-by Yishai Hadas <[email protected]>Signed-off-by Leon Romanovsky <[email protected]>drivers/infiniband/hw/mlx5/main.c | 157 +++++++++++++++++++++++ drivers/infiniband/hw/mlx5/mlx5_ib.h | 7 + include/uapi/rdma/mlx5_user_ioctl_cmds.h | 17 +++ 3 files changed, 181 insertions(+)drivers/infiniband/hw/mlx5/main.c +158
-0+++ b/drivers/infiniband/hw/mlx5/main.c@@ -2078,6 +2078,7 @@ static void mlx5_ib_mmap_free(struct rdma_user_mmap_entry *entry) {struct mlx5_user_mmap_entry *mentry =to_mmmap (entry);struct mlx5_ib_dev *dev =to_mdev (entry->ucontext->device); +struct mlx5_var_table *var_table = &dev->var_table;struct mlx5_ib_dm *mdm;switch (mentry->mmap_flag) {@@ -2087,6 +2088,12 @@ static void mlx5_ib_mmap_free(struct rdma_user_mmap_entry *entry) mdm->size);kfree (mdm);break ; +case MLX5_IB_MMAP_TYPE_VAR : +mutex_lock (&var_table->bitmap_lock); +clear_bit (mentry->page_idx, var_table->bitmap); +mutex_unlock (&var_table->bitmap_lock); +kfree (mentry); +break ;default :WARN_ON (true); }@@ -2255,6 +2262,15 @@ static int mlx5_ib_mmap_offset(struct mlx5_ib_dev *dev,return ret; } +static u64 mlx5_entry_to_mmap_offset (struct mlx5_user_mmap_entry *entry) +{ +u16 cmd = entry->rdma_entry.start_pgoff >>16 ; +u16 index = entry->rdma_entry.start_pgoff &0xFFFF ; + +return (((index >>8 ) <<16 ) | (cmd <<MLX5_IB_MMAP_CMD_SHIFT ) | + (index &0xFF )) <<PAGE_SHIFT ; +} +static int mlx5_ib_mmap (struct ib_ucontext *ibcontext,struct vm_area_struct *vma) {struct mlx5_ib_ucontext *context =to_mucontext (ibcontext);@@ -6034,6 +6050,145 @@ static void mlx5_ib_cleanup_multiport_master(struct mlx5_ib_dev *dev)mlx5_nic_vport_disable_roce (dev->mdev); } +static int var_obj_cleanup (struct ib_uobject *uobject, +enum rdma_remove_reason why, +struct uverbs_attr_bundle *attrs) +{ +struct mlx5_user_mmap_entry *obj = uobject->object; + +rdma_user_mmap_entry_remove (&obj->rdma_entry); +return 0 ; +} + +static struct mlx5_user_mmap_entry * +alloc_var_entry (struct mlx5_ib_ucontext *c) +{ +struct mlx5_user_mmap_entry *entry; +struct mlx5_var_table *var_table; +u32 page_idx; +int err; + + var_table = &to_mdev (c->ibucontext.device)->var_table; + entry =kzalloc (sizeof (*entry),GFP_KERNEL ); +if (!entry) +return ERR_PTR (-ENOMEM ); + +mutex_lock (&var_table->bitmap_lock); + page_idx =find_first_zero_bit (var_table->bitmap, + var_table->num_var_hw_entries); +if (page_idx >= var_table->num_var_hw_entries) { + err = -ENOSPC ; +mutex_unlock (&var_table->bitmap_lock); + goto end; + } + +set_bit (page_idx, var_table->bitmap); +mutex_unlock (&var_table->bitmap_lock); + + entry->address = var_table->hw_start_addr + + (page_idx * var_table->stride_size); + entry->page_idx = page_idx; + entry->mmap_flag =MLX5_IB_MMAP_TYPE_VAR ; + + err =rdma_user_mmap_entry_insert_range ( + &c->ibucontext, &entry->rdma_entry, var_table->stride_size, +MLX5_IB_MMAP_OFFSET_START <<16 , + (MLX5_IB_MMAP_OFFSET_END <<16 ) + (1UL <<16 ) -1 ); +if (err) + goto err_insert; + +return entry; + +err_insert: +mutex_lock (&var_table->bitmap_lock); +clear_bit (page_idx, var_table->bitmap); +mutex_unlock (&var_table->bitmap_lock); +end: +kfree (entry); +return ERR_PTR (err); +} + +static int UVERBS_HANDLER (MLX5_IB_METHOD_VAR_OBJ_ALLOC )( +struct uverbs_attr_bundle *attrs) +{ +struct ib_uobject *uobj =uverbs_attr_get_uobject ( + attrs,MLX5_IB_ATTR_VAR_OBJ_ALLOC_HANDLE ); +struct mlx5_ib_ucontext *c; +struct mlx5_user_mmap_entry *entry; +u64 mmap_offset; +u32 length; +int err; + + c =to_mucontext (ib_uverbs_get_ucontext (attrs)); +if (IS_ERR (c)) +return PTR_ERR (c); + + entry =alloc_var_entry (c); +if (IS_ERR (entry)) +return PTR_ERR (entry); + + mmap_offset =mlx5_entry_to_mmap_offset (entry); + length = entry->rdma_entry.npages *PAGE_SIZE ; + uobj->object = entry; + + err =uverbs_copy_to (attrs,MLX5_IB_ATTR_VAR_OBJ_ALLOC_MMAP_OFFSET , + &mmap_offset,sizeof (mmap_offset)); +if (err) + goto err; + + err =uverbs_copy_to (attrs,MLX5_IB_ATTR_VAR_OBJ_ALLOC_PAGE_ID , + &entry->page_idx,sizeof (entry->page_idx)); +if (err) + goto err; + + err =uverbs_copy_to (attrs,MLX5_IB_ATTR_VAR_OBJ_ALLOC_MMAP_LENGTH , + &length,sizeof (length)); +if (err) + goto err; + +return 0 ; + +err: +rdma_user_mmap_entry_remove (&entry->rdma_entry); +return err; +} + +DECLARE_UVERBS_NAMED_METHOD ( +MLX5_IB_METHOD_VAR_OBJ_ALLOC , +UVERBS_ATTR_IDR (MLX5_IB_ATTR_VAR_OBJ_ALLOC_HANDLE , +MLX5_IB_OBJECT_VAR , +UVERBS_ACCESS_NEW , +UA_MANDATORY ), +UVERBS_ATTR_PTR_OUT (MLX5_IB_ATTR_VAR_OBJ_ALLOC_PAGE_ID , +UVERBS_ATTR_TYPE (u32), +UA_MANDATORY ), +UVERBS_ATTR_PTR_OUT (MLX5_IB_ATTR_VAR_OBJ_ALLOC_MMAP_LENGTH , +UVERBS_ATTR_TYPE (u32), +UA_MANDATORY ), +UVERBS_ATTR_PTR_OUT (MLX5_IB_ATTR_VAR_OBJ_ALLOC_MMAP_OFFSET , +UVERBS_ATTR_TYPE (u64), +UA_MANDATORY )); + +DECLARE_UVERBS_NAMED_METHOD_DESTROY ( +MLX5_IB_METHOD_VAR_OBJ_DESTROY , +UVERBS_ATTR_IDR (MLX5_IB_ATTR_VAR_OBJ_DESTROY_HANDLE , +MLX5_IB_OBJECT_VAR , +UVERBS_ACCESS_DESTROY , +UA_MANDATORY )); + +DECLARE_UVERBS_NAMED_OBJECT (MLX5_IB_OBJECT_VAR , +UVERBS_TYPE_ALLOC_IDR (var_obj_cleanup), + &UVERBS_METHOD (MLX5_IB_METHOD_VAR_OBJ_ALLOC ), + &UVERBS_METHOD (MLX5_IB_METHOD_VAR_OBJ_DESTROY )); + +static bool var_is_supported (struct ib_device *device) +{ +struct mlx5_ib_dev *dev =to_mdev (device); + +return (MLX5_CAP_GEN_64 (dev->mdev, general_obj_types) & +MLX5_GENERAL_OBJ_TYPES_CAP_VIRTIO_NET_Q ); +} + ADD_UVERBS_ATTRIBUTES_SIMPLE( mlx5_ib_dm, UVERBS_OBJECT_DM,@@ -6064,6 +6219,8 @@ static const struct uapi_definition mlx5_ib_defs[] = {UAPI_DEF_CHAIN_OBJ_TREE (UVERBS_OBJECT_FLOW_ACTION , &mlx5_ib_flow_action),UAPI_DEF_CHAIN_OBJ_TREE (UVERBS_OBJECT_DM , &mlx5_ib_dm), +UAPI_DEF_CHAIN_OBJ_TREE_NAMED (MLX5_IB_OBJECT_VAR , + UAPI_DEF_IS_OBJ_SUPPORTED(var_is_supported)), {} };drivers/infiniband/hw/mlx5/mlx5_ib.h +7
-0+++ b/drivers/infiniband/hw/mlx5/mlx5_ib.h@@ -71,6 +71,11 @@#define MLX5_MKEY_PAGE_SHIFT_MASK __mlx5_mask(mkc, log_page_size) +enum { +MLX5_IB_MMAP_OFFSET_START =9 , +MLX5_IB_MMAP_OFFSET_END =255 , +}; +enum {MLX5_IB_MMAP_CMD_SHIFT =8 ,MLX5_IB_MMAP_CMD_MASK =0xff ,@@ -120,6 +125,7 @@ enum {enum mlx5_ib_mmap_type {MLX5_IB_MMAP_TYPE_MEMIC =1 , +MLX5_IB_MMAP_TYPE_VAR =2 , };
Quoted 11 lines
drivers/infiniband/hw/mlx5/mlx5_ib.h struct mlx5_user_mmap_entry { · line 572
Quoted 6 lines
drivers/infiniband/hw/mlx5/mlx5_ib.h +1
-0#define MLX5_LOG_SW_ICM_BLOCK_SIZE (dev) \@@ -563,6 +569,7 @@ struct mlx5_user_mmap_entry {struct rdma_user_mmap_entry rdma_entry;u8 mmap_flag;u64 address; +u32 page_idx;Why are we storing this in the global struct when it is never read except by the caller of
alloc_var_entry()? Return it fromalloc_var_entry?
It's required as part of mlx5_ib_mmap_free() to claer the matching bit map entry of the device var table, see above in this patch.
Quoted 3 lines
Also the final patch in the series should be here as at this point mmap will succeed but return the wrong cachability flags.
Right, let's squash it to this patch.
Quoted 1 line
Since Leon is away I can fix this two things if you agree.
Yes, thanks.
Yishai
Leon Romanovsky
AppliedQuoted 11 lines · Jason Gunthorpe
Quoted 10 lines · Leon Romanovsky
From: Leon Romanovsky <[email protected]>
Hi,
In this series, we introduce
VIRTIO_NET_QHW offload capability, so SW will be able to create special general object with relevant virtqueue properties.This series is based on -rc patches:
https://lore.kernel.org/linux-rdma/[email protected]
Quoted 8 lines
Quoted 5 lines
Thanks
Yishai Hadas (5):
net/mlx5: Add Virtio Emulation related device capabilities net/mlx5: Expose vDPA emulation device capabilitiesThis series looks OK enough to me. Saeed can you update the share branch with the two patches?
Merged, thanks,
ca1992c62cad net/mlx5: Expose vDPA emulation device capabilities 90fbca595243 net/mlx5: Add Virtio Emulation related device capabilities
Quoted 18 lines · Leon Romanovsky
Quoted 17 lines · Jason Gunthorpe
Quoted 15 lines · Leon Romanovsky
From: Leon Romanovsky <[email protected]>
Hi,
In this series, we introduce
VIRTIO_NET_QHW offload capability, so SW will be able to create special general object with relevant virtqueue properties.This series is based on -rc patches:
https://lore.kernel.org/linux-rdma/[email protected]Thanks
Yishai Hadas (5):
net/mlx5: Add Virtio Emulation related device capabilities net/mlx5: Expose vDPA emulation device capabilities
Quoted 7 lines
Quoted 2 lines
This series looks OK enough to me. Saeed can you update the share branch with the two patches?
Merged, thanks,
ca1992c62cadnet/mlx5: Expose vDPA emulation device capabilities90fbca595243net/mlx5: Add Virtio Emulation related device capabilities
Done, thanks
Jason