Support larger than 4K pages in DevX UMEMs
Subsystems (2)
cover Support larger than 4K pages in DevX UMEMs
From: Leon Romanovsky <[email protected]>
Hi,
This series removes existing limitation from DevX UMEM and allows user space to specify the page sizes which it can accept.
Thanks
Jason Gunthorpe (1):
RDMA/mlx5: Allow larger pages in DevX umem
Yishai Hadas (2):
IB/core: Drop WARN_ON() from ib_umem_find_best_pgsz() IB/core: Split uverbs_get_const/default to consider target type
drivers/infiniband/core/umem.c | 4 -- drivers/infiniband/core/uverbs_ioctl.c | 32 ++++++++-- drivers/infiniband/hw/mlx5/devx.c | 64 ++++++++++++++++--- drivers/infiniband/hw/mlx5/main.c | 1 + include/rdma/uverbs_ioctl.h | 80 ++++++++++++++++++++---- include/uapi/rdma/mlx5_user_ioctl_cmds.h | 1 + 6 files changed, 151 insertions(+), 31 deletions(-)
--
2.29.2
PATCH 1/3 IB/core: Drop WARN_ON() from ib_umem_find_best_pgsz()
From: Yishai Hadas <[email protected]>
The WARN_ON() issued as part of ib_umem_find_best_pgsz() blocked cases when only page sizes larger than PAGE_SIZE were set, drop it to enable those cases.
In addition, there is no need to have a specific check for zero pgsz_bitmap, the function will do its job and return 0 at the end if nothing match will be found.
--- drivers/infiniband/core/umem.c | 4 ---- 1 file changed, 4 deletions(-)
drivers/infiniband/core/umem.c +0 -4
@@ -100,10 +100,6 @@ unsigned long ib_umem_find_best_pgsz(struct ib_umem *umem,*/ pgsz_bitmap &=GENMASK (BITS_PER_LONG -1 ,PAGE_SHIFT ); -/* At minimum, drivers must support PAGE_SIZE or smaller */ -if (WARN_ON (!(pgsz_bitmap &GENMASK (PAGE_SHIFT ,0 )))) -return 0 ; - umem->iova = va = virt;/* The best result is the smallest page size that results in the minimum * number of required pages. Compute the largest page size that could
Signature · 1 line
2.29.2
PATCH 2/3 IB/core: Split uverbs_get_const/default to consider target type
In reply to Leon RomanovskyFrom: Yishai Hadas <[email protected]>
Change uverbs_get_const/uverbs_get_const_default to work properly with both signed/unsigned parameters.
Current APIs mix s64 and u64 which leads to incorrect check when u64 value was supplied and its upper bit was set. In that case uverbs_get_const() / uverbs_get_const_default() lower bound check may fail unexpectedly, target is unsigned (lower bound is 0) but value became negative as of the s64 usage.
Split to have two different APIs, no change to callers as the required API will be called internally according to the target type.
--- drivers/infiniband/core/uverbs_ioctl.c | 32 +++++++++-- drivers/infiniband/hw/mlx5/main.c | 1 + include/rdma/uverbs_ioctl.h | 80 +++++++++++++++++++++----- 3 files changed, 96 insertions(+), 17 deletions(-)
drivers/infiniband/core/uverbs_ioctl.c +28 -4
@@ -752,9 +752,10 @@ int uverbs_output_written(const struct uverbs_attr_bundle *bundle, size_t idx)return uverbs_set_output (bundle, attr); } -int _uverbs_get_const (s64 *to,const struct uverbs_attr_bundle *attrs_bundle, -size_t idx,s64 lower_bound,u64 upper_bound, -s64 *def_val) +int _uverbs_get_const_signed (s64 *to, +const struct uverbs_attr_bundle *attrs_bundle, +size_t idx,s64 lower_bound,u64 upper_bound, +s64 *def_val) {const struct uverbs_attr *attr;
@@ -773,7 +774,30 @@ int _uverbs_get_const(s64 *to, const struct uverbs_attr_bundle *attrs_bundle,return 0 ; } -EXPORT_SYMBOL (_uverbs_get_const); +EXPORT_SYMBOL (_uverbs_get_const_signed); + +int _uverbs_get_const_unsigned (u64 *to, +const struct uverbs_attr_bundle *attrs_bundle, +size_t idx,u64 upper_bound,u64 *def_val) +{ +const struct uverbs_attr *attr; + + attr =uverbs_attr_get (attrs_bundle, idx); +if (IS_ERR (attr)) { +if ((PTR_ERR (attr) != -ENOENT ) || !def_val) +return PTR_ERR (attr); + + *to = *def_val; + }else { + *to = attr->ptr_attr.data; + } + +if (*to > upper_bound) +return -EINVAL ; + +return 0 ; +} +EXPORT_SYMBOL (_uverbs_get_const_unsigned);int uverbs_copy_to_struct_or_zero (const struct uverbs_attr_bundle *bundle,size_t idx,const void *from,size_t size)
drivers/infiniband/hw/mlx5/main.c +1 -0
@@ -42,6 +42,7 @@#include "counters.h" #include <linux/mlx5/accel.h> #include <rdma/uverbs_std_types.h> +#include <rdma/uverbs_ioctl.h> #include <rdma/mlx5_user_ioctl_verbs.h> #include <rdma/mlx5_user_ioctl_cmds.h> #include <rdma/ib_umem_odp.h>
include/rdma/uverbs_ioctl.h +67 -13
103 lines not on this page · raw message
Signature · 1 line
2.29.2
PATCH 3/3 RDMA/mlx5: Allow larger pages in DevX umem
In reply to Leon RomanovskyFrom: Jason Gunthorpe <[email protected]>
The umem DMA list calculation was locked at 4k pages due to confusion around how this API works and is used when larger pages are present.
The conclusion is:
- umem's cannot extend past what is mapped into the process, so creating a lage page size and referring to a sub-range is not allowed
- umem's must always have a page offset of zero, except for sub PAGE_SIZE umems
- The feature of umem_offset to create multiple objects inside a umem is buggy and isn't used anyplace. Thus we can assume all users of the current API have umem_offset == 0 as well
Provide a new page size calculator that limits the DMA list to the VA range and enforces umem_offset == 0.
Allow user space to specify the page sizes which it can accept, this bitmap must be derived from the intended use of the umem, based on per-usage HW limitations.
--- drivers/infiniband/hw/mlx5/devx.c | 64 ++++++++++++++++++++---- include/uapi/rdma/mlx5_user_ioctl_cmds.h | 1 + 2 files changed, 55 insertions(+), 10 deletions(-)
drivers/infiniband/hw/mlx5/devx.c +54 -10
@@ -2185,27 +2185,69 @@ static int devx_umem_get(struct mlx5_ib_dev *dev, struct ib_ucontext *ucontext,return 0 ; } +static unsigned int devx_umem_find_best_pgsize (struct ib_umem *umem, +unsigned long pgsz_bitmap) +{ +unsigned long page_size; + +/* Don't bother checking larger page sizes as offset must be zero and +* total DEVX umem length must be equal to total umem length. +*/ + pgsz_bitmap &=GENMASK_ULL (max_t (u64,order_base_2 (umem->length), +PAGE_SHIFT ), +MLX5_ADAPTER_PAGE_SHIFT ); +if (!pgsz_bitmap) +return 0 ; + + page_size =ib_umem_find_best_pgoff (umem, pgsz_bitmap,U64_MAX ); +if (!page_size) +return 0 ; + +/* If the page_size is less than the CPU page size then we can use the +* offset and create a umem which is a subset of the page list. +* For larger page sizes we can't be sure the DMA list reflects the +* VA so we must ensure that the umem extent is exactly equal to the +* page list. Reduce the page size until one of these cases is true. +*/ +while ((ib_umem_dma_offset (umem, page_size) !=0 || + (umem->length % page_size) !=0 ) && + page_size >PAGE_SIZE ) + page_size /=2 ; + +return page_size; +} +static int devx_umem_reg_cmd_alloc (struct mlx5_ib_dev *dev,struct uverbs_attr_bundle *attrs,struct devx_umem *obj,struct devx_umem_reg_cmd *cmd) { +unsigned long pgsz_bitmap;unsigned int page_size;__be64 *mtt;void *umem; +int ret;/* -* We don't know what the user intends to use this umem for, but the HW -* restrictions must be met. MR, doorbell records, QP, WQ and CQ all -* have different requirements. Since we have no idea how to sort this -* out, only support PAGE_SIZE with the expectation that userspace will -* provide the necessary alignments inside the known PAGE_SIZE and that -* FW will check everything. +* If the user does not pass in pgsz_bitmap then the user promises not +* to use umem_offset!=0 in any commands that allocate on top of the +* umem. +* +* If the user wants to use a umem_offset then it must pass in +* pgsz_bitmap which guides the maximum page size and thus maximum +* object alignment inside the umem. See the PRM. +* +* Users are not allowed to use IOVA here, mkeys are not supported on +* umem. */ - page_size =ib_umem_find_best_pgoff ( - obj->umem,PAGE_SIZE , -__mlx5_page_offset_to_bitmask (__mlx5_bit_sz (umem, page_offset), -0 )); + ret =uverbs_get_const_default (&pgsz_bitmap, attrs, +MLX5_IB_ATTR_DEVX_UMEM_REG_PGSZ_BITMAP , +GENMASK_ULL (63 , +min (PAGE_SHIFT ,MLX5_ADAPTER_PAGE_SHIFT ))); +if (ret) +return ret; + + page_size =devx_umem_find_best_pgsize (obj->umem, pgsz_bitmap);if (!page_size)return -EINVAL ;
@@ -2791,6 +2833,8 @@ DECLARE_UVERBS_NAMED_METHOD( UA_MANDATORY),UVERBS_ATTR_FLAGS_IN (MLX5_IB_ATTR_DEVX_UMEM_REG_ACCESS ,enum ib_access_flags ), +UVERBS_ATTR_CONST_IN (MLX5_IB_ATTR_DEVX_UMEM_REG_PGSZ_BITMAP , +u64 ),UVERBS_ATTR_PTR_OUT (MLX5_IB_ATTR_DEVX_UMEM_REG_OUT_ID ,UVERBS_ATTR_TYPE (u32 ), UA_MANDATORY));
include/uapi/rdma/mlx5_user_ioctl_cmds.h +1 -0
@@ -154,6 +154,7 @@ enum mlx5_ib_devx_umem_reg_attrs {MLX5_IB_ATTR_DEVX_UMEM_REG_LEN ,MLX5_IB_ATTR_DEVX_UMEM_REG_ACCESS ,MLX5_IB_ATTR_DEVX_UMEM_REG_OUT_ID , + MLX5_IB_ATTR_DEVX_UMEM_REG_PGSZ_BITMAP, }; enum mlx5_ib_devx_umem_dereg_attrs {
--
2.29.2
Quoted 7 lines · Leon Romanovsky
From: Leon Romanovsky <[email protected]>
Hi,
This series removes existing limitation from DevX UMEM and allows user space to specify the page sizes which it can accept.
Quoted 8 lines
Thanks
Jason Gunthorpe (1):
RDMA/mlx5: Allow larger pages in DevX umemYishai Hadas (2):
IB/core: DropWARN_ON()fromib_umem_find_best_pgsz()IB/core: Split uverbs_get_const/default to consider target type
Applied to for-next, thanks
Jason