Skip to contentSkip to conversation

Support larger than 4K pages in DevX UMEMs

PATCH Applied Leon Romanovsky 2 participants 5 messages Mar 4, 2021 – Mar 12, 2021

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

cover Support larger than 4K pages in DevX UMEMs

From: Leon Romanovsky <[email protected]>

Hi,

This series removes existing limitation from DevX UMEM and allows user space to specify the page sizes which it can accept.

Thanks

Jason Gunthorpe (1):
RDMA/mlx5: Allow larger pages in DevX umem

Yishai Hadas (2):
IB/core: Drop WARN_ON() from ib_umem_find_best_pgsz() IB/core: Split uverbs_get_const/default to consider target type

 drivers/infiniband/core/umem.c           |  4 --
 drivers/infiniband/core/uverbs_ioctl.c   | 32 ++++++++--
 drivers/infiniband/hw/mlx5/devx.c        | 64 ++++++++++++++++---
 drivers/infiniband/hw/mlx5/main.c        |  1 +
 include/rdma/uverbs_ioctl.h              | 80 ++++++++++++++++++++----
 include/uapi/rdma/mlx5_user_ioctl_cmds.h |  1 +
 6 files changed, 151 insertions(+), 31 deletions(-)

--
2.29.2

#1 lore raw

PATCH 1/3 IB/core: Drop WARN_ON() from ib_umem_find_best_pgsz()

From: Yishai Hadas <[email protected]>

The WARN_ON() issued as part of ib_umem_find_best_pgsz() blocked cases when only page sizes larger than PAGE_SIZE were set, drop it to enable those cases.

In addition, there is no need to have a specific check for zero pgsz_bitmap, the function will do its job and return 0 at the end if nothing match will be found.

---
 drivers/infiniband/core/umem.c | 4 ----
 1 file changed, 4 deletions(-)
drivers/infiniband/core/umem.c +0 -4
index 2dde99a9ba07..2cd64649005b 100644
@@ -100,10 +100,6 @@ unsigned long ib_umem_find_best_pgsz(struct ib_umem *umem,
 	 */
 	pgsz_bitmap &= GENMASK(BITS_PER_LONG - 1, PAGE_SHIFT);
 
-	/* At minimum, drivers must support PAGE_SIZE or smaller */
-	if (WARN_ON(!(pgsz_bitmap & GENMASK(PAGE_SHIFT, 0))))
-		return 0;
-
 	umem->iova = va = virt;
 	/* The best result is the smallest page size that results in the minimum
 	 * number of required pages. Compute the largest page size that could
Signature · 1 line
2.29.2
#2 lore raw

PATCH 2/3 IB/core: Split uverbs_get_const/default to consider target type

In reply to Leon Romanovsky

From: Yishai Hadas <[email protected]>

Change uverbs_get_const/uverbs_get_const_default to work properly with both signed/unsigned parameters.

Current APIs mix s64 and u64 which leads to incorrect check when u64 value was supplied and its upper bit was set. In that case uverbs_get_const() / uverbs_get_const_default() lower bound check may fail unexpectedly, target is unsigned (lower bound is 0) but value became negative as of the s64 usage.

Split to have two different APIs, no change to callers as the required API will be called internally according to the target type.

---
 drivers/infiniband/core/uverbs_ioctl.c | 32 +++++++++--
 drivers/infiniband/hw/mlx5/main.c      |  1 +
 include/rdma/uverbs_ioctl.h            | 80 +++++++++++++++++++++-----
 3 files changed, 96 insertions(+), 17 deletions(-)
drivers/infiniband/core/uverbs_ioctl.c +28 -4
index ff047eb024ab..990f0724acc6 100644
@@ -752,9 +752,10 @@ int uverbs_output_written(const struct uverbs_attr_bundle *bundle, size_t idx)
 	return uverbs_set_output(bundle, attr);
 }
 
-int _uverbs_get_const(s64 *to, const struct uverbs_attr_bundle *attrs_bundle,
-		      size_t idx, s64 lower_bound, u64 upper_bound,
-		      s64  *def_val)
+int _uverbs_get_const_signed(s64 *to,
+			     const struct uverbs_attr_bundle *attrs_bundle,
+			     size_t idx, s64 lower_bound, u64 upper_bound,
+			     s64  *def_val)
 {
 	const struct uverbs_attr *attr;
 
@@ -773,7 +774,30 @@ int _uverbs_get_const(s64 *to, const struct uverbs_attr_bundle *attrs_bundle,
 
 	return 0;
 }
-EXPORT_SYMBOL(_uverbs_get_const);
+EXPORT_SYMBOL(_uverbs_get_const_signed);
+
+int _uverbs_get_const_unsigned(u64 *to,
+			       const struct uverbs_attr_bundle *attrs_bundle,
+			       size_t idx, u64 upper_bound, u64 *def_val)
+{
+	const struct uverbs_attr *attr;
+
+	attr = uverbs_attr_get(attrs_bundle, idx);
+	if (IS_ERR(attr)) {
+		if ((PTR_ERR(attr) != -ENOENT) || !def_val)
+			return PTR_ERR(attr);
+
+		*to = *def_val;
+	} else {
+		*to = attr->ptr_attr.data;
+	}
+
+	if (*to > upper_bound)
+		return -EINVAL;
+
+	return 0;
+}
+EXPORT_SYMBOL(_uverbs_get_const_unsigned);
 
 int uverbs_copy_to_struct_or_zero(const struct uverbs_attr_bundle *bundle,
 				  size_t idx, const void *from, size_t size)
drivers/infiniband/hw/mlx5/main.c +1 -0
index aa3e90594a88..5226664f1bda 100644
@@ -42,6 +42,7 @@
 #include "counters.h"
 #include <linux/mlx5/accel.h>
 #include <rdma/uverbs_std_types.h>
+#include <rdma/uverbs_ioctl.h>
 #include <rdma/mlx5_user_ioctl_verbs.h>
 #include <rdma/mlx5_user_ioctl_cmds.h>
 #include <rdma/ib_umem_odp.h>
include/rdma/uverbs_ioctl.h +67 -13

103 lines not on this page · raw message

Signature · 1 line
2.29.2
#3 lore raw

PATCH 3/3 RDMA/mlx5: Allow larger pages in DevX umem

In reply to Leon Romanovsky

From: Jason Gunthorpe <[email protected]>

The umem DMA list calculation was locked at 4k pages due to confusion around how this API works and is used when larger pages are present.

The conclusion is:

- umem's cannot extend past what is mapped into the process, so creating a lage page size and referring to a sub-range is not allowed

- umem's must always have a page offset of zero, except for sub PAGE_SIZE umems

- The feature of umem_offset to create multiple objects inside a umem is buggy and isn't used anyplace. Thus we can assume all users of the current API have umem_offset == 0 as well

Provide a new page size calculator that limits the DMA list to the VA range and enforces umem_offset == 0.

Allow user space to specify the page sizes which it can accept, this bitmap must be derived from the intended use of the umem, based on per-usage HW limitations.

---
 drivers/infiniband/hw/mlx5/devx.c        | 64 ++++++++++++++++++++----
 include/uapi/rdma/mlx5_user_ioctl_cmds.h |  1 +
 2 files changed, 55 insertions(+), 10 deletions(-)
drivers/infiniband/hw/mlx5/devx.c +54 -10
index de3c2fc6f361..17ab369efc5e 100644
@@ -2185,27 +2185,69 @@ static int devx_umem_get(struct mlx5_ib_dev *dev, struct ib_ucontext *ucontext,
 	return 0;
 }
 
+static unsigned int devx_umem_find_best_pgsize(struct ib_umem *umem,
+					       unsigned long pgsz_bitmap)
+{
+	unsigned long page_size;
+
+	/* Don't bother checking larger page sizes as offset must be zero and
+	 * total DEVX umem length must be equal to total umem length.
+	 */
+	pgsz_bitmap &= GENMASK_ULL(max_t(u64, order_base_2(umem->length),
+					 PAGE_SHIFT),
+				   MLX5_ADAPTER_PAGE_SHIFT);
+	if (!pgsz_bitmap)
+		return 0;
+
+	page_size = ib_umem_find_best_pgoff(umem, pgsz_bitmap, U64_MAX);
+	if (!page_size)
+		return 0;
+
+	/* If the page_size is less than the CPU page size then we can use the
+	 * offset and create a umem which is a subset of the page list.
+	 * For larger page sizes we can't be sure the DMA  list reflects the
+	 * VA so we must ensure that the umem extent is exactly equal to the
+	 * page list. Reduce the page size until one of these cases is true.
+	 */
+	while ((ib_umem_dma_offset(umem, page_size) != 0 ||
+		(umem->length % page_size) != 0) &&
+		page_size > PAGE_SIZE)
+		page_size /= 2;
+
+	return page_size;
+}
+
 static int devx_umem_reg_cmd_alloc(struct mlx5_ib_dev *dev,
 				   struct uverbs_attr_bundle *attrs,
 				   struct devx_umem *obj,
 				   struct devx_umem_reg_cmd *cmd)
 {
+	unsigned long pgsz_bitmap;
 	unsigned int page_size;
 	__be64 *mtt;
 	void *umem;
+	int ret;
 
 	/*
-	 * We don't know what the user intends to use this umem for, but the HW
-	 * restrictions must be met. MR, doorbell records, QP, WQ and CQ all
-	 * have different requirements. Since we have no idea how to sort this
-	 * out, only support PAGE_SIZE with the expectation that userspace will
-	 * provide the necessary alignments inside the known PAGE_SIZE and that
-	 * FW will check everything.
+	 * If the user does not pass in pgsz_bitmap then the user promises not
+	 * to use umem_offset!=0 in any commands that allocate on top of the
+	 * umem.
+	 *
+	 * If the user wants to use a umem_offset then it must pass in
+	 * pgsz_bitmap which guides the maximum page size and thus maximum
+	 * object alignment inside the umem. See the PRM.
+	 *
+	 * Users are not allowed to use IOVA here, mkeys are not supported on
+	 * umem.
 	 */
-	page_size = ib_umem_find_best_pgoff(
-		obj->umem, PAGE_SIZE,
-		__mlx5_page_offset_to_bitmask(__mlx5_bit_sz(umem, page_offset),
-					      0));
+	ret = uverbs_get_const_default(&pgsz_bitmap, attrs,
+			MLX5_IB_ATTR_DEVX_UMEM_REG_PGSZ_BITMAP,
+			GENMASK_ULL(63,
+				    min(PAGE_SHIFT, MLX5_ADAPTER_PAGE_SHIFT)));
+	if (ret)
+		return ret;
+
+	page_size = devx_umem_find_best_pgsize(obj->umem, pgsz_bitmap);
 	if (!page_size)
 		return -EINVAL;
 
@@ -2791,6 +2833,8 @@ DECLARE_UVERBS_NAMED_METHOD(
 			   UA_MANDATORY),
 	UVERBS_ATTR_FLAGS_IN(MLX5_IB_ATTR_DEVX_UMEM_REG_ACCESS,
 			     enum ib_access_flags),
+	UVERBS_ATTR_CONST_IN(MLX5_IB_ATTR_DEVX_UMEM_REG_PGSZ_BITMAP,
+			     u64),
 	UVERBS_ATTR_PTR_OUT(MLX5_IB_ATTR_DEVX_UMEM_REG_OUT_ID,
 			    UVERBS_ATTR_TYPE(u32),
 			    UA_MANDATORY));
include/uapi/rdma/mlx5_user_ioctl_cmds.h +1 -0
index 3fd9b380a091..3f0bc7597ba7 100644
@@ -154,6 +154,7 @@ enum mlx5_ib_devx_umem_reg_attrs {
 	MLX5_IB_ATTR_DEVX_UMEM_REG_LEN,
 	MLX5_IB_ATTR_DEVX_UMEM_REG_ACCESS,
 	MLX5_IB_ATTR_DEVX_UMEM_REG_OUT_ID,
+	MLX5_IB_ATTR_DEVX_UMEM_REG_PGSZ_BITMAP,
 };
 
 enum mlx5_ib_devx_umem_dereg_attrs {

--
2.29.2

maintainer Applied
#4 lore raw
In reply to Leon Romanovsky
Quoted 7 lines · Leon Romanovsky

From: Leon Romanovsky <[email protected]>

Hi,

This series removes existing limitation from DevX UMEM and allows user space to specify the page sizes which it can accept.

Quoted 8 lines

Thanks

Jason Gunthorpe (1):
RDMA/mlx5: Allow larger pages in DevX umem

Yishai Hadas (2):
IB/core: Drop WARN_ON() from ib_umem_find_best_pgsz() IB/core: Split uverbs_get_const/default to consider target type

Applied to for-next, thanks

Jason