RDMA/umem: Set iova in ODP flow
PATCH 1/1 RDMA/umem: Set iova in ODP flow
From: Michael Guralnik <[email protected]>
Fixing the ODP registration flow to set the iova correctly. The calculation in ib_umem_num_dma_blocks() function assumes the iova of the umem is set correctly.
When iova is not set, the calculation in ib_umem_num_dma_blocks() is equivalent to length/page_size, which is true only when memory is aligned. For unaligned memory, iova must be set for the ALIGN() in the ib_umem_num_dma_blocks() to take effect and return a correct value.
mlx5_ib uses ib_umem_num_dma_blocks() to decide the mkey size to use for the MR. Without this fix, when registering unaligned ODP MR, a wrong size mkey might be chosen and this might cause the UMR to fail.
UMR would fail over insufficient size to update the mkey translation:
infiniband mlx5_0: dump_cqe:273:(pid 0): dump error cqe 00000000: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00000010: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00000020: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00000030: 00 00 00 00 0f 00 78 06 25 00 00 58 00 da ac d2 infiniband mlx5_0: mlx5_ib_post_send_wait:806:(pid 20311): reg umr failed (6)
infiniband mlx5_0: pagefault_real_mr:661:(pid 20311): Failed to update mkey page tables
--- drivers/infiniband/core/umem.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)
drivers/infiniband/core/umem.c +2 -1
@@ -85,6 +85,8 @@ unsigned long ib_umem_find_best_pgsz(struct ib_umem *umem,dma_addr_t mask;int i; + umem->iova = va = virt; +if (umem->is_odp) {unsigned int page_size =BIT (to_ib_umem_odp (umem)->page_shift);
@@ -100,7 +102,6 @@ unsigned long ib_umem_find_best_pgsz(struct ib_umem *umem,*/ pgsz_bitmap &=GENMASK (BITS_PER_LONG -1 ,PAGE_SHIFT ); - umem->iova = va = virt;/* The best result is the smallest page size that results in the minimum * number of required pages. Compute the largest page size that could * work based on VA address bits that don't change.
Signature · 1 line
2.41.0
Leon Romanovsky
AppliedQuoted 10 lines · Leon Romanovsky
Fixing the ODP registration flow to set the iova correctly. The calculation in
ib_umem_num_dma_blocks()function assumes the iova of the umem is set correctly.When iova is not set, the calculation in
ib_umem_num_dma_blocks()is equivalent to length/page_size, which is true only when memory is aligned. For unaligned memory, iova must be set for theALIGN()in theib_umem_num_dma_blocks()to take effect and return a correct value.[...]
Applied, thanks!
[1/1] RDMA/umem: Set iova in ODP flow
https://git.kernel.org/rdma/rdma/c/186b169cf1e4be
Best regards,
Signature · 1 line
Leon Romanovsky <[email protected]>