RDMA/mlx5: Set mkeys for dmabuf at PAGE_SIZE
Subsystems (2)
PATCH 1/1 RDMA/mlx5: Set mkeys for dmabuf at PAGE_SIZE
From: Chiara Meiohas <[email protected]>
Set the mkey for dmabuf at PAGE_SIZE to support any SGL after a move operation.
ib_umem_find_best_pgsz returns 0 on error, so it is incorrect to check the returned page_size against PAGE_SIZE
--- drivers/infiniband/hw/mlx5/mlx5_ib.h | 13 +++++++++++++ drivers/infiniband/hw/mlx5/odp.c | 6 ++---- 2 files changed, 15 insertions(+), 4 deletions(-)
drivers/infiniband/hw/mlx5/mlx5_ib.h +13 -0
@@ -115,6 +115,19 @@ unsigned long __mlx5_umem_find_best_quantized_pgoff( __mlx5_bit_sz(typ, page_offset_fld), 0, scale, \ page_offset_quantized) +static inline unsigned long +mlx5_umem_dmabuf_find_best_pgsz (struct ib_umem_dmabuf *umem_dmabuf) +{ +/* +* mkeys used for dmabuf are fixed at PAGE_SIZE because we must be able +* to hold any sgl after a move operation. Ideally the mkc page size +* could be changed at runtime to be optimal, but right now the driver +* cannot do that. +*/ +return ib_umem_find_best_pgsz (&umem_dmabuf->umem,PAGE_SIZE , + umem_dmabuf->umem.iova); +} +enum {MLX5_IB_MMAP_OFFSET_START =9 ,MLX5_IB_MMAP_OFFSET_END =255 ,
drivers/infiniband/hw/mlx5/odp.c +2 -4
@@ -705,10 +705,8 @@ static int pagefault_dmabuf_mr(struct mlx5_ib_mr *mr, size_t bcnt,return err; } - page_size =mlx5_umem_find_best_pgsz (&umem_dmabuf->umem, mkc, - log_page_size,0 , - umem_dmabuf->umem.iova); -if (unlikely (page_size <PAGE_SIZE )) { + page_size =mlx5_umem_dmabuf_find_best_pgsz (umem_dmabuf); +if (!page_size) {ib_umem_dmabuf_unmap_pages (umem_dmabuf); err = -EINVAL ; }else {
Signature · 1 line
2.45.2
Jason Gunthorpe
maintainerQuoted 7 lines · Leon Romanovsky
From: Chiara Meiohas <[email protected]>
Set the mkey for dmabuf at
PAGE_SIZEto support any SGL after a move operation.
ib_umem_find_best_pgszreturns 0 on error, so it is incorrect to check the returnedpage_sizeagainstPAGE_SIZE
This commit message is not clear enough for something that need to be backported:
RDMA/mlx5: Support non-page size aligned DMABUF mkeys
The mkey page size for DMABUF is fixed at PAGE_SIZE because we have to support a move operation that could change a large-sized page list into a small page-list and the mkey must be able to represent it.
The test for this is not quite correct, instead of checking the output of mlx5_umem_find_best_pgsz() the call to ib_umem_find_best_pgsz should specify the exact HW/SW restriction - only PAGE_SIZE is accepted.
Then the normal logic for dealing with leading/trailing sub page alignment works correctly and sub page size DMBUF mappings can be supported.
This is particularly painful on 64K kernels.
Jason
Quoted 11 lines · Jason Gunthorpe
Quoted 7 lines · Leon Romanovsky
From: Chiara Meiohas <[email protected]>
Set the mkey for dmabuf at
PAGE_SIZEto support any SGL after a move operation.
ib_umem_find_best_pgszreturns 0 on error, so it is incorrect to check the returnedpage_sizeagainstPAGE_SIZEThis commit message is not clear enough for something that need to be backported:
This patch is going to be backported without any relation to the commit message as it has Fixes line.
Thanks
Quoted 12 lines
RDMA/mlx5: Support non-page size aligned DMABUF mkeys
The mkey page size for DMABUF is fixed at
PAGE_SIZEbecause we have to support a move operation that could change a large-sized page list into a small page-list and the mkey must be able to represent it.The test for this is not quite correct, instead of checking the output of
mlx5_umem_find_best_pgsz()the call toib_umem_find_best_pgszshould specify the exact HW/SW restriction - onlyPAGE_SIZEis accepted.
Quoted 5 lines
Then the normal logic for dealing with leading/trailing sub page alignment works correctly and sub page size DMBUF mappings can be supported.
This is particularly painful on 64K kernels.
Unfortunately, the patch was already merged, so I can't change the commit message in for-next branch.
Thanks
Quoted 2 lines
Jason
Jason Gunthorpe
maintainerQuoted 11 lines · Leon Romanovsky
Quoted 8 lines
Quoted 5 lines
Set the mkey for dmabuf at
PAGE_SIZEto support any SGL after a move operation.
ib_umem_find_best_pgszreturns 0 on error, so it is incorrect to check the returnedpage_sizeagainstPAGE_SIZEThis commit message is not clear enough for something that need to be backported:
This patch is going to be backported without any relation to the commit message as it has Fixes line.
People doing backports complain with some regularity about poor commit messages, especailly now that so many patches get a CVE. We need to do better.
Quoted 11 lines
Quoted 11 lines
RDMA/mlx5: Support non-page size aligned DMABUF mkeys
The mkey page size for DMABUF is fixed at
PAGE_SIZEbecause we have to support a move operation that could change a large-sized page list into a small page-list and the mkey must be able to represent it.The test for this is not quite correct, instead of checking the output of
mlx5_umem_find_best_pgsz()the call toib_umem_find_best_pgszshould specify the exact HW/SW restriction - onlyPAGE_SIZEis accepted.
Quoted 8 lines
Quoted 5 lines
Then the normal logic for dealing with leading/trailing sub page alignment works correctly and sub page size DMBUF mappings can be supported.
This is particularly painful on 64K kernels.
Unfortunately, the patch was already merged, so I can't change the commit message in for-next branch.
How is it already merged? There was no message from your script - are you loosing emails??
Jason
Quoted 10 lines · Jason Gunthorpe
Quoted 9 lines · Leon Romanovsky
Quoted 8 lines
Quoted 5 lines
Set the mkey for dmabuf at
PAGE_SIZEto support any SGL after a move operation.
ib_umem_find_best_pgszreturns 0 on error, so it is incorrect to check the returnedpage_sizeagainstPAGE_SIZEThis commit message is not clear enough for something that need to be backported:
Quoted 6 lines
Quoted 2 lines
This patch is going to be backported without any relation to the commit message as it has Fixes line.
People doing backports complain with some regularity about poor commit messages, especailly now that so many patches get a CVE. We need to do better.
It is always true.
Quoted 16 lines
Quoted 16 lines
Quoted 16 lines
RDMA/mlx5: Support non-page size aligned DMABUF mkeys
The mkey page size for DMABUF is fixed at
PAGE_SIZEbecause we have to support a move operation that could change a large-sized page list into a small page-list and the mkey must be able to represent it.The test for this is not quite correct, instead of checking the output of
mlx5_umem_find_best_pgsz()the call toib_umem_find_best_pgszshould specify the exact HW/SW restriction - onlyPAGE_SIZEis accepted.Then the normal logic for dealing with leading/trailing sub page alignment works correctly and sub page size DMBUF mappings can be supported.
Quoted 7 lines
Quoted 4 lines
Quoted 1 line
This is particularly painful on 64K kernels.
Unfortunately, the patch was already merged, so I can't change the commit message in for-next branch.
How is it already merged? There was no message from your script - are you loosing emails??
In this specific case, no.
I switched too early from branch and my thank-you message (b4 ty ..) simply wasn't sent.
Thanks
Quoted 2 lines
Jason