IB/rdmavt: Fix frwr memory registration
Subsystems (3)
PATCH 1/1 IB/rdmavt: Fix frwr memory registration
From: Josh Collier <[email protected]>
Current implementation was not properly handling frwr memory registrations. This was uncovered by:
commit 27f26cec761das
xprtrdma: Plant XID in on-the-wire RDMA offset (FRWR)
in which xprtrdma, which is used for NFS over RDMA, started failing as it was the first ULP to modify the ib_mr iova resulting in the NFS server getting REMOTE ACCESS ERROR when attempting to perform RDMA Writes to the client.
The fix is to properly capture the true iova, offset, and length in the call to ib_map_mr_sg, and then update the iova when processing the IB_WR_REG_MEM on the send queue.
--- drivers/infiniband/sw/rdmavt/mr.c | 17 ++++++++++------- 1 files changed, 10 insertions(+), 7 deletions(-)
drivers/infiniband/sw/rdmavt/mr.c +10 -7
@@ -608,11 +608,6 @@ static int rvt_set_page(struct ib_mr *ibmr, u64 addr)if (unlikely (mapped_segs == mr->mr.max_segs))return -ENOMEM ; -if (mr->mr.length ==0 ) { - mr->mr.user_base = addr; - mr->mr.iova = addr; - } - m = mapped_segs /RVT_SEGSZ ; n = mapped_segs %RVT_SEGSZ ; mr->mr.map[m]->segs[n].vaddr = (void *)addr;
@@ -630,17 +625,24 @@ static int rvt_set_page(struct ib_mr *ibmr, u64 addr)* @sg_nents: number of entries in sg * @sg_offset: offset in bytes into sg * +* Overwrite rvt_mr length with mr length calculated by ib_sg_to_pages. +* * Return: number of sg elements mapped to the memory region */ int rvt_map_mr_sg (struct ib_mr *ibmr,struct scatterlist *sg,int sg_nents,unsigned int *sg_offset) {struct rvt_mr *mr =to_imr (ibmr); +int ret; mr->mr.length =0 ; mr->mr.page_shift =PAGE_SHIFT ; -return ib_sg_to_pages (ibmr, sg, sg_nents, sg_offset, - rvt_set_page); + ret =ib_sg_to_pages (ibmr, sg, sg_nents, sg_offset, rvt_set_page); + mr->mr.user_base = ibmr->iova; + mr->mr.iova = ibmr->iova; + mr->mr.offset = ibmr->iova - (u64 )mr->mr.map[0 ]->segs[0 ].vaddr; + mr->mr.length = (size_t )ibmr->length; +return ret; }/**
@@ -671,6 +673,7 @@ int rvt_fast_reg_mr(struct rvt_qp *qp, struct ib_mr *ibmr, u32 key, ibmr->rkey = key; mr->mr.lkey = key; mr->mr.access_flags = access; + mr->mr.iova = ibmr->iova;atomic_set (&mr->mr.lkey_invalid,0 );return 0 ;
Quoted 47 lines · Dennis Dalessandro
From: Josh Collier <[email protected]>
Current implementation was not properly handling frwr memory registrations. This was uncovered by:
commit 27f26cec761das
xprtrdma: Plant XID in on-the-wire RDMA offset (FRWR)
in which xprtrdma, which is used for NFS over RDMA, started failing as it was the first ULP to modify theib_mriova resulting in the NFS server getting REMOTE ACCESS ERROR when attempting to perform RDMA Writes to the client.The fix is to properly capture the true iova, offset, and length in the call to
ib_map_mr_sg, and then update the iova when processing theIB_WR_REG_MEMon the send queue.Fixes a41081aa5936 ("IB/rdmavt: Add support for ib_map_mr_sg")Reviewed-by Mike Marciniszyn <[email protected]>Reviewed-by Dennis Dalessandro <[email protected]>Reviewed-by Michael J. Ruhl <[email protected]>Signed-off-by Josh Collier <[email protected]>Signed-off-by Dennis Dalessandro <[email protected]>--- drivers/infiniband/sw/rdmavt/mr.c | 17 ++++++++++------- 1 files changed, 10 insertions(+), 7 deletions(-)drivers/infiniband/sw/rdmavt/mr.c +1
-5@@ -608,11 +608,6 @@ static int rvt_set_page(struct ib_mr *ibmr, u64 addr)if (unlikely (mapped_segs == mr->mr.max_segs))return -ENOMEM ; -if (mr->mr.length ==0 ) { - mr->mr.user_base = addr; - mr->mr.iova = addr; - } - m = mapped_segs /RVT_SEGSZ ; n = mapped_segs %RVT_SEGSZ ; mr->mr.map[m]->segs[n].vaddr = (void *)addr;@@ -630,17 +625,24 @@ static int rvt_set_page(struct ib_mr *ibmr, u64 addr)* @sg_nents: number of entries in sg * @sg_offset: offset in bytes into sg * +* Overwrite rvt_mr length with mr length calculated by ib_sg_to_pages.
drivers/infiniband/sw/rdmavt/mr.c static int rvt_set_page(struct ib_mr *ibmr, u64 addr) · line 638
Quoted 10 lines
drivers/infiniband/sw/rdmavt/mr.c +2
-0+* * Return: number of sg elements mapped to the memory region */ int rvt_map_mr_sg (struct ib_mr *ibmr,struct scatterlist *sg,int sg_nents,unsigned int *sg_offset) {struct rvt_mr *mr =to_imr (ibmr); +int ret; mr->mr.length =0 ;
The above mr.length assignment is no longer needed, since it is unconditionally overwritten several lines below.
Quoted 13 lines
drivers/infiniband/sw/rdmavt/mr.c +6
-2mr->mr.page_shift =PAGE_SHIFT ; -return ib_sg_to_pages (ibmr, sg, sg_nents, sg_offset, - rvt_set_page); + ret =ib_sg_to_pages (ibmr, sg, sg_nents, sg_offset, rvt_set_page); + mr->mr.user_base = ibmr->iova; + mr->mr.iova = ibmr->iova; + mr->mr.offset = ibmr->iova - (u64 )mr->mr.map[0 ]->segs[0 ].vaddr; + mr->mr.length = (size_t )ibmr->length; +return ret; }/**
drivers/infiniband/sw/rdmavt/mr.c int rvt_fast_reg_mr(struct rvt_qp *qp, struct ib_mr *ibmr, u32 key, · line 676
Quoted 4 lines
drivers/infiniband/sw/rdmavt/mr.c +1
-0@@ -671,6 +673,7 @@ int rvt_fast_reg_mr(struct rvt_qp *qp, struct ib_mr *ibmr, u32 key, ibmr->rkey = key; mr->mr.lkey = key; mr->mr.access_flags = access; + mr->mr.iova = ibmr->iova;
Isn't this a redundant assignment? If it isn't, then why is the mr->mr.offset not being recalculated?
Quoted 6 lines
drivers/infiniband/sw/rdmavt/mr.c +0
-0atomic_set (&mr->mr.lkey_invalid,0 );return 0 ;
Quoted 72 lines
-----Original Message-----
From: Tom Talpey [mailto:[email protected]]
Sent: Monday, April 15, 2019 3:15 PM
To: Dalessandro, Dennis <[email protected]>; [email protected]; [email protected]
Cc: [email protected]; Ruhl, Michael J <[email protected]>; Marciniszyn, Mike <[email protected]>; Collier, Josh D <[email protected]>; [email protected]
Subject: Re: [PATCH for-rc] IB/rdmavt: Fix frwr memory registrationQuoted 9 lines · Dennis Dalessandro
From: Josh Collier <[email protected]>
Current implementation was not properly handling frwr memory registrations. This was uncovered by:
commit 27f26cec761das
xprtrdma: Plant XID in on-the-wire RDMA offset (FRWR) in which
xprtrdma, which is used for NFS over RDMA, started failing as it was the first ULP to modify theib_mriova resulting in the NFS server getting REMOTE ACCESS ERROR when attempting to perform RDMA Writesto
drivers/infiniband/sw/rdmavt/mr.c static int rvt_set_page(struct ib_mr *ibmr, u64
Quoted 23 lines
the client.
The fix is to properly capture the true iova, offset, and length in the call to
ib_map_mr_sg, and then update the iova when processing theIB_WR_REG_MEMon the send queue.Fixes a41081aa5936 ("IB/rdmavt: Add support for ib_map_mr_sg")Reviewed-by Mike Marciniszyn <[email protected]>Reviewed-by Dennis Dalessandro <[email protected]>Reviewed-by Michael J. Ruhl <[email protected]>Signed-off-by Josh Collier <[email protected]>Signed-off-by Dennis Dalessandro <[email protected]>--- drivers/infiniband/sw/rdmavt/mr.c | 17 ++++++++++------- 1 files changed, 10 insertions(+), 7 deletions(-)drivers/infiniband/sw/rdmavt/mr.c +0
-0@@ -608,11 +608,6 @@ static int rvt_set_page(struct ib_mr *ibmr, u64addr)
drivers/infiniband/sw/rdmavt/mr.c static int rvt_set_page(struct ib_mr *ibmr, u64 addr)
Quoted 11 lines
drivers/infiniband/sw/rdmavt/mr.c +0
-5if (unlikely (mapped_segs == mr->mr.max_segs))return -ENOMEM ; -if (mr->mr.length ==0 ) { - mr->mr.user_base = addr; - mr->mr.iova = addr; - } - m = mapped_segs /RVT_SEGSZ ; n = mapped_segs %RVT_SEGSZ ; mr->mr.map[m]->segs[n].vaddr = (void *)addr; @@ -630,17 +625,24@@
Quoted 14 lines
static int
rvt_set_page(struct ib_mr *ibmr, u64 addr)
* @sg_nents: number of entries in sg
* @sg_offset: offset in bytes into sg
*
+ * Overwritervt_mrlength with mr length calculated byib_sg_to_pages.
+ *
* Return: number of sg elements mapped to the memory region */
int rvt_map_mr_sg(structib_mr*ibmr, struct scatterlist *sg,
intsg_nents, unsigned int*sg_offset)
{
structrvt_mr*mr =to_imr(ibmr);drivers/infiniband/sw/rdmavt/mr.c +1
-0+int ret;
Quoted 4 lines
Quoted 1 line
mr->mr.length = 0;The above mr.length assignment is no longer needed, since it is unconditionally overwritten several lines below.
The mr.length is still temporarily used by our rvt_set_page function effectively as a page index and needs to continue to be initialized for that purpose.
Quoted 15 lines
drivers/infiniband/sw/rdmavt/mr.c int rvt_fast_reg_mr(struct rvt_qp *qp, struct ib_mr
Quoted 14 lines
mr->mr.page_shift = PAGE_SHIFT;drivers/infiniband/sw/rdmavt/mr.c +6
-2-return ib_sg_to_pages (ibmr, sg, sg_nents, sg_offset, - rvt_set_page); + ret =ib_sg_to_pages (ibmr, sg, sg_nents, sg_offset, rvt_set_page); + mr->mr.user_base = ibmr->iova; + mr->mr.iova = ibmr->iova; + mr->mr.offset = ibmr->iova - (u64 )mr->mr.map[0 ]->segs[0 ].vaddr; + mr->mr.length = (size_t )ibmr->length; +return ret; }/** @@ -671,6 +673,7 @@ int rvt_fast_reg_mr(struct rvt_qp *qp, struct ib_mr*ibmr, u32 key,
Quoted 7 lines
drivers/infiniband/sw/rdmavt/mr.c int rvt_fast_reg_mr(struct rvt_qp *qp, struct ib_mr *ibmr, u32 key, · line 676
Quoted 4 lines
drivers/infiniband/sw/rdmavt/mr.c +1
-0ibmr->rkey = key; mr->mr.lkey = key; mr->mr.access_flags = access; + mr->mr.iova = ibmr->iova;Isn't this a redundant assignment? If it isn't, then why is the
mr->mr.offsetnot being recalculated?
The ib_mr iova can be changed by the ULP to any value, but any value should index to the previously mapped MR's starting page and offset, hence those do not change, and we only update the iova.
Quoted 7 lines
Quoted 7 lines
drivers/infiniband/sw/rdmavt/mr.c +0
-0atomic_set (&mr->mr.lkey_invalid,0 );return 0 ;
Quoted 72 lines · Josh D
Quoted 72 lines
-----Original Message-----
From: Tom Talpey [mailto:[email protected]]
Sent: Monday, April 15, 2019 3:15 PM
To: Dalessandro, Dennis <[email protected]>; [email protected]; [email protected]
Cc: [email protected]; Ruhl, Michael J <[email protected]>; Marciniszyn, Mike <[email protected]>; Collier, Josh D <[email protected]>; [email protected]
Subject: Re: [PATCH for-rc] IB/rdmavt: Fix frwr memory registrationQuoted 9 lines · Dennis Dalessandro
From: Josh Collier <[email protected]>
Current implementation was not properly handling frwr memory registrations. This was uncovered by:
commit 27f26cec761das
xprtrdma: Plant XID in on-the-wire RDMA offset (FRWR) in which
xprtrdma, which is used for NFS over RDMA, started failing as it was the first ULP to modify theib_mriova resulting in the NFS server getting REMOTE ACCESS ERROR when attempting to perform RDMA Writesto
drivers/infiniband/sw/rdmavt/mr.c static int rvt_set_page(struct ib_mr *ibmr, u64
Quoted 23 lines
the client.
The fix is to properly capture the true iova, offset, and length in the call to
ib_map_mr_sg, and then update the iova when processing theIB_WR_REG_MEMon the send queue.Fixes a41081aa5936 ("IB/rdmavt: Add support for ib_map_mr_sg")Reviewed-by Mike Marciniszyn <[email protected]>Reviewed-by Dennis Dalessandro <[email protected]>Reviewed-by Michael J. Ruhl <[email protected]>Signed-off-by Josh Collier <[email protected]>Signed-off-by Dennis Dalessandro <[email protected]>--- drivers/infiniband/sw/rdmavt/mr.c | 17 ++++++++++------- 1 files changed, 10 insertions(+), 7 deletions(-)drivers/infiniband/sw/rdmavt/mr.c +0
-0@@ -608,11 +608,6 @@ static int rvt_set_page(struct ib_mr *ibmr, u64addr)
drivers/infiniband/sw/rdmavt/mr.c static int rvt_set_page(struct ib_mr *ibmr, u64 addr)
Quoted 11 lines
drivers/infiniband/sw/rdmavt/mr.c +0
-5if (unlikely (mapped_segs == mr->mr.max_segs))return -ENOMEM ; -if (mr->mr.length ==0 ) { - mr->mr.user_base = addr; - mr->mr.iova = addr; - } - m = mapped_segs /RVT_SEGSZ ; n = mapped_segs %RVT_SEGSZ ; mr->mr.map[m]->segs[n].vaddr = (void *)addr; @@ -630,17 +625,24@@
Quoted 14 lines
static int
rvt_set_page(struct ib_mr *ibmr, u64 addr)
* @sg_nents: number of entries in sg
* @sg_offset: offset in bytes into sg
*
+ * Overwritervt_mrlength with mr length calculated byib_sg_to_pages.
+ *
* Return: number of sg elements mapped to the memory region */
int rvt_map_mr_sg(structib_mr*ibmr, struct scatterlist *sg,
intsg_nents, unsigned int*sg_offset){ struct rvt_mr *mr = to_imr(ibmr);drivers/infiniband/sw/rdmavt/mr.c +1
-0+int ret;
Quoted 7 lines
Quoted 4 lines
Quoted 1 line
mr->mr.length = 0;The above mr.length assignment is no longer needed, since it is unconditionally overwritten several lines below.
The mr.length is still temporarily used by our
rvt_set_pagefunction effectively as a page index and needs to continue to be initialized for that purpose.
Hmm, ok I guess. It seems quite odd to need to initialize it to zero just to have it set by the callee to the proper value.
Quoted 22 lines
Quoted 14 lines
drivers/infiniband/sw/rdmavt/mr.c int rvt_fast_reg_mr(struct rvt_qp *qp, struct ib_mr
Quoted 13 lines
mr->mr.page_shift = PAGE_SHIFT;drivers/infiniband/sw/rdmavt/mr.c +6
-2-return ib_sg_to_pages (ibmr, sg, sg_nents, sg_offset, - rvt_set_page); + ret =ib_sg_to_pages (ibmr, sg, sg_nents, sg_offset, rvt_set_page); + mr->mr.user_base = ibmr->iova; + mr->mr.iova = ibmr->iova; + mr->mr.offset = ibmr->iova - (u64 )mr->mr.map[0 ]->segs[0 ].vaddr; + mr->mr.length = (size_t )ibmr->length; +return ret; }/** @@ -671,6 +673,7 @@ int rvt_fast_reg_mr(struct rvt_qp *qp, struct ib_mr*ibmr, u32 key,
Quoted 7 lines
drivers/infiniband/sw/rdmavt/mr.c int rvt_fast_reg_mr(struct rvt_qp *qp, struct ib_mr *ibmr, u32 key, · line 676
Quoted 4 lines
drivers/infiniband/sw/rdmavt/mr.c +1
-0ibmr->rkey = key; mr->mr.lkey = key; mr->mr.access_flags = access; + mr->mr.iova = ibmr->iova;Isn't this a redundant assignment? If it isn't, then why is the
mr->mr.offsetnot being recalculated?
Quoted 3 lines
The
ib_mriova can be changed by the ULP to any value, but any value should index to the previously mapped MR's starting page and offset, hence those do not change, and we only update the iova.
Ah, got it. The ULP changes iova between the map and the fastreg, so the mr captures the fastreg value. Agreed.
Tom.
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
drivers/infiniband/sw/rdmavt/mr.c +0
-0atomic_set (&mr->mr.lkey_invalid,0 );return 0 ;
Jason Gunthorpe
AppliedQuoted 22 lines · Dennis Dalessandro
From: Josh Collier <[email protected]>
Current implementation was not properly handling frwr memory registrations. This was uncovered by:
commit 27f26cec761das
xprtrdma: Plant XID in on-the-wire RDMA offset (FRWR)
in which xprtrdma, which is used for NFS over RDMA, started failing as it was the first ULP to modify theib_mriova resulting in the NFS server getting REMOTE ACCESS ERROR when attempting to perform RDMA Writes to the client.The fix is to properly capture the true iova, offset, and length in the call to
ib_map_mr_sg, and then update the iova when processing theIB_WR_REG_MEMon the send queue.Fixes a41081aa5936 ("IB/rdmavt: Add support for ib_map_mr_sg")Reviewed-by Mike Marciniszyn <[email protected]>Reviewed-by Dennis Dalessandro <[email protected]>Reviewed-by Michael J. Ruhl <[email protected]>Signed-off-by Josh Collier <[email protected]>Signed-off-by Dennis Dalessandro <[email protected]>
Quoted 3 lines
--- drivers/infiniband/sw/rdmavt/mr.c | 17 ++++++++++------- 1 files changed, 10 insertions(+), 7 deletions(-)
Applied to for-rc
Thanks,
Jason