Subject: RDMA/rxe: Handle zero length rdma
Subsystems (2)
PATCH v4 1/1 Subject: RDMA/rxe: Handle zero length rdma
Currently the rxe driver does not handle all cases of zero length rdma operations correctly. The client does not have to provide an rkey for zero length RDMA read or write operations so the rkey provided may be invalid and should not be used to lookup an mr.
This patch corrects the driver to ignore the provided rkey if the reth length is zero for read or write operations and make sure to set the mr to NULL. In read_reply() if length is zero rxe_recheck_mr() is not called. Warnings are added in the routines in rxe_mr.c to catch NULL MRs when the length is non-zero.
---
v4:
Fixed a regression in flush operations because the rkey must be valid in that case even if the length is zero which maybe the case for selectivity level of the entire memory region.
v3:
Fixed my fat finger typing on v2. Mangled the patch.
v2:
Rebased to current for-next.
Cleaned up description to be a little more accurate.
--- drivers/infiniband/sw/rxe/rxe_mr.c | 7 ++++ drivers/infiniband/sw/rxe/rxe_resp.c | 59 +++++++++++++++++++++------- 2 files changed, 51 insertions(+), 15 deletions(-)
drivers/infiniband/sw/rxe/rxe_mr.c +7 -0
@@ -314,6 +314,9 @@ int rxe_mr_copy(struct rxe_mr *mr, u64 iova, void *addr,if (length ==0 )return 0 ; +if (WARN_ON (!mr)) +return -EINVAL ; +if (mr->ibmr.type ==IB_MR_TYPE_DMA ) {rxe_mr_copy_dma (mr, iova, addr, length, dir);return 0 ;
@@ -432,6 +435,10 @@ int rxe_flush_pmem_iova(struct rxe_mr *mr, u64 iova, unsigned int length)int err;u8 *va; +/* mr must be valid even if length is zero */ +if (WARN_ON (!mr)) +return -EINVAL ; +if (length ==0 )return 0 ;
drivers/infiniband/sw/rxe/rxe_resp.c +44 -15
125 lines not on this page · raw message
base-commit: 01884059c09fa770206ca03bf3660502f97d5fdd
Signature · 1 line
2.37.2
Quoted 12 lines · Bob Pearson
Currently the rxe driver does not handle all cases of zero length rdma operations correctly. The client does not have to provide an rkey for zero length RDMA read or write operations so the rkey provided may be invalid and should not be used to lookup an mr.
This patch corrects the driver to ignore the provided rkey if the reth length is zero for read or write operations and make sure to set the mr to NULL. In
read_reply()if length is zerorxe_recheck_mr()is not called. Warnings are added in the routines inrxe_mr.cto catch NULL MRs when the length is non-zero.
<...>
drivers/infiniband/sw/rxe/rxe_mr.c int rxe_flush_pmem_iova(struct rxe_mr *mr, u64 iova, unsigned int length) · line 440
Quoted 7 lines
drivers/infiniband/sw/rxe/rxe_mr.c +3
-0@@ -432,6 +435,10 @@ int rxe_flush_pmem_iova(struct rxe_mr *mr, u64 iova, unsigned int length)int err;u8 *va; +/* mr must be valid even if length is zero */ +if (WARN_ON (!mr)) +return -EINVAL ;
If 'mr' is NULL, NULL pointer dereference can occur in process_flush() before reaching here. Isn't it better to do the check in process_flush()?
Quoted 18 lines
drivers/infiniband/sw/rxe/rxe_mr.c +1
-0+if (length ==0 )return 0 ;drivers/infiniband/sw/rxe/rxe_resp.c +5
-0@@ -420,13 +420,23 @@ static enum resp_states rxe_resp_check_length(struct rxe_qp *qp,return RESPST_CHK_RKEY ; } +/* if the reth length field is zero we can assume nothing +* about the rkey value and should not validate or use it. +* Instead set qp->resp.rkey to 0 which is an invalid rkey +* value since the minimum index part is 1. +*/ static void qp_resp_from_reth (struct rxe_qp *qp,struct rxe_pkt_info *pkt)
drivers/infiniband/sw/rxe/rxe_resp.c static enum resp_states rxe_resp_check_length(struct rxe_qp *qp, · line 435
Quoted 10 lines
drivers/infiniband/sw/rxe/rxe_resp.c +4
-3{ +unsigned int length =reth_len (pkt); + qp->resp.va =reth_va (pkt); qp->resp.offset =0 ; - qp->resp.rkey =reth_rkey (pkt); - qp->resp.resid =reth_len (pkt); - qp->resp.length =reth_len (pkt); + qp->resp.resid = length; + qp->resp.length = length;
As you know, the comment above this function is applicable only to RDMA Read and Write. What do you say to mentioning FLUSH here rather than at the one in rxe_flush_pmem_iova().
Thanks,
Daisuke
Quoted 6 lines
drivers/infiniband/sw/rxe/rxe_resp.c +4
-0+if (pkt->mask &RXE_READ_OR_WRITE_MASK && length ==0 ) + qp->resp.rkey =0 ; +else + qp->resp.rkey =reth_rkey (pkt); }
<...>
Quoted 63 lines · Daisuke Matsuda (Fujitsu)
Quoted 12 lines · Bob Pearson
Currently the rxe driver does not handle all cases of zero length rdma operations correctly. The client does not have to provide an rkey for zero length RDMA read or write operations so the rkey provided may be invalid and should not be used to lookup an mr.
This patch corrects the driver to ignore the provided rkey if the reth length is zero for read or write operations and make sure to set the mr to NULL. In
read_reply()if length is zerorxe_recheck_mr()is not called. Warnings are added in the routines inrxe_mr.cto catch NULL MRs when the length is non-zero.<...>
drivers/infiniband/sw/rxe/rxe_mr.c int rxe_flush_pmem_iova(struct rxe_mr *mr, u64 iova, unsigned int length) · line 440
Quoted 7 lines
drivers/infiniband/sw/rxe/rxe_mr.c +3
-0@@ -432,6 +435,10 @@ int rxe_flush_pmem_iova(struct rxe_mr *mr, u64 iova, unsigned int length)int err;u8 *va; +/* mr must be valid even if length is zero */ +if (WARN_ON (!mr)) +return -EINVAL ;If 'mr' is NULL, NULL pointer dereference can occur in
process_flush()before reaching here. Isn't it better to do the check inprocess_flush()?Quoted 18 lines
drivers/infiniband/sw/rxe/rxe_mr.c +1
-0+if (length ==0 )return 0 ;drivers/infiniband/sw/rxe/rxe_resp.c +5
-0@@ -420,13 +420,23 @@ static enum resp_states rxe_resp_check_length(struct rxe_qp *qp,return RESPST_CHK_RKEY ; } +/* if the reth length field is zero we can assume nothing +* about the rkey value and should not validate or use it. +* Instead set qp->resp.rkey to 0 which is an invalid rkey +* value since the minimum index part is 1. +*/ static void qp_resp_from_reth (struct rxe_qp *qp,struct rxe_pkt_info *pkt)drivers/infiniband/sw/rxe/rxe_resp.c static enum resp_states rxe_resp_check_length(struct rxe_qp *qp, · line 435
Quoted 10 lines
drivers/infiniband/sw/rxe/rxe_resp.c +4
-3{ +unsigned int length =reth_len (pkt); + qp->resp.va =reth_va (pkt); qp->resp.offset =0 ; - qp->resp.rkey =reth_rkey (pkt); - qp->resp.resid =reth_len (pkt); - qp->resp.length =reth_len (pkt); + qp->resp.resid = length; + qp->resp.length = length;As you know, the comment above this function is applicable only to RDMA Read and Write. What do you say to mentioning FLUSH here rather than at the one in
rxe_flush_pmem_iova().Thanks,
Daisuke
Quoted 9 lines
Quoted 6 lines
drivers/infiniband/sw/rxe/rxe_resp.c +4
-0+if (pkt->mask &RXE_READ_OR_WRITE_MASK && length ==0 ) + qp->resp.rkey =0 ; +else + qp->resp.rkey =reth_rkey (pkt); }<...>
Daisuke,
The updated patch no longer sets mr = NULL for flush. This check is mainly to defend against someone changing it again in the future and is near where mr is dereferenced. You are correct about the comment I can change it. Will give it a day or two to see if anything else comes in.
Regards,
Bob Pearson
Daisuke Matsuda (Fujitsu)
Reviewed-byQuoted 9 lines · Bob Pearson
Quoted 7 lines · Daisuke Matsuda (Fujitsu)
Quoted 6 lines · Bob Pearson
Currently the rxe driver does not handle all cases of zero length rdma operations correctly. The client does not have to provide an rkey for zero length RDMA read or write operations so the rkey provided may be invalid and should not be used to lookup an mr.
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
This patch corrects the driver to ignore the provided rkey if the reth length is zero for read or write operations and make sure to set the mr to NULL. In
read_reply()if length is zerorxe_recheck_mr()is not called. Warnings are added in the routines inrxe_mr.cto catch NULL MRs when the length is non-zero.
The change looks good. I will leave it to you whether to fix the comment.
Thanks,
Daisuke
Quoted 68 lines
Quoted 50 lines
<...>
drivers/infiniband/sw/rxe/rxe_mr.c int rxe_flush_pmem_iova(struct rxe_mr *mr, u64 iova, unsigned int length) · line 440
Quoted 7 lines
drivers/infiniband/sw/rxe/rxe_mr.c +3
-0@@ -432,6 +435,10 @@ int rxe_flush_pmem_iova(struct rxe_mr *mr, u64 iova, unsigned int length)int err;u8 *va; +/* mr must be valid even if length is zero */ +if (WARN_ON (!mr)) +return -EINVAL ;If 'mr' is NULL, NULL pointer dereference can occur in
process_flush()before reaching here. Isn't it better to do the check inprocess_flush()?Quoted 18 lines
drivers/infiniband/sw/rxe/rxe_mr.c +1
-0+if (length ==0 )return 0 ;drivers/infiniband/sw/rxe/rxe_resp.c +5
-0@@ -420,13 +420,23 @@ static enum resp_states rxe_resp_check_length(struct rxe_qp *qp,return RESPST_CHK_RKEY ; } +/* if the reth length field is zero we can assume nothing +* about the rkey value and should not validate or use it. +* Instead set qp->resp.rkey to 0 which is an invalid rkey +* value since the minimum index part is 1. +*/ static void qp_resp_from_reth (struct rxe_qp *qp,struct rxe_pkt_info *pkt)drivers/infiniband/sw/rxe/rxe_resp.c static enum resp_states rxe_resp_check_length(struct rxe_qp *qp, · line 435
Quoted 10 lines
drivers/infiniband/sw/rxe/rxe_resp.c +4
-3{ +unsigned int length =reth_len (pkt); + qp->resp.va =reth_va (pkt); qp->resp.offset =0 ; - qp->resp.rkey =reth_rkey (pkt); - qp->resp.resid =reth_len (pkt); - qp->resp.length =reth_len (pkt); + qp->resp.resid = length; + qp->resp.length = length;As you know, the comment above this function is applicable only to RDMA Read and Write. What do you say to mentioning FLUSH here rather than at the one in
rxe_flush_pmem_iova().Thanks,
DaisukeQuoted 9 lines
Quoted 6 lines
drivers/infiniband/sw/rxe/rxe_resp.c +4
-0+if (pkt->mask &RXE_READ_OR_WRITE_MASK && length ==0 ) + qp->resp.rkey =0 ; +else + qp->resp.rkey =reth_rkey (pkt); }<...>
Daisuke,
The updated patch no longer sets mr = NULL for flush. This check is mainly to defend against someone changing it again in the future and is near where mr is dereferenced. You are correct about the comment I can change it. Will give it a day or two to see if anything else comes in.
Regards,
Bob Pearson
Quoted 10 lines · Bob Pearson
drivers/infiniband/sw/rxe/rxe_resp.c +4
-0+/* resolve the packet rkey to qp->resp.mr or set qp->resp.mr to NULL +* if an invalid rkey is received or the rdma length is zero. For middle +* or last packets use the stored value of mr. +*/ static enum resp_states check_rkey (struct rxe_qp *qp,struct rxe_pkt_info *pkt) {@@ -473,10 +487,12 @@ static enum resp_states check_rkey(struct rxe_qp *qp,return RESPST_EXECUTE ; }
drivers/infiniband/sw/rxe/rxe_resp.c static enum resp_states check_rkey(struct rxe_qp *qp, · line 495
Quoted 10 lines
drivers/infiniband/sw/rxe/rxe_resp.c +5
-3-/* A zero-byte op is not required to set an addr or rkey. See C9-88 */ +/* A zero-byte read or write op is not required to +* set an addr or rkey. See C9-88 +*/ if ((pkt->mask &RXE_READ_OR_WRITE_MASK ) && - (pkt->mask &RXE_RETH_MASK ) && -reth_len (pkt) ==0 ) { + (pkt->mask &RXE_RETH_MASK ) &&reth_len (pkt) ==0 ) { + qp->resp.mr =NULL ;
You are making sure 'qp->resp.mr = NULL', but I doubt the previous qp->resp.mr will leak after this assignment when its value is not NULL.
Thanks
Zhijian
Quoted 9 lines
drivers/infiniband/sw/rxe/rxe_resp.c +1
-0return RESPST_EXECUTE ; }@@ -555,6 +571,7 @@ static enum resp_states check_rkey(struct rxe_qp *qp,return RESPST_EXECUTE ; err: + qp->resp.mr =NULL ;if (mr)
Quoted 13 lines · Zhijian/李 智坚
Quoted 11 lines · Bob Pearson
drivers/infiniband/sw/rxe/rxe_resp.c +4
-0+/* resolve the packet rkey to qp->resp.mr or set qp->resp.mr to NULL +* if an invalid rkey is received or the rdma length is zero. For middle +* or last packets use the stored value of mr. +*/ static enum resp_states check_rkey (struct rxe_qp *qp,struct rxe_pkt_info *pkt) {@@ -473,10 +487,12 @@ static enum resp_states check_rkey(struct rxe_qp *qp,return RESPST_EXECUTE ; }
Quoted 12 lines
drivers/infiniband/sw/rxe/rxe_resp.c static enum resp_states check_rkey(struct rxe_qp *qp, · line 495
Quoted 9 lines
drivers/infiniband/sw/rxe/rxe_resp.c +5
-3-/* A zero-byte op is not required to set an addr or rkey. See C9-88 */ +/* A zero-byte read or write op is not required to +* set an addr or rkey. See C9-88 +*/ if ((pkt->mask &RXE_READ_OR_WRITE_MASK ) && - (pkt->mask &RXE_RETH_MASK ) && -reth_len (pkt) ==0 ) { + (pkt->mask &RXE_RETH_MASK ) &&reth_len (pkt) ==0 ) { + qp->resp.mr =NULL ;You are making sure '
qp->resp.mr= NULL', but I doubt the previousqp->resp.mrwill leak after this assignment when its value is not NULL.
I do not see what you mean by " the previous qp->resp.mr ". As far as I understand, 'qp->resp.mr' is set to NULL in cleanup() before responder completes its work, so it is not supposed to be reused unlike 'res->read.rkey' being retained for multi-packet Read responses. Could you elaborate on your view?
Daisuke
Quoted 14 lines
Thanks
ZhijianQuoted 9 lines
drivers/infiniband/sw/rxe/rxe_resp.c +1
-0return RESPST_EXECUTE ; }@@ -555,6 +571,7 @@ static enum resp_states check_rkey(struct rxe_qp *qp,return RESPST_EXECUTE ; err: + qp->resp.mr =NULL ;if (mr)
Quoted 27 lines · Daisuke/松田 大輔
Quoted 13 lines · Zhijian/李 智坚
Quoted 11 lines · Bob Pearson
drivers/infiniband/sw/rxe/rxe_resp.c +4
-0+/* resolve the packet rkey to qp->resp.mr or set qp->resp.mr to NULL +* if an invalid rkey is received or the rdma length is zero. For middle +* or last packets use the stored value of mr. +*/ static enum resp_states check_rkey (struct rxe_qp *qp,struct rxe_pkt_info *pkt) {@@ -473,10 +487,12 @@ static enum resp_states check_rkey(struct rxe_qp *qp,return RESPST_EXECUTE ; }Quoted 12 lines
drivers/infiniband/sw/rxe/rxe_resp.c static enum resp_states check_rkey(struct rxe_qp *qp, · line 495
Quoted 9 lines
drivers/infiniband/sw/rxe/rxe_resp.c +5
-3-/* A zero-byte op is not required to set an addr or rkey. See C9-88 */ +/* A zero-byte read or write op is not required to +* set an addr or rkey. See C9-88 +*/ if ((pkt->mask &RXE_READ_OR_WRITE_MASK ) && - (pkt->mask &RXE_RETH_MASK ) && -reth_len (pkt) ==0 ) { + (pkt->mask &RXE_RETH_MASK ) &&reth_len (pkt) ==0 ) { + qp->resp.mr =NULL ;You are making sure '
qp->resp.mr= NULL', but I doubt the previousqp->resp.mrwill leak after this assignment when its value is not NULL.
Quoted 5 lines
I do not see what you mean by " the previous
qp->resp.mr". As far as I understand, 'qp->resp.mr' is set to NULL incleanup()before responder completes its work, so it is not supposed to be reused unlike 'res->read.rkey' being retained for multi-packet Read responses. Could you elaborate on your view?
IMO every 'qp->resp.mr' assignment will take a mr reference, so we should drop the this reference if we want to assign it a new mr again.
Theoretically, it should be changed to something like
if (qp->resp.mr) {
rxe_put(qp->resp.mr)
qp->resp.mr = NULL;
}Quoted 9 lines
Quoted 9 lines
drivers/infiniband/sw/rxe/rxe_resp.c static enum resp_states check_rkey(struct rxe_qp *qp, · line 574
Quoted 9 lines
drivers/infiniband/sw/rxe/rxe_resp.c +1
-0return RESPST_EXECUTE ; }@@ -555,6 +571,7 @@ static enum resp_states check_rkey(struct rxe_qp *qp,return RESPST_EXECUTE ; err: + qp->resp.mr =NULL ;
ditto
Thanks
Zhijian
Quoted 1 line
Quoted 1 line
Quoted 1 line
(quoted diff) +0
-0if (mr)
Quoted 42 lines · Zhijian/李 智坚
Quoted 14 lines · Daisuke/松田 大輔
Quoted 13 lines · Zhijian/李 智坚
Quoted 11 lines · Bob Pearson
drivers/infiniband/sw/rxe/rxe_resp.c +4
-0+/* resolve the packet rkey to qp->resp.mr or set qp->resp.mr to NULL +* if an invalid rkey is received or the rdma length is zero. For middle +* or last packets use the stored value of mr. +*/ static enum resp_states check_rkey (struct rxe_qp *qp,struct rxe_pkt_info *pkt) {@@ -473,10 +487,12 @@ static enum resp_states check_rkey(struct rxe_qp *qp,return RESPST_EXECUTE ; }Quoted 11 lines
Quoted 11 lines
drivers/infiniband/sw/rxe/rxe_resp.c static enum resp_states check_rkey(struct rxe_qp *qp, · line 495
Quoted 9 lines
drivers/infiniband/sw/rxe/rxe_resp.c +5
-3-/* A zero-byte op is not required to set an addr or rkey. See C9-88 */ +/* A zero-byte read or write op is not required to +* set an addr or rkey. See C9-88 +*/ if ((pkt->mask &RXE_READ_OR_WRITE_MASK ) && - (pkt->mask &RXE_RETH_MASK ) && -reth_len (pkt) ==0 ) { + (pkt->mask &RXE_RETH_MASK ) &&reth_len (pkt) ==0 ) { + qp->resp.mr =NULL ;You are making sure '
qp->resp.mr= NULL', but I doubt the previousqp->resp.mrwill leak after this assignmentwhen its
Quoted 7 lines
Quoted 1 line
value is not NULL.
I do not see what you mean by " the previous
qp->resp.mr". As far as I understand, 'qp->resp.mr' is set to NULL incleanup()before responder completes its work, so it is not supposed to be reused unlike 'res->read.rkey' being retained for multi-packet Read responses. Could you elaborate on your view?IMO every '
qp->resp.mr' assignment will take a mr reference, so we should drop the this reference if we want to assign it a new mr again.
Quoted 5 lines
Theoretically, it should be changed to something like
if (qp->resp.mr) { rxe_put(qp->resp.mr) qp->resp.mr = NULL; }
You are correct about what you wrote.
' qp->resp.mr = NULL;' and 'rxe_put(qp->resp.mr);' come in pairs in other parts of rxe. I agree the same principle should be applied here.
However, it seems to me that this 'qp->resp.mr' is always NULL. For all operations other than multi-packet Read responses, cleanup() must be executed before responder completes the work. On the other hand, for multi-packet Read replies, responder does not execute cleanup() and thus does not drop the reference until the last reply is sent, but it does not execute check_rkey() either since it uses the fast path jumping from get_req() to read_reply().
From practical perspective, we can just remove ' qp->resp.mr = NULL;', and then 'qp->resp.mr' will still be NULL. From theoretical perspective, we can make the change as Zhijian suggested. What other guys think about this?
Thanks,
Daisuke
Quoted 20 lines
Quoted 5 lines
Quoted 5 lines
Quoted 8 lines
Quoted 8 lines
drivers/infiniband/sw/rxe/rxe_resp.c static enum resp_states check_rkey(struct rxe_qp *qp, · line 574
Quoted 8 lines
drivers/infiniband/sw/rxe/rxe_resp.c +1
-0return RESPST_EXECUTE ; }@@ -555,6 +571,7 @@ static enum resp_states check_rkey(struct rxe_qp *qp,return RESPST_EXECUTE ; err: + qp->resp.mr =NULL ;ditto
Thanks
ZhijianQuoted 1 line
Quoted 1 line
Quoted 1 line
(quoted diff) +0
-0if (mr)
Quoted 36 lines · On 2/9/23 04:20, [email protected] wrote:
Quoted 27 lines · Daisuke/松田 大輔
Quoted 13 lines · Zhijian/李 智坚
Quoted 11 lines · Bob Pearson
drivers/infiniband/sw/rxe/rxe_resp.c +4
-0+/* resolve the packet rkey to qp->resp.mr or set qp->resp.mr to NULL +* if an invalid rkey is received or the rdma length is zero. For middle +* or last packets use the stored value of mr. +*/ static enum resp_states check_rkey (struct rxe_qp *qp,struct rxe_pkt_info *pkt) {@@ -473,10 +487,12 @@ static enum resp_states check_rkey(struct rxe_qp *qp,return RESPST_EXECUTE ; }Quoted 12 lines
drivers/infiniband/sw/rxe/rxe_resp.c static enum resp_states check_rkey(struct rxe_qp *qp, · line 495
Quoted 9 lines
drivers/infiniband/sw/rxe/rxe_resp.c +5
-3-/* A zero-byte op is not required to set an addr or rkey. See C9-88 */ +/* A zero-byte read or write op is not required to +* set an addr or rkey. See C9-88 +*/ if ((pkt->mask &RXE_READ_OR_WRITE_MASK ) && - (pkt->mask &RXE_RETH_MASK ) && -reth_len (pkt) ==0 ) { + (pkt->mask &RXE_RETH_MASK ) &&reth_len (pkt) ==0 ) { + qp->resp.mr =NULL ;You are making sure '
qp->resp.mr= NULL', but I doubt the previousqp->resp.mrwill leak after this assignment when its value is not NULL.Quoted 5 lines
I do not see what you mean by " the previous
qp->resp.mr". As far as I understand, 'qp->resp.mr' is set to NULL incleanup()before responder completes its work, so it is not supposed to be reused unlike 'res->read.rkey' being retained for multi-packet Read responses. Could you elaborate on your view?
Quoted 9 lines
IMO every '
qp->resp.mr' assignment will take a mr reference, so we should drop the this reference if we want to assign it a new mr again.Theoretically, it should be changed to something like
if (qp->resp.mr) { rxe_put(qp->resp.mr) qp->resp.mr = NULL; }
Generally I agree with the idea that a (few) assignments like this, even if after minutes of looking over the code can be shown to be unneeded because it is already NULL or will get deleted, are helpful if it makes the code easier to understand and prevents later changes to unintentionally break things.
Bob
Quoted 19 lines
Quoted 4 lines
Quoted 4 lines
Quoted 8 lines
Quoted 8 lines
drivers/infiniband/sw/rxe/rxe_resp.c static enum resp_states check_rkey(struct rxe_qp *qp, · line 574
Quoted 8 lines
drivers/infiniband/sw/rxe/rxe_resp.c +1
-0return RESPST_EXECUTE ; }@@ -555,6 +571,7 @@ static enum resp_states check_rkey(struct rxe_qp *qp,return RESPST_EXECUTE ; err: + qp->resp.mr =NULL ;ditto
Thanks
ZhijianQuoted 1 line
Quoted 1 line
Quoted 1 line
(quoted diff) +0
-0if (mr)
Quoted 37 lines · Bob Pearson
Quoted 36 lines · On 2/9/23 04:20, [email protected] wrote:
Quoted 32 lines · Daisuke/松田 大輔
Quoted 13 lines · Zhijian/李 智坚
Quoted 11 lines · Bob Pearson
drivers/infiniband/sw/rxe/rxe_resp.c +4
-0+/* resolve the packet rkey to qp->resp.mr or set qp->resp.mr to NULL +* if an invalid rkey is received or the rdma length is zero. For middle +* or last packets use the stored value of mr. +*/ static enum resp_states check_rkey (struct rxe_qp *qp,struct rxe_pkt_info *pkt) {@@ -473,10 +487,12 @@ static enum resp_states check_rkey(struct rxe_qp *qp,return RESPST_EXECUTE ; }Quoted 12 lines
drivers/infiniband/sw/rxe/rxe_resp.c static enum resp_states check_rkey(struct rxe_qp *qp, · line 495
Quoted 9 lines
drivers/infiniband/sw/rxe/rxe_resp.c +5
-3-/* A zero-byte op is not required to set an addr or rkey. See C9-88 */ +/* A zero-byte read or write op is not required to +* set an addr or rkey. See C9-88 +*/ if ((pkt->mask &RXE_READ_OR_WRITE_MASK ) && - (pkt->mask &RXE_RETH_MASK ) && -reth_len (pkt) ==0 ) { + (pkt->mask &RXE_RETH_MASK ) &&reth_len (pkt) ==0 ) { + qp->resp.mr =NULL ;You are making sure '
qp->resp.mr= NULL', but I doubt the previousqp->resp.mrwill leak after this assignment when its value is not NULL.I do not see what you mean by " the previous
qp->resp.mr". As far as I understand, 'qp->resp.mr' is set to NULL incleanup()before responder completes its work, so it is not supposed to be reused unlike 'res->read.rkey' being retained for multi-packet Read responses. Could you elaborate on your view?
Quoted 8 lines
Quoted 8 lines
IMO every '
qp->resp.mr' assignment will take a mr reference, so we should drop the this reference if we want to assign it a new mr again.Theoretically, it should be changed to something like
if (qp->resp.mr) { rxe_put(qp->resp.mr) qp->resp.mr = NULL; }
Without "qp->resp.mr = NULL;", it is possible to cause use after free error if qp->resp.mr is used again.
Zhu Yanjun
Quoted 26 lines
Quoted 1 line
Generally I agree with the idea that a (few) assignments like this, even if after minutes of looking over the code can be shown to be unneeded because it is already NULL or will get deleted, are helpful if it makes the code easier to understand and prevents later changes to unintentionally break things.
Bob
Quoted 19 lines
Quoted 4 lines
Quoted 4 lines
Quoted 8 lines
Quoted 8 lines
drivers/infiniband/sw/rxe/rxe_resp.c static enum resp_states check_rkey(struct rxe_qp *qp, · line 574
Quoted 8 lines
drivers/infiniband/sw/rxe/rxe_resp.c +1
-0return RESPST_EXECUTE ; }@@ -555,6 +571,7 @@ static enum resp_states check_rkey(struct rxe_qp *qp,return RESPST_EXECUTE ; err: + qp->resp.mr =NULL ;ditto
Thanks
ZhijianQuoted 1 line
Quoted 1 line
Quoted 1 line
(quoted diff) +0
-0if (mr)
Quoted 13 lines · Bob Pearson
Currently the rxe driver does not handle all cases of zero length rdma operations correctly. The client does not have to provide an rkey for zero length RDMA read or write operations so the rkey provided may be invalid and should not be used to lookup an mr.
This patch corrects the driver to ignore the provided rkey if the reth length is zero for read or write operations and make sure to set the mr to NULL. In
read_reply()if length is zerorxe_recheck_mr()is not called. Warnings are added in the routines inrxe_mr.cto catch NULL MRs when the length is non-zero.Signed-off-by Bob Pearson <[email protected]>Reviewed-by Daisuke Matsuda <[email protected]>
Quoted 7 lines
---
v4:
Fixed a regression in flush operations because the rkey must be valid in that case even if the length is zero which maybe the case for selectivity level of the entire memory region.Reported-by Daisuke Matsuda <[email protected]>
Applied to for-next, I added a fixes line for day 0
Jason