Skip to contentSkip to conversation

RDMA/rxe: Fix parameter errors

PATCH In review Bob Pearson 2 participants 6 messages Jan 19, 2023 – Mar 13, 2023

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

PATCH 1/1 RDMA/rxe: Fix parameter errors

Correct errors in rxe_param.h caused by extending the range of indices for MRs allowing it to overlap the range for MWs. Since the driver determines whether an rkey is for an MR or MW by comparing the index part of the rkey with these ranges this can cause an MR to be incorrectly determined to be an MW.

Additionally the parameters which determine the size of the index ranges for MR, MW, QP and SRQ are incorrect since the actual number of integers in the range [min, max] is (max - min + 1) not (max - min).

This patch corrects these errors.

Fixes 0994a1bcd5f7 ("RDMA/rxe: Bump up default maximum values used via uverbs")
---
 drivers/infiniband/sw/rxe/rxe_param.h | 27 +++++++++++++++++++--------
 1 file changed, 19 insertions(+), 8 deletions(-)
drivers/infiniband/sw/rxe/rxe_param.h +19 -8
index a754fc902e3d..14baa84d1d9d 100644
@@ -91,18 +91,29 @@ enum rxe_device_param {
 
 	RXE_MIN_QP_INDEX		= 16,
 	RXE_MAX_QP_INDEX		= DEFAULT_MAX_VALUE,
-	RXE_MAX_QP			= DEFAULT_MAX_VALUE - RXE_MIN_QP_INDEX,
+	RXE_MAX_QP			= RXE_MAX_QP_INDEX
+						- RXE_MIN_QP_INDEX + 1,
 
 	RXE_MIN_SRQ_INDEX		= 0x00020001,
 	RXE_MAX_SRQ_INDEX		= DEFAULT_MAX_VALUE,
-	RXE_MAX_SRQ			= DEFAULT_MAX_VALUE - RXE_MIN_SRQ_INDEX,
-
-	RXE_MIN_MR_INDEX		= 0x00000001,
+	RXE_MAX_SRQ			= RXE_MAX_SRQ_INDEX
+						- RXE_MIN_SRQ_INDEX + 1,
+
+	/*
+	 * MR and MW indices are converted to rkeys by shifting
+	 * left 8 bits and oring in an 8 bit key which either
+	 * belongs to the driver or the user depending on the
+	 * MR type. In order to determine if the rkey is an MR
+	 * or an MW the index ranges below must not overlap.
+	 */
+	RXE_MIN_MR_INDEX		= 1,
 	RXE_MAX_MR_INDEX		= DEFAULT_MAX_VALUE,
-	RXE_MAX_MR			= DEFAULT_MAX_VALUE - RXE_MIN_MR_INDEX,
-	RXE_MIN_MW_INDEX		= 0x00010001,
-	RXE_MAX_MW_INDEX		= 0x00020000,
-	RXE_MAX_MW			= 0x00001000,
+	RXE_MAX_MR			= RXE_MAX_MR_INDEX
+						- RXE_MIN_MR_INDEX + 1,
+	RXE_MIN_MW_INDEX		= DEFAULT_MAX_VALUE + 1,
+	RXE_MAX_MW_INDEX		= 2*DEFAULT_MAX_VALUE,
+	RXE_MAX_MW			= RXE_MAX_MW_INDEX
+						- RXE_MIN_MW_INDEX + 1,
 
 	RXE_MAX_PKT_PER_ACK		= 64,
 
Signature · 1 line
2.37.2
maintainer Tested-by Li Zhijian Reviewed-by Li Zhijian on v1 1/1
#1 lore raw
Quoted 11 lines · Bob Pearson

Correct errors in rxe_param.h caused by extending the range of indices for MRs allowing it to overlap the range for MWs. Since the driver determines whether an rkey is for an MR or MW by comparing the index part of the rkey with these ranges this can cause an MR to be incorrectly determined to be an MW.

Additionally the parameters which determine the size of the index ranges for MR, MW, QP and SRQ are incorrect since the actual number of integers in the range [min, max] is (max - min + 1) not (max - min).

Quoted 7 lines

This patch corrects these errors.

Fixes 0994a1bcd5f7 ("RDMA/rxe: Bump up default maximum values used via uverbs")
---
 drivers/infiniband/sw/rxe/rxe_param.h | 27 +++++++++++++++++++--------
 1 file changed, 19 insertions(+), 8 deletions(-)

This

commit 1aefe5c177c1922119afb4ee443ddd6ac3140b37

Author: Daisuke Matsuda <[email protected]>
Date:   Tue Dec 20 17:08:48 2022 +0900

RDMA/rxe: Prevent faulty rkey generation

If you create MRs more than 0x10000 times after loading the module, responder starts to reply NAKs for RDMA/Atomic operations because of rkey violation detected in check_rkey(). The root cause is that rkeys are incremented each time a new MR is created and the value overflows into the range reserved for MWs.

This commit also increases the value of RXE_MAX_MW that has been limited unlike other parameters.

Fixes 0994a1bcd5f7 ("RDMA/rxe: Bump up default maximum values used via uverbs")

Is already in v6.2-rc and conflicts with this patch, it looks like it is doing the same thing, can you sort it out please?

Thanks,
Jason

#2 lore raw
Quoted 45 lines · Jason Gunthorpe
Quoted 11 lines · Bob Pearson

Correct errors in rxe_param.h caused by extending the range of indices for MRs allowing it to overlap the range for MWs. Since the driver determines whether an rkey is for an MR or MW by comparing the index part of the rkey with these ranges this can cause an MR to be incorrectly determined to be an MW.

Additionally the parameters which determine the size of the index ranges for MR, MW, QP and SRQ are incorrect since the actual number of integers in the range [min, max] is (max - min + 1) not (max - min).

Quoted 7 lines

This patch corrects these errors.

Fixes 0994a1bcd5f7 ("RDMA/rxe: Bump up default maximum values used via uverbs")
---
 drivers/infiniband/sw/rxe/rxe_param.h | 27 +++++++++++++++++++--------
 1 file changed, 19 insertions(+), 8 deletions(-)

This

commit 1aefe5c177c1922119afb4ee443ddd6ac3140b37

Author: Daisuke Matsuda <[email protected]>
Date:   Tue Dec 20 17:08:48 2022 +0900

RDMA/rxe: Prevent faulty rkey generation

If you create MRs more than 0x10000 times after loading the module, responder starts to reply NAKs for RDMA/Atomic operations because of rkey violation detected in check_rkey(). The root cause is that rkeys are incremented each time a new MR is created and the value overflows into the range reserved for MWs.

This commit also increases the value of RXE_MAX_MW that has been limited unlike other parameters.

Fixes 0994a1bcd5f7 ("RDMA/rxe: Bump up default maximum values used via uverbs")
Quoted 5 lines

Is already in v6.2-rc and conflicts with this patch, it looks like it is doing the same thing, can you sort it out please?

Thanks,
Jason

Missed that one. Yes, they are basically identical except he cut the range in half and gave one to each and I doubled it. The other change I made is still a bug but much less important. It reports an incorrect max_xxx number in hca attributes but has no ill effect. We can leave it the way it is for now.

Bob

#3 lore raw
In reply to Jason Gunthorpe
Quoted 45 lines · Jason Gunthorpe
Quoted 11 lines · Bob Pearson

Correct errors in rxe_param.h caused by extending the range of indices for MRs allowing it to overlap the range for MWs. Since the driver determines whether an rkey is for an MR or MW by comparing the index part of the rkey with these ranges this can cause an MR to be incorrectly determined to be an MW.

Additionally the parameters which determine the size of the index ranges for MR, MW, QP and SRQ are incorrect since the actual number of integers in the range [min, max] is (max - min + 1) not (max - min).

Quoted 7 lines

This patch corrects these errors.

Fixes 0994a1bcd5f7 ("RDMA/rxe: Bump up default maximum values used via uverbs")
---
 drivers/infiniband/sw/rxe/rxe_param.h | 27 +++++++++++++++++++--------
 1 file changed, 19 insertions(+), 8 deletions(-)

This

commit 1aefe5c177c1922119afb4ee443ddd6ac3140b37

Author: Daisuke Matsuda <[email protected]>
Date:   Tue Dec 20 17:08:48 2022 +0900

RDMA/rxe: Prevent faulty rkey generation

If you create MRs more than 0x10000 times after loading the module, responder starts to reply NAKs for RDMA/Atomic operations because of rkey violation detected in check_rkey(). The root cause is that rkeys are incremented each time a new MR is created and the value overflows into the range reserved for MWs.

This commit also increases the value of RXE_MAX_MW that has been limited unlike other parameters.

Fixes 0994a1bcd5f7 ("RDMA/rxe: Bump up default maximum values used via uverbs")
Quoted 5 lines

Is already in v6.2-rc and conflicts with this patch, it looks like it is doing the same thing, can you sort it out please?

Thanks,
Jason

Did this get lost? for-next is now at 6.2-rc3 now and the bug is still in rxe_param.h.

Bob

maintainer
#4 lore raw
Quoted 46 lines · Bob Pearson
Quoted 45 lines · Jason Gunthorpe
Quoted 11 lines · Bob Pearson

Correct errors in rxe_param.h caused by extending the range of indices for MRs allowing it to overlap the range for MWs. Since the driver determines whether an rkey is for an MR or MW by comparing the index part of the rkey with these ranges this can cause an MR to be incorrectly determined to be an MW.

Additionally the parameters which determine the size of the index ranges for MR, MW, QP and SRQ are incorrect since the actual number of integers in the range [min, max] is (max - min + 1) not (max - min).

Quoted 7 lines

This patch corrects these errors.

Fixes 0994a1bcd5f7 ("RDMA/rxe: Bump up default maximum values used via uverbs")
---
 drivers/infiniband/sw/rxe/rxe_param.h | 27 +++++++++++++++++++--------
 1 file changed, 19 insertions(+), 8 deletions(-)

This

commit 1aefe5c177c1922119afb4ee443ddd6ac3140b37

Author: Daisuke Matsuda <[email protected]>
Date:   Tue Dec 20 17:08:48 2022 +0900

RDMA/rxe: Prevent faulty rkey generation

If you create MRs more than 0x10000 times after loading the module, responder starts to reply NAKs for RDMA/Atomic operations because of rkey violation detected in check_rkey(). The root cause is that rkeys are incremented each time a new MR is created and the value overflows into the range reserved for MWs.

This commit also increases the value of RXE_MAX_MW that has been limited unlike other parameters.

Fixes 0994a1bcd5f7 ("RDMA/rxe: Bump up default maximum values used via uverbs")
Quoted 8 lines
Quoted 5 lines

Is already in v6.2-rc and conflicts with this patch, it looks like it is doing the same thing, can you sort it out please?

Thanks,
Jason

Did this get lost? for-next is now at 6.2-rc3 now and the bug is still in rxe_param.h.

Check again we are at v6.3-rc1 now, if something needs to be fixed send a new patch..

Jason

#5 lore raw
Quoted 53 lines · Jason Gunthorpe
Quoted 52 lines · Bob Pearson
Quoted 50 lines · Jason Gunthorpe
Quoted 11 lines · Bob Pearson

Correct errors in rxe_param.h caused by extending the range of indices for MRs allowing it to overlap the range for MWs. Since the driver determines whether an rkey is for an MR or MW by comparing the index part of the rkey with these ranges this can cause an MR to be incorrectly determined to be an MW.

Additionally the parameters which determine the size of the index ranges for MR, MW, QP and SRQ are incorrect since the actual number of integers in the range [min, max] is (max - min + 1) not (max - min).

Quoted 7 lines

This patch corrects these errors.

Fixes 0994a1bcd5f7 ("RDMA/rxe: Bump up default maximum values used via uverbs")
---
 drivers/infiniband/sw/rxe/rxe_param.h | 27 +++++++++++++++++++--------
 1 file changed, 19 insertions(+), 8 deletions(-)

This

commit 1aefe5c177c1922119afb4ee443ddd6ac3140b37

Author: Daisuke Matsuda <[email protected]>
Date:   Tue Dec 20 17:08:48 2022 +0900

RDMA/rxe: Prevent faulty rkey generation

If you create MRs more than 0x10000 times after loading the module, responder starts to reply NAKs for RDMA/Atomic operations because of rkey violation detected in check_rkey(). The root cause is that rkeys are incremented each time a new MR is created and the value overflows into the range reserved for MWs.

This commit also increases the value of RXE_MAX_MW that has been limited unlike other parameters.

Fixes 0994a1bcd5f7 ("RDMA/rxe: Bump up default maximum values used via uverbs")

Is already in v6.2-rc and conflicts with this patch, it looks like it is doing the same thing, can you sort it out please?

Thanks,
Jason

Quoted 7 lines
Quoted 2 lines

Did this get lost? for-next is now at 6.2-rc3 now and the bug is still in rxe_param.h.

Check again we are at v6.3-rc1 now, if something needs to be fixed send a new patch..

Jason

Just checked. It now looks good in for-next.

Thanks

Bob