Skip to contentSkip to conversation

RDMA/core: Prevent rereg_mr for dmabuf umems

PATCH Applied Jacob Moroni 2 participants 3 messages Jul 28 – Aug 5

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

PATCH 1/2 RDMA/core: Prevent rereg_mr for dmabuf umems

The rereg_mr method has always been explicitly blocked in rdma-core for dmabuf MRs anyway, so add a check to the ib_umem_check_rereg helper so that each driver doesn't need to handle it.

Depending on how the driver handled rereg_mr, this also has the benefit of preventing rereg_mr from being used to add the IB_ACCESS_MW_BIND flag to a dmabuf MR. This flag is not allowed during registration, so it seems sensible to prevent it from being added back with rereg_mr.

Preventing IB_ACCESS_MW_BIND is important for drivers that support revocable dmabufs and implement "revoke" by issuing a dereg_mr command to the HW because most(?) HW will reject this command if the MR has windows bound to it, and a failure to revoke is supposed to trigger a function reset.

---
 drivers/infiniband/core/umem.c | 3 +++
 1 file changed, 3 insertions(+)
drivers/infiniband/core/umem.c +3 -0
index 81f44dadfa52..88110b9661f5 100644
@@ -677,6 +677,9 @@ int ib_umem_check_rereg(struct ib_umem *umem, int flags, int new_access_flags)
 	if (!umem)
 		return 0;
 
+	if (umem->is_dmabuf)
+		return -EOPNOTSUPP;
+
 	if ((flags & IB_MR_REREG_ACCESS) && !(flags & IB_MR_REREG_TRANS))
 		if (ib_access_writable(new_access_flags) && !umem->writable)
 			return -EACCES;
Signature · 1 line
2.55.0.229.g6434b31f56-goog
#1 lore raw

PATCH 2/2 RDMA/irdma: Remove dmabuf checks in rereg_mr

Now that ib_umem_check_rereg entirely blocks rereg for dmabuf umems, the extra logic can be removed from irdma.

---
 drivers/infiniband/hw/irdma/verbs.c | 31 +----------------------------
 1 file changed, 1 insertion(+), 30 deletions(-)
drivers/infiniband/hw/irdma/verbs.c +1 -30
index 16c80367270f..5e03cf39fa6e 100644
@@ -3885,8 +3885,6 @@ static struct ib_mr *irdma_rereg_user_mr(struct ib_mr *ib_mr, int flags,
 	struct irdma_device *iwdev = to_iwdev(ib_mr->device);
 	struct irdma_mr *iwmr = to_iwmr(ib_mr);
 	struct irdma_pbl *iwpbl = &iwmr->iwpbl;
-	bool dmabuf_revocable = iwmr->region && iwmr->region->is_dmabuf;
-	struct ib_umem_dmabuf *umem_dmabuf;
 	int ret;
 
 	ret = ib_no_udata_io(udata);
@@ -3903,26 +3901,9 @@ static struct ib_mr *irdma_rereg_user_mr(struct ib_mr *ib_mr, int flags,
 	if (ret)
 		return ERR_PTR(ret);
 
-	if (dmabuf_revocable) {
-		umem_dmabuf = to_ib_umem_dmabuf(iwmr->region);
-
-		ib_umem_dmabuf_revoke_lock(umem_dmabuf);
-
-		/* If the dmabuf has been revoked, it means that the region has
-		 * been invalidated in HW. We must not allow it to become valid
-		 * again unless the user is requesting a change in translation
-		 * which will end up dropping the umem dmabuf and allocating an
-		 * entirely new umem anyway.
-		 */
-		if (umem_dmabuf->revoked && !(flags & IB_MR_REREG_TRANS)) {
-			ret = -EINVAL;
-			goto err_unlock;
-		}
-	}
-
 	ret = irdma_hwdereg_mr(ib_mr);
 	if (ret)
-		goto err_unlock;
+		return ERR_PTR(ret);
 
 	if (flags & IB_MR_REREG_ACCESS)
 		iwmr->access = new_access;
@@ -3939,12 +3920,6 @@ static struct ib_mr *irdma_rereg_user_mr(struct ib_mr *ib_mr, int flags,
 			iwpbl->pbl_allocated = false;
 		}
 
-		if (dmabuf_revocable) {
-			/* Must unlock before release to prevent deadlock */
-			ib_umem_dmabuf_revoke_unlock(umem_dmabuf);
-			dmabuf_revocable = false;
-		}
-
 		if (iwmr->region) {
 			ib_umem_release(iwmr->region);
 			iwmr->region = NULL;
@@ -3955,10 +3930,6 @@ static struct ib_mr *irdma_rereg_user_mr(struct ib_mr *ib_mr, int flags,
 		ret = irdma_hwreg_mr(iwdev, iwmr, iwmr->access);
 	}
 
-err_unlock:
-	if (dmabuf_revocable)
-		ib_umem_dmabuf_revoke_unlock(umem_dmabuf);
-
 	return ret ? ERR_PTR(ret) : NULL;
 }
 
Signature · 1 line
2.55.0.229.g6434b31f56-goog
#2 lore raw
In reply to Jacob Moroni
Quoted 5 lines · Jacob Moroni

The rereg_mr method has always been explicitly blocked in rdma-core for dmabuf MRs anyway, so add a check to the ib_umem_check_rereg helper so that each driver doesn't need to handle it.

Quoted 7 lines

Depending on how the driver handled rereg_mr, this also has the benefit of preventing rereg_mr from being used to add the IB_ACCESS_MW_BIND flag to a dmabuf MR. This flag is not allowed during registration, so it seems sensible to prevent it from being added back with rereg_mr.

[...]

Applied, thanks!

[1/2] RDMA/core: Prevent rereg_mr for dmabuf umems

      https://git.kernel.org/rdma/rdma/c/09d09e5d79f317
[2/2] RDMA/irdma: Remove dmabuf checks in rereg_mr
      https://git.kernel.org/rdma/rdma/c/d24ce259185e76

Best regards,

Signature · 1 line
Leon Romanovsky <[email protected]>