Skip to contentSkip to conversation

RDMA/siw: Clear association under lock if siw_qp_modify fails in siw_accept

PATCH Applied Guoqing Jiang 4 participants 6 messages Aug 27 – Sep 4

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

PATCH v2 1/1 RDMA/siw: Clear association under lock if siw_qp_modify fails in siw_accept

We need to clear cep before release state_lock as siw_qp_llp_close and siw_qp_modify->siw_qp_llp_close did.

Otherwise if siw_qp_modify() fails in siw_accept(), the QP's state_lock is released before the error path cleanup. A concurrent ibv_modify_qp() transitioning the QP to ERROR can race in this window:

siw_accept() ibv_modify_qp(ERROR)
---------------------- ----------------------
siw_qp_modify() fails
up_write(&qp->state_lock)

                                     down_write(&qp->state_lock)
                                     nextstate_from_idle():
				     if (qp->cep)
                                       siw_cep_put(qp->cep) <- frees cep
                                       qp->cep = NULL
  goto error
    cep->qp = NULL                   <- UAF

Clear qp->cep and drop the association reference taken by siw_cep_get(), all under the write lock held from the initial down_write(&qp->state_lock). Thread B therefore sees qp->cep == NULL, skips its own put, and cannot free the cep before siw_accept() is done with it.

---
V2: remove redundant code per Bernard's review

 drivers/infiniband/sw/siw/siw_cm.c | 7 +++++--
 1 file changed, 5 insertions(+), 2 deletions(-)
drivers/infiniband/sw/siw/siw_cm.c +5 -2
index 0245b25e7271..ed49818793dd 100644
@@ -1719,9 +1719,12 @@ int siw_accept(struct iw_cm_id *id, struct iw_cm_conn_param *params)
 			   SIW_QP_ATTR_STATE | SIW_QP_ATTR_LLP_HANDLE |
 				   SIW_QP_ATTR_ORD | SIW_QP_ATTR_IRD |
 				   SIW_QP_ATTR_MPA);
+	if (rv) {
+		qp->cep = NULL;
+		siw_cep_put(cep);
+		goto error_unlock;
+	}
 	up_write(&qp->state_lock);
-	if (rv)
-		goto error;
 
 	siw_dbg_cep(cep, "[QP %u]: send mpa reply, %d byte pdata\n",
 		    qp_id(qp), params->private_data_len);
Signature · 1 line
2.35.3
maintainer Acked-by on v2 1/1
#1 lore raw
Quoted 42 lines · Guoqing Jiang

We need to clear cep before release state_lock as siw_qp_llp_close and siw_qp_modify->siw_qp_llp_close did.

Otherwise if siw_qp_modify() fails in siw_accept(), the QP's state_lock is released before the error path cleanup. A concurrent ibv_modify_qp() transitioning the QP to ERROR can race in this window:

   siw_accept()                       ibv_modify_qp(ERROR)
   ----------------------             ----------------------
   siw_qp_modify() fails
   up_write(&qp->state_lock)
                                      down_write(&qp->state_lock)
                                      nextstate_from_idle():
				     if (qp->cep)
                                        siw_cep_put(qp->cep) <- frees cep
                                        qp->cep = NULL
   goto error
     cep->qp = NULL                   <- UAF

Clear qp->cep and drop the association reference taken by siw_cep_get(), all under the write lock held from the initial down_write(&qp->state_lock). Thread B therefore sees qp->cep == NULL, skips its own put, and cannot free the cep before siw_accept() is done with it.

---
V2: remove redundant code per Bernard's review

  drivers/infiniband/sw/siw/siw_cm.c | 7 +++++--
  1 file changed, 5 insertions(+), 2 deletions(-)
drivers/infiniband/sw/siw/siw_cm.c +1 -0
index 0245b25e7271..ed49818793dd 100644
@@ -1719,9 +1719,12 @@ int siw_accept(struct iw_cm_id *id, struct iw_cm_conn_param *params)
  			   SIW_QP_ATTR_STATE | SIW_QP_ATTR_LLP_HANDLE |
  				   SIW_QP_ATTR_ORD | SIW_QP_ATTR_IRD |
  				   SIW_QP_ATTR_MPA);
+	if (rv) {

drivers/infiniband/sw/siw/siw_cm.c int siw_accept(struct iw_cm_id *id, struct iw_cm_conn_param *params) · line 1730

Quoted 10 lines
drivers/infiniband/sw/siw/siw_cm.c +4 -2
+		qp->cep = NULL;
+		siw_cep_put(cep);
+		goto error_unlock;
+	}
  	up_write(&qp->state_lock);
-	if (rv)
-		goto error;
 
  	siw_dbg_cep(cep, "[QP %u]: send mpa reply, %d byte pdata\n",
  		    qp_id(qp), params->private_data_len);

Looks good. Thank you, Guoqing.

#2 lore raw
Quoted 55 lines · Bernard Metzler
Quoted 41 lines · Guoqing Jiang

We need to clear cep before release state_lock as siw_qp_llp_close and siw_qp_modify->siw_qp_llp_close did.

Otherwise if siw_qp_modify() fails in siw_accept(), the QP's state_lock is released before the error path cleanup. A concurrent ibv_modify_qp() transitioning the QP to ERROR can race in this window:

   siw_accept()                       ibv_modify_qp(ERROR)
   ----------------------             ----------------------
   siw_qp_modify() fails
   up_write(&qp->state_lock)
                                      down_write(&qp->state_lock)
                                      nextstate_from_idle():
				     if (qp->cep)
                                        siw_cep_put(qp->cep) <- frees cep
                                        qp->cep = NULL
   goto error
     cep->qp = NULL                   <- UAF

Clear qp->cep and drop the association reference taken by siw_cep_get(), all under the write lock held from the initial down_write(&qp->state_lock). Thread B therefore sees qp->cep == NULL, skips its own put, and cannot free the cep before siw_accept() is done with it.

---
V2: remove redundant code per Bernard's review

  drivers/infiniband/sw/siw/siw_cm.c | 7 +++++--
  1 file changed, 5 insertions(+), 2 deletions(-)
drivers/infiniband/sw/siw/siw_cm.c +0 -0
index 0245b25e7271..ed49818793dd 100644
@@ -1719,9 +1719,12 @@ int siw_accept(struct iw_cm_id *id, struct iw_cm_conn_param *params)
  			   SIW_QP_ATTR_STATE | SIW_QP_ATTR_LLP_HANDLE |
  				   SIW_QP_ATTR_ORD | SIW_QP_ATTR_IRD |
  				   SIW_QP_ATTR_MPA);

drivers/infiniband/sw/siw/siw_cm.c int siw_accept(struct iw_cm_id *id, struct iw_cm_conn_param *params) · line 1729

Quoted 10 lines
drivers/infiniband/sw/siw/siw_cm.c +5 -2
+	if (rv) {
+		qp->cep = NULL;
+		siw_cep_put(cep);
+		goto error_unlock;
+	}
  	up_write(&qp->state_lock);
-	if (rv)
-		goto error;
  	siw_dbg_cep(cep, "[QP %u]: send mpa reply, %d byte pdata\n",
  		    qp_id(qp), params->private_data_len);

Looks good. Thank you, Guoqing.

Quoted 1 line

Bernard,

I will take this patch. However, siw_accept() looks wrong to me. It mixes locking, qp->cep handling, and siw_cep_put() in a seemingly inconsistent manner.

For example, here you clear qp->cep and call siw_cep_put() while holding the lock, whereas in the error unwind path you do exactly the opposite.

Be aware that AI-generated suggestions for SIW are complete garbage. If you do not clean up these code paths, your driver will soon end up in a broken and unmaintainable state. The same applies to RXE.

Thanks

maintainer
#3 lore raw
Quoted 64 lines · Leon Romanovsky
Quoted 55 lines · Bernard Metzler
Quoted 41 lines · Guoqing Jiang

We need to clear cep before release state_lock as siw_qp_llp_close and siw_qp_modify->siw_qp_llp_close did.

Otherwise if siw_qp_modify() fails in siw_accept(), the QP's state_lock is released before the error path cleanup. A concurrent ibv_modify_qp() transitioning the QP to ERROR can race in this window:

    siw_accept()                       ibv_modify_qp(ERROR)
    ----------------------             ----------------------
    siw_qp_modify() fails
    up_write(&qp->state_lock)
                                       down_write(&qp->state_lock)
                                       nextstate_from_idle():
				     if (qp->cep)
                                         siw_cep_put(qp->cep) <- frees cep
                                         qp->cep = NULL
    goto error
      cep->qp = NULL                   <- UAF

Clear qp->cep and drop the association reference taken by siw_cep_get(), all under the write lock held from the initial down_write(&qp->state_lock). Thread B therefore sees qp->cep == NULL, skips its own put, and cannot free the cep before siw_accept() is done with it.

---
V2: remove redundant code per Bernard's review

   drivers/infiniband/sw/siw/siw_cm.c | 7 +++++--
   1 file changed, 5 insertions(+), 2 deletions(-)
drivers/infiniband/sw/siw/siw_cm.c +0 -0
index 0245b25e7271..ed49818793dd 100644
@@ -1719,9 +1719,12 @@ int siw_accept(struct iw_cm_id *id, struct iw_cm_conn_param *params)
   			   SIW_QP_ATTR_STATE | SIW_QP_ATTR_LLP_HANDLE |
   				   SIW_QP_ATTR_ORD | SIW_QP_ATTR_IRD |
   				   SIW_QP_ATTR_MPA);

drivers/infiniband/sw/siw/siw_cm.c int siw_accept(struct iw_cm_id *id, struct iw_cm_conn_param *params) · line 1729

Quoted 10 lines
drivers/infiniband/sw/siw/siw_cm.c +5 -2
+	if (rv) {
+		qp->cep = NULL;
+		siw_cep_put(cep);
+		goto error_unlock;
+	}
   	up_write(&qp->state_lock);
-	if (rv)
-		goto error;
   	siw_dbg_cep(cep, "[QP %u]: send mpa reply, %d byte pdata\n",
   		    qp_id(qp), params->private_data_len);

Looks good. Thank you, Guoqing.

Quoted 1 line

Bernard,

I will take this patch. However, siw_accept() looks wrong to me. It mixes locking, qp->cep handling, and siw_cep_put() in a seemingly inconsistent manner.

Quoted 8 lines

For example, here you clear qp->cep and call siw_cep_put() while holding the lock, whereas in the error unwind path you do exactly the opposite.

Be aware that AI-generated suggestions for SIW are complete garbage. If you do not clean up these code paths, your driver will soon end up in a broken and unmaintainable state. The same applies to RXE.

Thanks

That's probably an accurate assessment. Thanks for your candid remarks. I will try to clean up some obvious things.

I share you opinion on those current AI generated remarks.

Thnk you,
Bernard.

#4 lore raw
In reply to Leon Romanovsky
Quoted 64 lines · Leon Romanovsky
Quoted 55 lines · Bernard Metzler
Quoted 41 lines · Guoqing Jiang

We need to clear cep before release state_lock as siw_qp_llp_close and siw_qp_modify->siw_qp_llp_close did.

Otherwise if siw_qp_modify() fails in siw_accept(), the QP's state_lock is released before the error path cleanup. A concurrent ibv_modify_qp() transitioning the QP to ERROR can race in this window:

    siw_accept()                       ibv_modify_qp(ERROR)
    ----------------------             ----------------------
    siw_qp_modify() fails
    up_write(&qp->state_lock)
                                       down_write(&qp->state_lock)
                                       nextstate_from_idle():
				     if (qp->cep)
                                         siw_cep_put(qp->cep) <- frees cep
                                         qp->cep = NULL
    goto error
      cep->qp = NULL                   <- UAF

Clear qp->cep and drop the association reference taken by siw_cep_get(), all under the write lock held from the initial down_write(&qp->state_lock). Thread B therefore sees qp->cep == NULL, skips its own put, and cannot free the cep before siw_accept() is done with it.

---
V2: remove redundant code per Bernard's review

   drivers/infiniband/sw/siw/siw_cm.c | 7 +++++--
   1 file changed, 5 insertions(+), 2 deletions(-)
drivers/infiniband/sw/siw/siw_cm.c +0 -0
index 0245b25e7271..ed49818793dd 100644
@@ -1719,9 +1719,12 @@ int siw_accept(struct iw_cm_id *id, struct iw_cm_conn_param *params)
   			   SIW_QP_ATTR_STATE | SIW_QP_ATTR_LLP_HANDLE |
   				   SIW_QP_ATTR_ORD | SIW_QP_ATTR_IRD |
   				   SIW_QP_ATTR_MPA);

drivers/infiniband/sw/siw/siw_cm.c int siw_accept(struct iw_cm_id *id, struct iw_cm_conn_param *params) · line 1729

Quoted 10 lines
drivers/infiniband/sw/siw/siw_cm.c +5 -2
+	if (rv) {
+		qp->cep = NULL;
+		siw_cep_put(cep);
+		goto error_unlock;
+	}
   	up_write(&qp->state_lock);
-	if (rv)
-		goto error;
   	siw_dbg_cep(cep, "[QP %u]: send mpa reply, %d byte pdata\n",
   		    qp_id(qp), params->private_data_len);

Looks good. Thank you, Guoqing.

Quoted 1 line

Bernard,

I will take this patch. However, siw_accept() looks wrong to me. It mixes locking, qp->cep handling, and siw_cep_put() in a seemingly inconsistent manner.

Quoted 6 lines

For example, here you clear qp->cep and call siw_cep_put() while holding the lock, whereas in the error unwind path you do exactly the opposite.

Be aware that AI-generated suggestions for SIW are complete garbage. If you do not clean up these code paths, your driver will soon end up in a broken and unmaintainable state. The same applies to RXE.

Agree with you. Thanks a lot.

Zhu Yanjun

Quoted 2 lines

Thanks

#5 lore raw
In reply to Guoqing Jiang
Quoted 8 lines · Guoqing Jiang

We need to clear cep before release state_lock as siw_qp_llp_close and siw_qp_modify->siw_qp_llp_close did.

Otherwise if siw_qp_modify() fails in siw_accept(), the QP's state_lock is released before the error path cleanup. A concurrent ibv_modify_qp() transitioning the QP to ERROR can race in this window:

[...]

Applied, thanks!

[1/1] RDMA/siw: Clear association under lock if siw_qp_modify fails in siw_accept

      https://git.kernel.org/rdma/rdma/c/32cd87f54dd107

Best regards,

Signature · 1 line
Leon Romanovsky <[email protected]>