Skip to contentSkip to conversation

RDMA/siw: Fix CEP use-after-free in siw_connect()

PATCH In review Shuangpeng Bai 2 participants 2 messages Aug 16 – Aug 21

Subsystems (3)
lore
Share & tools
lore
#0 lore raw

PATCH 1/1 RDMA/siw: Fix CEP use-after-free in siw_connect()

siw_connect() publishes a newly allocated CEP through qp->cep and gives the QP an association reference.

If connection setup fails while the QP is being destroyed, QP teardown can clear qp->cep and drop that reference first. The error path nevertheless drops what it assumes is the association reference. This consumes the local allocation reference and frees the CEP before the error path updates cep->state and releases cep->in_use, causing a use-after-free.

Serialize CEP association and error-side detachment with qp->state_lock. Only drop the association reference when qp->cep still points to the CEP being cleaned up, leaving the local reference valid until cleanup is complete.

Fixes 6c52fdc244b5 ("rdma/siw: connection management")
---
 drivers/infiniband/sw/siw/siw_cm.c | 15 ++++++++++-----
 1 file changed, 10 insertions(+), 5 deletions(-)
drivers/infiniband/sw/siw/siw_cm.c +10 -5
index 87c79527ac09..7b32158ea49f 100644
@@ -1467,11 +1467,13 @@ int siw_connect(struct iw_cm_id *id, struct iw_cm_conn_param *params)
 	siw_cep_set_inuse(cep);
 
 	/* Associate QP with CEP */
+	down_write(&qp->state_lock);
 	siw_cep_get(cep);
 	qp->cep = cep;
 
 	/* siw_qp_get(qp) already done by QP lookup */
 	cep->qp = qp;
+	up_write(&qp->state_lock);
 
 	id->add_ref(id);
 	cep->cm_id = id;
@@ -1564,16 +1566,19 @@ int siw_connect(struct iw_cm_id *id, struct iw_cm_conn_param *params)
 		siw_socket_disassoc(s);
 		sock_release(s);
 
-		cep->qp = NULL;
-
 		cep->cm_id = NULL;
 		id->rem_ref(id);
 
-		qp->cep = NULL;
-		siw_cep_put(cep);
-
 		cep->state = SIW_EPSTATE_CLOSED;
 
+		down_write(&qp->state_lock);
+		cep->qp = NULL;
+		if (qp->cep == cep) {
+			qp->cep = NULL;
+			siw_cep_put(cep);
+		}
+		up_write(&qp->state_lock);
+
 		siw_cep_set_free_and_put(cep);
 
 	} else if (s) {
Signature · 1 line
2.43.0
maintainer
#1 lore raw
Quoted 10 lines · Shuangpeng Bai

siw_connect() publishes a newly allocated CEP through qp->cep and gives the QP an association reference.

If connection setup fails while the QP is being destroyed, QP teardown can clear qp->cep and drop that reference first. The error path nevertheless drops what it assumes is the association reference. This consumes the local allocation reference and frees the CEP before the error path updates cep->state and releases cep->in_use, causing a use-after-free.

Was that use-after-free encountered?

Can we see a stack trace?

Quoted 17 lines

Serialize CEP association and error-side detachment with qp->state_lock. Only drop the association reference when qp->cep still points to the CEP being cleaned up, leaving the local reference valid until cleanup is complete.

Fixes 6c52fdc244b5 ("rdma/siw: connection management")
---
  drivers/infiniband/sw/siw/siw_cm.c | 15 ++++++++++-----
  1 file changed, 10 insertions(+), 5 deletions(-)
drivers/infiniband/sw/siw/siw_cm.c +0 -0
index 87c79527ac09..7b32158ea49f 100644

drivers/infiniband/sw/siw/siw_cm.c int siw_connect(struct iw_cm_id *id, struct iw_cm_conn_param *params) · line 1470

Quoted 4 lines
drivers/infiniband/sw/siw/siw_cm.c +1 -0
@@ -1467,11 +1467,13 @@ int siw_connect(struct iw_cm_id *id, struct iw_cm_conn_param *params)
  	siw_cep_set_inuse(cep);
 
  	/* Associate QP with CEP */
+	down_write(&qp->state_lock);

What could destroy the QP at this point, so that we need a lock? The RDMA core should serialize any user initiated QP destroy during the connect.

Quoted 13 lines
drivers/infiniband/sw/siw/siw_cm.c +1 -0
  	siw_cep_get(cep);
  	qp->cep = cep;
 
  	/* siw_qp_get(qp) already done by QP lookup */
  	cep->qp = qp;
+	up_write(&qp->state_lock);
 
  	id->add_ref(id);
  	cep->cm_id = id;
@@ -1564,16 +1566,19 @@ int siw_connect(struct iw_cm_id *id, struct iw_cm_conn_param *params)
  		siw_socket_disassoc(s);
  		sock_release(s);
 

drivers/infiniband/sw/siw/siw_cm.c int siw_connect(struct iw_cm_id *id, struct iw_cm_conn_param *params) · line 1573

Quoted 10 lines
drivers/infiniband/sw/siw/siw_cm.c +0 -5
-		cep->qp = NULL;
-
  		cep->cm_id = NULL;
  		id->rem_ref(id);
 
-		qp->cep = NULL;
-		siw_cep_put(cep);
-
  		cep->state = SIW_EPSTATE_CLOSED;
 

What can destroy the QP at this point?

The only possibility I see is a connection drop from peer side, which races with the current connect() processing. Having a lock at the cep is sufficient to serialize that.
We hold the cep lock via siw_cep_set_inuse(), which releasedonly the th every end of the connect via siw_cep_set_free() (success), or siw_cep_set_free_and_put() (failure.

Quoted 11 lines
drivers/infiniband/sw/siw/siw_cm.c +8 -0
+		down_write(&qp->state_lock);
+		cep->qp = NULL;
+		if (qp->cep == cep) {
+			qp->cep = NULL;
+			siw_cep_put(cep);
+		}
+		up_write(&qp->state_lock);
+
  		siw_cep_set_free_and_put(cep);
 
  	} else if (s) {