RDMA/siw: Clear association under lock if siw_qp_modify fails in siw_accept
Subsystems (2)
PATCH 1/1 RDMA/siw: Clear association under lock if siw_qp_modify fails in siw_accept
We need to clear qp and cep before release state_lock as siw_qp_llp_close and siw_qp_modify->siw_qp_llp_close did.
Otherwise if siw_qp_modify() fails in siw_accept(), the QP's state_lock is released before the error path cleanup. A concurrent ibv_modify_qp() transitioning the QP to ERROR can race in this window:
siw_accept() ibv_modify_qp(ERROR)
---------------------- ----------------------
siw_qp_modify() fails
up_write(&qp->state_lock)
down_write(&qp->state_lock)
nextstate_from_idle():
if (qp->cep)
siw_cep_put(qp->cep) <- frees cep
qp->cep = NULL
goto error
cep->qp = NULL <- UAFClear qp->cep and cep->qp, and drop the association reference taken by siw_cep_get(), all under the write lock held from the initial down_write(&qp->state_lock). Thread B therefore sees qp->cep == NULL, skips its own put, and cannot free the cep before siw_accept() is done with it.
--- drivers/infiniband/sw/siw/siw_cm.c | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-)
drivers/infiniband/sw/siw/siw_cm.c +6 -2
@@ -1719,9 +1719,13 @@ int siw_accept(struct iw_cm_id *id, struct iw_cm_conn_param *params)SIW_QP_ATTR_STATE |SIW_QP_ATTR_LLP_HANDLE |SIW_QP_ATTR_ORD |SIW_QP_ATTR_IRD | SIW_QP_ATTR_MPA); +if (rv) { + cep->qp =NULL ; + qp->cep =NULL ; +siw_cep_put (cep); + goto error_unlock; + }up_write (&qp->state_lock); -if (rv) - goto error;siw_dbg_cep (cep,"[QP %u]: send mpa reply, %d byte pdata\n" ,qp_id (qp), params->private_data_len);
Signature · 1 line
2.35.3
Bernard Metzler
maintainerQuoted 37 lines · Guoqing Jiang
We need to clear qp and cep before release
state_lockassiw_qp_llp_closeoandsiw_qp_modify->siw_qp_llp_closedid.Otherwise if
siw_qp_modify()fails insiw_accept(), the QP'sstate_lockis released before the error path cleanup. A concurrentibv_modify_qp()transitioning the QP to ERROR can race in this window:siw_accept() ibv_modify_qp(ERROR) ---------------------- ---------------------- siw_qp_modify() fails up_write(&qp->state_lock) down_write(&qp->state_lock) nextstate_from_idle(): if (qp->cep) siw_cep_put(qp->cep) <- frees cep qp->cep = NULL goto error cep->qp = NULL <- UAFClear
qp->cepandcep->qp, and drop the association reference taken bysiw_cep_get(), all under the write lock held from the initialdown_write(&qp->state_lock). Thread B therefore seesqp->cep== NULL, skips its own put, and cannot free the cep beforesiw_accept()is done with it.Reported-by Shuangpeng Bai <[email protected]>Link https://lore.kernel.org/linux-rdma/[email protected]/T/#m5876c1ff2de8686a9a1173b8f1aa0ff5363a785c open hereSigned-off-by Guoqing Jiang <[email protected]>--- drivers/infiniband/sw/siw/siw_cm.c | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-)drivers/infiniband/sw/siw/siw_cm.c +0
-0
drivers/infiniband/sw/siw/siw_cm.c int siw_accept(struct iw_cm_id *id, struct iw_cm_conn_param *params) · line 1723
Quoted 5 lines
drivers/infiniband/sw/siw/siw_cm.c +2
-0@@ -1719,9 +1719,13 @@ int siw_accept(struct iw_cm_id *id, struct iw_cm_conn_param *params)SIW_QP_ATTR_STATE |SIW_QP_ATTR_LLP_HANDLE |SIW_QP_ATTR_ORD |SIW_QP_ATTR_IRD | SIW_QP_ATTR_MPA); +if (rv) { + cep->qp =NULL ;
This can better be omitted since done in error_unlock path anyway and is redundant otherwise. It would also better retain the logic of calling siw_qp_put(qp) right after clearing cep's reference to the qp, as done in error path.
Thank you!
Bernard.> + qp->cep = NULL;
Quoted 9 lines
drivers/infiniband/sw/siw/siw_cm.c +3
-2+siw_cep_put (cep); + goto error_unlock; + }up_write (&qp->state_lock); -if (rv) - goto error;siw_dbg_cep (cep,"[QP %u]: send mpa reply, %d byte pdata\n" ,qp_id (qp), params->private_data_len);
Hi Bernard,
Quoted 33 lines · Bernard Metzler
Quoted 32 lines · Guoqing Jiang
We need to clear qp and cep before release
state_lockassiw_qp_llp_closeoandsiw_qp_modify->siw_qp_llp_closedid.Otherwise if
siw_qp_modify()fails insiw_accept(), the QP'sstate_lockis released before the error path cleanup. A concurrentibv_modify_qp()transitioning the QP to ERROR can race in this window:siw_accept()
ibv_modify_qp(ERROR)---------------------- ----------------------siw_qp_modify()fails
up_write(&qp->state_lock)down_write(&qp->state_lock)
nextstate_from_idle():
if (qp->cep)
siw_cep_put(qp->cep)<- frees cepqp->cep= NULL
goto error
cep->qp= NULL <- UAFClear
qp->cepandcep->qp, and drop the association reference taken bysiw_cep_get(), all under the write lock held from the initialdown_write(&qp->state_lock). Thread B therefore seesqp->cep== NULL, skips its own put, and cannot free the cep beforesiw_accept()is done with it.Reported-by Shuangpeng Bai <[email protected]>Link https://lore.kernel.org/linux-rdma/[email protected]/T/#m5876c1ff2de8686a9a1173b8f1aa0ff5363a785c open hereSigned-off-by Guoqing Jiang <[email protected]>---
drivers/infiniband/sw/siw/siw_cm.c| 8 ++++++--1 file changed, 6 insertions(+), 2 deletions(-)
Quoted 13 lines
Quoted 11 lines
drivers/infiniband/sw/siw/siw_cm.c +0
-0@@ -1719,9 +1719,13 @@ int siw_accept(struct iw_cm_id *id, struct iw_cm_conn_param *params)
SIW_QP_ATTR_STATE|SIW_QP_ATTR_LLP_HANDLE|SIW_QP_ATTR_ORD|SIW_QP_ATTR_IRD|SIW_QP_ATTR_MPA);
+ if (rv) {
+cep->qp= NULL;This can better be omitted since done in
error_unlockpath anyway and is redundant otherwise.
Right, thanks for the review! And probably call siw_cep_put before set qp->cep to NULL as other places did.
Quoted 2 lines
It would also better retain the logic of calling
siw_qp_put(qp)right after clearing cep's reference to the qp, as done in error path.
I suppose this change could be okay.
@@ -1719,9 +1719,12 @@ int siw_accept(struct iw_cm_id *id, struct iw_cm_conn_param *params) SIW_QP_ATTR_STATE | SIW_QP_ATTR_LLP_HANDLE | SIW_QP_ATTR_ORD | SIW_QP_ATTR_IRD | SIW_QP_ATTR_MPA);
+ if (rv) {
+ siw_cep_put(cep);
+ qp->cep = NULL;
+ goto error_unlock;
+ }
up_write(&qp->state_lock);
- if (rv)
- goto error;
I am wondering if there is another use-after-free case in siw_qp_cm_drop. Because the beginning of siw_qp_cm_drop borrows the qp->cep pointer without holding a reference. If another thread put the last reference of cep, then siw_qp_cm_drop wakes and deferences cep which has been freed.
Quoted 11 lines
Thank you!
Bernard.> +qp->cep= NULL;Quoted 8 lines
+
siw_cep_put(cep);
+ gotoerror_unlock;
+ }
up_write(&qp->state_lock);
- if (rv)
- goto error;
siw_dbg_cep(cep, "[QP %u]: send mpa reply, %d byte pdata\n", qp_id(qp), params->private_data_len);
Thanks,
Guoqing
Bernard Metzler
maintainerQuoted 60 lines · Guoqing Jiang
Hi Bernard,
Quoted 33 lines · Bernard Metzler
Quoted 32 lines · Guoqing Jiang
We need to clear qp and cep before release
state_lockassiw_qp_llp_closeoandsiw_qp_modify->siw_qp_llp_closedid.Otherwise if
siw_qp_modify()fails insiw_accept(), the QP'sstate_lockis released before the error path cleanup. A concurrentibv_modify_qp()transitioning the QP to ERROR can race in this window:siw_accept()
ibv_modify_qp(ERROR)---------------------- ----------------------siw_qp_modify()fails
up_write(&qp->state_lock)down_write(&qp->state_lock)
nextstate_from_idle():
if (qp->cep)
siw_cep_put(qp->cep)<- frees cepqp->cep= NULL
goto error
cep->qp= NULL <- UAFClear
qp->cepandcep->qp, and drop the association reference taken bysiw_cep_get(), all under the write lock held from the initialdown_write(&qp->state_lock). Thread B therefore seesqp->cep== NULL, skips its own put, and cannot free the cep beforesiw_accept()is done with it.Reported-by Shuangpeng Bai <[email protected]>Link https://lore.kernel.org/linux-rdma/[email protected]/T/#m5876c1ff2de8686a9a1173b8f1aa0ff5363a785c open hereSigned-off-by Guoqing Jiang <[email protected]>---
drivers/infiniband/sw/siw/siw_cm.c| 8 ++++++--1 file changed, 6 insertions(+), 2 deletions(-)Quoted 12 lines
Quoted 10 lines
drivers/infiniband/sw/siw/siw_cm.c +0
-0@@ -1719,9 +1719,13 @@ int siw_accept(struct iw_cm_id *id, struct iw_cm_conn_param *params)
SIW_QP_ATTR_STATE|SIW_QP_ATTR_LLP_HANDLE|SIW_QP_ATTR_ORD|SIW_QP_ATTR_IRD|SIW_QP_ATTR_MPA);
+ if (rv) {
+cep->qp= NULL;This can better be omitted since done in
error_unlockpath anyway and is redundant otherwise.Right, thanks for the review! And probably call
siw_cep_putbefore setqp->cepto NULL as other places did.Quoted 2 lines
It would also better retain the logic of calling
siw_qp_put(qp)right after clearing cep's reference to the qp, as done in error path.I suppose this change could be okay.
drivers/infiniband/sw/siw/siw_cm.c +0
-0@@ -1719,9 +1719,12 @@ int siw_accept(struct iw_cm_id *id, struct iw_cm_conn_param *params) SIW_QP_ATTR_STATE | SIW_QP_ATTR_LLP_HANDLE | SIW_QP_ATTR_ORD | SIW_QP_ATTR_IRD |
drivers/infiniband/sw/siw/siw_cm.c int siw_accept(struct iw_cm_id *id, struct iw_cm_conn_param *params) · line 1728
Quoted 10 lines
drivers/infiniband/sw/siw/siw_cm.c +5
-2SIW_QP_ATTR_MPA); + if (rv) { + siw_cep_put(cep); + qp->cep = NULL; + goto error_unlock; + } up_write(&qp->state_lock); - if (rv) - goto error;
That looks good, while I think we should always NULL a pointer to something before releasing a hold/reference to it. Putting the reference implies a memory barrier and so it's guaranteed others see that NULL as well. You are right - there are places where this is not done as it should.
Quoted 5 lines
I am wondering if there is another use-after-free case in
siw_qp_cm_drop. Because the beginning ofsiw_qp_cm_dropborrows theqp->ceppointer without holding a reference. If another thread put the last reference of cep, thensiw_qp_cm_dropwakes and deferences cep which has been freed.
Let's put this in an extra thread. You may have a point ;)
Thanks,
Bernard.
Quoted 14 lines
Quoted 11 lines
Thank you!
Bernard.> +qp->cep= NULL;Quoted 8 lines
+
siw_cep_put(cep);
+ gotoerror_unlock;
+ }
up_write(&qp->state_lock);
- if (rv)
- goto error;
siw_dbg_cep(cep, "[QP %u]: send mpa reply, %d byte pdata\n", qp_id(qp), params->private_data_len);Thanks,
Guoqing
Quoted 68 lines · Bernard Metzler
Quoted 67 lines · Guoqing Jiang
Hi Bernard,
Quoted 33 lines · Bernard Metzler
Quoted 32 lines · Guoqing Jiang
We need to clear qp and cep before release
state_lockassiw_qp_llp_closeoandsiw_qp_modify->siw_qp_llp_closedid.Otherwise if
siw_qp_modify()fails insiw_accept(), the QP'sstate_lockis released before the error path cleanup. A concurrentibv_modify_qp()transitioning the QP to ERROR can race in this window:siw_accept()
ibv_modify_qp(ERROR)---------------------- ----------------------siw_qp_modify()fails
up_write(&qp->state_lock)down_write(&qp->state_lock)
nextstate_from_idle():
if (qp->cep)siw_cep_put(qp->cep)<- frees cep
qp->cep= NULL goto error
cep->qp= NULL <- UAFClear
qp->cepandcep->qp, and drop the association reference taken bysiw_cep_get(), all under the write lock held from the initialdown_write(&qp->state_lock). Thread B therefore seesqp->cep== NULL, skips its own put, and cannot free the cep beforesiw_accept()is done with it.Reported-by Shuangpeng Bai <[email protected]>Link https://lore.kernel.org/linux-rdma/[email protected]/T/#m5876c1ff2de8686a9a1173b8f1aa0ff5363a785c open hereSigned-off-by Guoqing Jiang <[email protected]>---
drivers/infiniband/sw/siw/siw_cm.c| 8 ++++++--1 file changed, 6 insertions(+), 2 deletions(-)Quoted 13 lines
Quoted 11 lines
drivers/infiniband/sw/siw/siw_cm.c +0
-0@@ -1719,9 +1719,13 @@ int siw_accept(struct iw_cm_id *id, struct iw_cm_conn_param *params)
SIW_QP_ATTR_STATE|SIW_QP_ATTR_LLP_HANDLE|SIW_QP_ATTR_ORD|SIW_QP_ATTR_IRD|SIW_QP_ATTR_MPA);
+ if (rv) {
+cep->qp= NULL;This can better be omitted since done in
error_unlockpath anyway and is redundant otherwise.Right, thanks for the review! And probably call
siw_cep_putbefore setqp->cepto NULL as other places did.Quoted 2 lines
It would also better retain the logic of calling
siw_qp_put(qp)right after clearing cep's reference to the qp, as done in error path.I suppose this change could be okay.
(quoted diff) +0
-0@@ -1719,9 +1719,12 @@ int siw_accept(struct iw_cm_id *id, struct iw_cm_conn_param *params)
SIW_QP_ATTR_STATE|SIW_QP_ATTR_LLP_HANDLE|SIW_QP_ATTR_ORD|SIW_QP_ATTR_IRD|SIW_QP_ATTR_MPA);
+ if (rv) {
+siw_cep_put(cep);
+qp->cep= NULL;
+ gotoerror_unlock;
Quoted 8 lines
Quoted 5 lines
+ }
up_write(&qp->state_lock);
- if (rv)
- goto error;That looks good, while I think we should always NULL a pointer to something before releasing a hold/reference to it. Putting the reference implies a memory barrier and so it's guaranteed others see that NULL as well.
Hmm, good point.
Quoted 8 lines
You are right - there are places where this is not done as it should.
Quoted 5 lines
I am wondering if there is another use-after-free case in
siw_qp_cm_drop. Because the beginning ofsiw_qp_cm_dropborrows theqp->ceppointer without holding a reference. If another thread put the last reference of cep, thensiw_qp_cm_dropwakes and deferences cep which has been freed.Let's put this in an extra thread. You may have a point ;)
I will dig a little bit more before start a new thread.
Thanks,
Guoqing