IB/qib,staging/rdma/hfi1: Dual lock pre-reqs
Subsystems (2)
cover IB/qib,staging/rdma/hfi1: Dual lock pre-reqs
This patch series includes some prerequisite patches in order to add a dual lock on the send side. It also brings in a patch based on one submitted by Hari.
Series applies on the previously submitted "Update retry timer" for hfi1 and can be seen in context at: https://github.com/ddalessa/kernel/tree/for-4.6
---
Hari Prasath Gujulan Elango (1):
IB/qib,staging/rdma/hfi1: use setup_timer api
Mike Marciniszyn (5):
staging/rdma/hfi1: remove duplicate timeout print staging/rdma/hfi1: add s_retry to diagnostics staging/rdma/hfi1: Insure last cursor is updated prior to complete IB/qib: Insure last cursor is updated prior to complete IB/rdmavt: remove unused qp field
drivers/infiniband/hw/qib/qib_rc.c | 20 ++++++++++++++++---- drivers/infiniband/hw/qib/qib_ruc.c | 12 +++++++----- drivers/infiniband/hw/qib/qib_verbs.c | 4 +--- drivers/staging/rdma/hfi1/qp.c | 4 ++-- drivers/staging/rdma/hfi1/rc.c | 20 ++++++++++++++++---- drivers/staging/rdma/hfi1/ruc.c | 12 +++++++----- drivers/staging/rdma/hfi1/verbs.c | 4 +--- include/rdma/rdmavt_qp.h | 1 - 8 files changed, 50 insertions(+), 27 deletions(-)
Signature · 5 lines
-Denny -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
A message that is not in the archive (sent off-list or not yet received): 20160204190024.8378.5080.stgit-9QXIwq+3FY+1XWohqUldA0EOCMrvLtNR@public.gmane.org
PATCH 1/6 staging/rdma/hfi1: remove duplicate timeout print
From: Mike Marciniszyn <[email protected]>
The qp->timeout field is duplicated in the
seqfile print.
Remove it.
--- drivers/staging/rdma/hfi1/qp.c | 3 +-- 1 files changed, 1 insertions(+), 2 deletions(-)
drivers/staging/rdma/hfi1/qp.c +1 -2
@@ -547,7 +547,7 @@ void qp_iter_print(struct seq_file *s, struct qp_iter *iter) sde =qp_to_sdma_engine (qp, priv->s_sc); wqe =rvt_get_swqe_ptr (qp, qp->s_last); seq_printf(s, - "N %d %s QP%u R %u %s %u %u %u f=%x %u %u %u %u %u PSN %x %x %x %x %x (%u %u %u %u %u %u) QP%u LID %x SL %u MTU %d %u %u %u SDE %p,%u\n", + "N %d %s QP%u R %u %s %u %u %u f=%x %u %u %u %u %u PSN %x %x %x %x %x (%u %u %u %u %u %u) QP%u LID %x SL %u MTU %u %u %u SDE %p,%u\n", iter->n, qp_idle(qp) ? "I" : "B", qp->ibqp.qp_num,
@@ -572,7 +572,6 @@ void qp_iter_print(struct seq_file *s, struct qp_iter *iter) qp->remote_ah_attr.sl, qp->pmtu, qp->s_retry_cnt, - qp->timeout, qp->s_rnr_retry_cnt, sde, sde ? sde->this_idx : 0);
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
PATCH 2/6 staging/rdma/hfi1: add s_retry to diagnostics
In reply to a message not in the archiveFrom: Mike Marciniszyn <[email protected]>
This is needed to debug ULP issues with getting retry attributes correctly specified.
--- drivers/staging/rdma/hfi1/qp.c | 3 ++- 1 files changed, 2 insertions(+), 1 deletions(-)
drivers/staging/rdma/hfi1/qp.c +2 -1
@@ -547,7 +547,7 @@ void qp_iter_print(struct seq_file *s, struct qp_iter *iter) sde =qp_to_sdma_engine (qp, priv->s_sc); wqe =rvt_get_swqe_ptr (qp, qp->s_last); seq_printf(s, - "N %d %s QP%u R %u %s %u %u %u f=%x %u %u %u %u %u PSN %x %x %x %x %x (%u %u %u %u %u %u) QP%u LID %x SL %u MTU %u %u %u SDE %p,%u\n", + "N %d %s QP%u R %u %s %u %u %u f=%x %u %u %u %u %u PSN %x %x %x %x %x (%u %u %u %u %u %u) QP%u LID %x SL %u MTU %u %u %u %u SDE %p,%u\n", iter->n, qp_idle(qp) ? "I" : "B", qp->ibqp.qp_num,
@@ -571,6 +571,7 @@ void qp_iter_print(struct seq_file *s, struct qp_iter *iter) qp->remote_ah_attr.dlid, qp->remote_ah_attr.sl, qp->pmtu, + qp->s_retry, qp->s_retry_cnt, qp->s_rnr_retry_cnt, sde,
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
PATCH 3/6 staging/rdma/hfi1: Insure last cursor is updated prior to complete
In reply to a message not in the archiveFrom: Mike Marciniszyn <[email protected]>
This patch is a prerequisite for adding a separate lock for post send.
The timing of updating s_last needs to be before returning any send completion to avoid a race between a poll cq seeing a completion and the post send checking for a full queue.
--- drivers/staging/rdma/hfi1/rc.c | 20 ++++++++++++++++---- drivers/staging/rdma/hfi1/ruc.c | 12 +++++++----- 2 files changed, 23 insertions(+), 9 deletions(-)
drivers/staging/rdma/hfi1/rc.c +16 -4
@@ -1123,10 +1123,18 @@ void hfi1_rc_send_complete(struct rvt_qp *qp, struct hfi1_ib_header *hdr)hfi1_add_retry_timer (qp);while (qp->s_last != qp->s_acked) { +u32 s_last; + wqe =rvt_get_swqe_ptr (qp, qp->s_last);if (cmp_psn (wqe->lpsn, qp->s_sending_psn) >=0 &&cmp_psn (qp->s_sending_psn, qp->s_sending_hpsn) <=0 )break ; + s_last = qp->s_last; +if (++s_last >= qp->s_size) + s_last =0 ; + qp->s_last = s_last; +/* see post_send() */ +barrier ();for (i =0 ; i < wqe->wr.num_sge; i++) {struct rvt_sge *sge = &wqe->sg_list[i];
@@ -1143,8 +1151,6 @@ void hfi1_rc_send_complete(struct rvt_qp *qp, struct hfi1_ib_header *hdr) wc.qp = &qp->ibqp;rvt_cq_enter (ibcq_to_rvtcq (qp->ibqp.send_cq), &wc,0 ); } -if (++qp->s_last >= qp->s_size) - qp->s_last =0 ; }/* * If we were waiting for sends to complete before re-sending,
@@ -1184,11 +1190,19 @@ static struct rvt_swqe *do_rc_completion(struct rvt_qp *qp,*/ if (cmp_psn (wqe->lpsn, qp->s_sending_psn) <0 ||cmp_psn (qp->s_sending_psn, qp->s_sending_hpsn) >0 ) { +u32 s_last; +for (i =0 ; i < wqe->wr.num_sge; i++) {struct rvt_sge *sge = &wqe->sg_list[i];rvt_put_mr (sge->mr); } + s_last = qp->s_last; +if (++s_last >= qp->s_size) + s_last =0 ; + qp->s_last = s_last; +/* see post_send() */ +barrier ();/* Post a send completion queue entry if requested. */ if (!(qp->s_flags &RVT_S_SIGNAL_REQ_WR ) || (wqe->wr.send_flags &IB_SEND_SIGNALED )) {
@@ -1200,8 +1214,6 @@ static struct rvt_swqe *do_rc_completion(struct rvt_qp *qp, wc.qp = &qp->ibqp;rvt_cq_enter (ibcq_to_rvtcq (qp->ibqp.send_cq), &wc,0 ); } -if (++qp->s_last >= qp->s_size) - qp->s_last =0 ; }else {struct hfi1_pportdata *ppd =ppd_from_ibp (ibp);
drivers/staging/rdma/hfi1/ruc.c +7 -5
@@ -921,6 +921,13 @@ void hfi1_send_complete(struct rvt_qp *qp, struct rvt_swqe *wqe,if (!(ib_rvt_state_ops[qp->state] &RVT_PROCESS_OR_FLUSH_SEND ))return ; + last = qp->s_last; + old_last = last; +if (++last >= qp->s_size) + last =0 ; + qp->s_last = last; +/* See post_send() */ +barrier ();for (i =0 ; i < wqe->wr.num_sge; i++) {struct rvt_sge *sge = &wqe->sg_list[i];
@@ -948,11 +955,6 @@ void hfi1_send_complete(struct rvt_qp *qp, struct rvt_swqe *wqe, status != IB_WC_SUCCESS); } - last = qp->s_last; - old_last = last; -if (++last >= qp->s_size) - last =0 ; - qp->s_last = last;if (qp->s_acked == old_last) qp->s_acked = last;if (qp->s_cur == old_last)
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
PATCH 4/6 IB/qib: Insure last cursor is updated prior to complete
In reply to a message not in the archiveFrom: Mike Marciniszyn <[email protected]>
This patch is a prerequisite for adding a separate lock for post send.
The timing of updating s_last needs to be before returning any send completion to avoid a race between a poll cq seeing a completion and the post send checking for a full queue.
--- drivers/infiniband/hw/qib/qib_rc.c | 20 ++++++++++++++++---- drivers/infiniband/hw/qib/qib_ruc.c | 12 +++++++----- 2 files changed, 23 insertions(+), 9 deletions(-)
drivers/infiniband/hw/qib/qib_rc.c +16 -4
@@ -1008,10 +1008,18 @@ void qib_rc_send_complete(struct rvt_qp *qp, struct qib_ib_header *hdr)start_timer (qp);while (qp->s_last != qp->s_acked) { +u32 s_last; + wqe =rvt_get_swqe_ptr (qp, qp->s_last);if (qib_cmp24 (wqe->lpsn, qp->s_sending_psn) >=0 &&qib_cmp24 (qp->s_sending_psn, qp->s_sending_hpsn) <=0 )break ; + s_last = qp->s_last; +if (++s_last >= qp->s_size) + s_last =0 ; + qp->s_last = s_last; +/* see post_send() */ +barrier ();for (i =0 ; i < wqe->wr.num_sge; i++) {struct rvt_sge *sge = &wqe->sg_list[i];
@@ -1028,8 +1036,6 @@ void qib_rc_send_complete(struct rvt_qp *qp, struct qib_ib_header *hdr) wc.qp = &qp->ibqp;rvt_cq_enter (ibcq_to_rvtcq (qp->ibqp.send_cq), &wc,0 ); } -if (++qp->s_last >= qp->s_size) - qp->s_last =0 ; }/* * If we were waiting for sends to complete before resending,
@@ -1068,11 +1074,19 @@ static struct rvt_swqe *do_rc_completion(struct rvt_qp *qp,*/ if (qib_cmp24 (wqe->lpsn, qp->s_sending_psn) <0 ||qib_cmp24 (qp->s_sending_psn, qp->s_sending_hpsn) >0 ) { +u32 s_last; +for (i =0 ; i < wqe->wr.num_sge; i++) {struct rvt_sge *sge = &wqe->sg_list[i];rvt_put_mr (sge->mr); } + s_last = qp->s_last; +if (++s_last >= qp->s_size) + s_last =0 ; + qp->s_last = s_last; +/* see post_send() */ +barrier ();/* Post a send completion queue entry if requested. */ if (!(qp->s_flags &RVT_S_SIGNAL_REQ_WR ) || (wqe->wr.send_flags &IB_SEND_SIGNALED )) {
@@ -1084,8 +1098,6 @@ static struct rvt_swqe *do_rc_completion(struct rvt_qp *qp, wc.qp = &qp->ibqp;rvt_cq_enter (ibcq_to_rvtcq (qp->ibqp.send_cq), &wc,0 ); } -if (++qp->s_last >= qp->s_size) - qp->s_last =0 ; }else this_cpu_inc (*ibp->rvp.rc_delayed_comp);
drivers/infiniband/hw/qib/qib_ruc.c +7 -5
@@ -795,6 +795,13 @@ void qib_send_complete(struct rvt_qp *qp, struct rvt_swqe *wqe,if (!(ib_rvt_state_ops[qp->state] &RVT_PROCESS_OR_FLUSH_SEND ))return ; + last = qp->s_last; + old_last = last; +if (++last >= qp->s_size) + last =0 ; + qp->s_last = last; +/* See post_send() */ +barrier ();for (i =0 ; i < wqe->wr.num_sge; i++) {struct rvt_sge *sge = &wqe->sg_list[i];
@@ -822,11 +829,6 @@ void qib_send_complete(struct rvt_qp *qp, struct rvt_swqe *wqe, status != IB_WC_SUCCESS); } - last = qp->s_last; - old_last = last; -if (++last >= qp->s_size) - last =0 ; - qp->s_last = last;if (qp->s_acked == old_last) qp->s_acked = last;if (qp->s_cur == old_last)
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
PATCH 5/6 IB/rdmavt: remove unused qp field
In reply to a message not in the archiveFrom: Mike Marciniszyn <[email protected]>
The field is a vestige from ipath.
--- include/rdma/rdmavt_qp.h | 1 - 1 files changed, 0 insertions(+), 1 deletions(-)
include/rdma/rdmavt_qp.h +0 -1
@@ -251,7 +251,6 @@ struct rvt_qp {u32 pmtu;/* decoded from path_mtu */ u32 qkey;/* QKEY for this QP (for UD or RD) */ u32 s_size;/* send work queue size */ -u32 s_rnr_timeout;/* number of milliseconds for RNR timeout */ u32 s_ahgpsn;/* set to the psn in the copy of the header */ u8 state;/* QP state */
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
PATCH 6/6 IB/qib,staging/rdma/hfi1: use setup_timer api
In reply to a message not in the archiveFrom: Hari Prasath Gujulan Elango <[email protected]>
Replace the timer API's to initialize a timer & then assign the callback function by the setup_timer() API.
--- drivers/infiniband/hw/qib/qib_verbs.c | 4 +--- drivers/staging/rdma/hfi1/verbs.c | 4 +--- 2 files changed, 2 insertions(+), 6 deletions(-)
drivers/infiniband/hw/qib/qib_verbs.c +1 -3
@@ -1602,9 +1602,7 @@ int qib_register_ib_device(struct qib_devdata *dd)init_ibport (ppd + i);/* Only need to initialize non-zero fields. */ -init_timer (&dev->mem_timer); - dev->mem_timer.function = mem_timer; - dev->mem_timer.data = (unsigned long ) dev; +setup_timer (&dev->mem_timer, mem_timer, (unsigned long )dev); qpt_mask = dd->qpn_mask;
drivers/staging/rdma/hfi1/verbs.c +1 -3
@@ -1455,9 +1455,7 @@ int hfi1_register_ib_device(struct hfi1_devdata *dd)/* Only need to initialize non-zero fields. */ -init_timer (&dev->mem_timer); - dev->mem_timer.function = mem_timer; - dev->mem_timer.data = (unsigned long ) dev; +setup_timer (&dev->mem_timer, mem_timer, (unsigned long )dev);seqlock_init (&dev->iowait_lock);INIT_LIST_HEAD (&dev->txwait);
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html