IB/hfi1,qib,rdmavt: Patches for 4.11
Subsystems (3)
cover IB/hfi1,qib,rdmavt: Patches for 4.11
Doug,
Here is a set of patches for our drivers for 4.11. This is a collection of bug fixes, perf improvements, and clean ups.
This does not include the port reordering patch series or vnic as those are still undergoing discussion on the list.
Patches apply to your for-4.11 branch and can also be found in my GitHub repo at:
https://github.com/ddalessa/kernel/tree/for-4.11
---
Brian Welty (5):
IB/hfi1,qib,rdmavt: Move two IB event functions into rdmavt IB/hfi1,qib,rdmavt: Move AETH credit functions into rdmavt IB/hfi1,rdmavt: Update copy_sge to use boolean arguments IB/hfi1,rdmavt: Move SGE state helper routines into rdmavt IB/qib: Updates to use rdmavt's SGE helper routines
Don Hiatt (3):
IB/hfi1: Add transmit fault injection feature IB/hfi1: Add rvt_rnr_tbl_to_usec function IB/hfi1,qib,rdmavt: Move AETH defines to rdma/ib_hdrs.h
Easwar Hariharan (1):
IB/hfi1: Use static CTLE with Preset 6 for integrated HFIs
Jakub Byczkowski (1):
IB/hfi1: Modify logging frequency of DCC errors
Michael J. Ruhl (2):
IB/hfi1: Do not set physical link state if DC is in the shutdown state IB/hfi1: Code reuse with memdup_copy
Mike Marciniszyn (7):
IB/hfi1: Correct defered count after processing qp_wait_list IB/hfi1: Process qp wait list in IRQ thread periodically IB/hfi1: Ensure read of producer s_head is correct IB/hfi1: Correct error calldown locking
IB/hfi1: Add additional fields to qp_stats IB/rdmavt,IB/hfi1,IB/qib: Correct ack count for passive (RTR) QPs IB/hfi1: Add receive fault injection feature
Sebastian Sanchez (5):
IB/hfi1: Access hfi1_ibport through rcd pointer IB/rdmavt: Use per-CPU reference count for MRs IB/hfi1: Allocate context data on memory node IB/hfi1: Reduce oversized fields in struct hfi1_packet IB/hfi1: Check upper-case EFI variables
Venkata Sandeep Dhanalakota (3):
IB/rdmavt: Adding timer logic to rdmavt
IB/hfi1: Use new rdmavt timers
IB/qib: Use new rdmavt timers
drivers/infiniband/hw/hfi1/Kconfig | 6 + drivers/infiniband/hw/hfi1/chip.c | 42 ++-- drivers/infiniband/hw/hfi1/common.h | 4 drivers/infiniband/hw/hfi1/debugfs.c | 269 ++++++++++++++++++++++++-- drivers/infiniband/hw/hfi1/debugfs.h | 41 ++++ drivers/infiniband/hw/hfi1/driver.c | 144 +++++++++----- drivers/infiniband/hw/hfi1/efivar.c | 26 ++- drivers/infiniband/hw/hfi1/hfi.h | 18 +- drivers/infiniband/hw/hfi1/init.c | 17 +- drivers/infiniband/hw/hfi1/pcie.c | 4 drivers/infiniband/hw/hfi1/qp.c | 177 +---------------- drivers/infiniband/hw/hfi1/qp.h | 22 -- drivers/infiniband/hw/hfi1/rc.c | 296 +++++------------------------ drivers/infiniband/hw/hfi1/ruc.c | 55 +---- drivers/infiniband/hw/hfi1/trace.c | 4 drivers/infiniband/hw/hfi1/trace_misc.h | 48 +++++ drivers/infiniband/hw/hfi1/uc.c | 16 +- drivers/infiniband/hw/hfi1/ud.c | 18 +- drivers/infiniband/hw/hfi1/user_exp_rcv.c | 17 +- drivers/infiniband/hw/hfi1/user_sdma.c | 17 +- drivers/infiniband/hw/hfi1/verbs.c | 170 ++++++----------- drivers/infiniband/hw/hfi1/verbs.h | 29 +-- drivers/infiniband/hw/qib/qib_common.h | 4 drivers/infiniband/hw/qib/qib_qp.c | 135 ------------- drivers/infiniband/hw/qib/qib_rc.c | 179 +++++------------- drivers/infiniband/hw/qib/qib_ruc.c | 47 ----- drivers/infiniband/hw/qib/qib_uc.c | 15 - drivers/infiniband/hw/qib/qib_ud.c | 8 - drivers/infiniband/hw/qib/qib_verbs.c | 96 +-------- drivers/infiniband/hw/qib/qib_verbs.h | 10 - drivers/infiniband/sw/rdmavt/Makefile | 4 drivers/infiniband/sw/rdmavt/mr.c | 59 ++++-- drivers/infiniband/sw/rdmavt/pd.c | 2 drivers/infiniband/sw/rdmavt/qp.c | 233 +++++++++++++++++++++++ drivers/infiniband/sw/rdmavt/rc.c | 189 +++++++++++++++++++ include/rdma/ib_hdrs.h | 6 + include/rdma/ib_pack.h | 2 include/rdma/rdma_vt.h | 21 ++ include/rdma/rdmavt_mr.h | 60 +++++- include/rdma/rdmavt_qp.h | 46 +++++ 40 files changed, 1331 insertions(+), 1225 deletions(-) create mode 100644 drivers/infiniband/sw/rdmavt/rc.c
--
-Denny
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
A message that is not in the archive (sent off-list or not yet received): 20170208132142.16442.69329.stgit-9QXIwq+3FY+1XWohqUldA0EOCMrvLtNR@public.gmane.org
PATCH 1/27 IB/hfi1: Correct defered count after processing qp_wait_list
From: Mike Marciniszyn <[email protected]>
The qp_wait_list processing leaves the defered ack count at its prior value.
This can result in a premature send of an ack.
Fixed by unconditionally reseting the defered ack count in hfi1_send_rc_ack().
--- drivers/infiniband/hw/hfi1/rc.c | 4 ++++ 1 files changed, 4 insertions(+), 0 deletions(-)
drivers/infiniband/hw/hfi1/rc.c +4 -0
@@ -853,6 +853,10 @@ void hfi1_send_rc_ack(struct hfi1_ctxtdata *rcd, struct rvt_qp *qp,struct ib_header hdr;struct ib_other_headers *ohdr;unsigned long flags; +struct hfi1_qp_priv *priv = qp->priv; + +/* clear the defer count */ + priv->r_adefered =0 ;/* Don't send ACK or NAK if a RDMA read or atomic is pending. */ if (qp->s_flags &RVT_S_RESP_PENDING )
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
PATCH 2/27 IB/hfi1: Process qp wait list in IRQ thread periodically
In reply to a message not in the archiveFrom: Mike Marciniszyn <[email protected]>
In the event that the IRQ thread is extremely busy, the processing of an rcd wait list can be delayed by quite a bit until the IRQ thread completes its work.
The QP reset reference count wait can then appear to be stuck, thus causing up a QP destroy to emit the hung task diagnostic.
Fix by processing the qp wait list periodically from the thread. The interval is a multiple (currently 4) of the MAX_PKT_RECV.
Also, reduce some of the excessive inlining. The guidelines
are per packet is ok inline, otherwise the choice is based on likelyhood of execution.
--- drivers/infiniband/hw/hfi1/driver.c | 121 +++++++++++++++++++---------------- 1 files changed, 67 insertions(+), 54 deletions(-)
drivers/infiniband/hw/hfi1/driver.c +67 -54
@@ -100,6 +100,11 @@* MAX_PKT_RCV is the max # if packets processed per receive interrupt. */ #define MAX_PKT_RECV 64 +/* +* MAX_PKT_THREAD_RCV is the max # of packets processed before +* the qp_wait_list queue is flushed. +*/ +#define MAX_PKT_RECV_THREAD (MAX_PKT_RECV * 4)#define EGR_HEAD_UPDATE_THRESHOLD 16struct hfi1_ib_stats hfi1_stats;
@@ -259,7 +264,7 @@ int hfi1_count_units(int *npresentp, int *nupp)* allowed size ranges for the respective type and, optionally, * return the proper encoding. */ -inline int hfi1_rcvbuf_validate (u32 size,u8 type,u16 *encoded) +int hfi1_rcvbuf_validate (u32 size,u8 type,u16 *encoded) {if (unlikely (!PAGE_ALIGNED (size)))return 0 ;
@@ -654,24 +659,68 @@ static void __prescan_rxq(struct hfi1_packet *packet) } } -static inline int skip_rcv_packet (struct hfi1_packet *packet,int thread) +static void process_rcv_qp_work (struct hfi1_ctxtdata *rcd) +{ +struct rvt_qp *qp, *nqp; + +/* +* Iterate over all QPs waiting to respond. +* The list won't change since the IRQ is only run on one CPU. +*/ +list_for_each_entry_safe (qp, nqp, &rcd->qp_wait_list, rspwait) { +list_del_init (&qp->rspwait); +if (qp->r_flags &RVT_R_RSP_NAK ) { + qp->r_flags &= ~RVT_R_RSP_NAK ; +hfi1_send_rc_ack (rcd, qp,0 ); + } +if (qp->r_flags &RVT_R_RSP_SEND ) { +unsigned long flags; + + qp->r_flags &= ~RVT_R_RSP_SEND ; +spin_lock_irqsave (&qp->s_lock, flags); +if (ib_rvt_state_ops[qp->state] & +RVT_PROCESS_OR_FLUSH_SEND ) +hfi1_schedule_send (qp); +spin_unlock_irqrestore (&qp->s_lock, flags); + } +rvt_put_qp (qp); + } +} + +static noinline int max_packet_exceeded(struct hfi1_packet *packet, int thread) +{ +if (thread) { +if ((packet->numpkt & (MAX_PKT_RECV_THREAD -1 )) ==0 ) +/* allow defered processing */ +process_rcv_qp_work (packet->rcd); +cond_resched (); +return RCV_PKT_OK ; + }else { +this_cpu_inc (*packet->rcd->dd->rcv_limit); +return RCV_PKT_LIMIT ; + } +} + +static inline int check_max_packet (struct hfi1_packet *packet,int thread) {int ret =RCV_PKT_OK ; +if (unlikely ((packet->numpkt & (MAX_PKT_RECV -1 )) ==0 )) + ret =max_packet_exceeded (packet, thread); +return ret; +} + +static noinline int skip_rcv_packet(struct hfi1_packet *packet, int thread) +{ +int ret; +/* Set up for the next packet */ packet->rhqoff += packet->rsize;if (packet->rhqoff >= packet->maxcnt) packet->rhqoff =0 ; packet->numpkt++; -if (unlikely ((packet->numpkt & (MAX_PKT_RECV -1 )) ==0 )) { -if (thread) { -cond_resched (); - }else { - ret =RCV_PKT_LIMIT ; -this_cpu_inc (*packet->rcd->dd->rcv_limit); - } - } + ret =check_max_packet (packet, thread); packet->rhf_addr = (__le32 *)packet->rcd->rcvhdrq + packet->rhqoff + packet->rcd->dd->rhf_offset;
@@ -682,7 +731,7 @@ static inline int skip_rcv_packet(struct hfi1_packet *packet, int thread)static inline int process_rcv_packet (struct hfi1_packet *packet,int thread) { -int ret =RCV_PKT_OK ; +int ret; packet->hdr =hfi1_get_msgheader (packet->rcd->dd, packet->rhf_addr);
@@ -723,14 +772,7 @@ static inline int process_rcv_packet(struct hfi1_packet *packet, int thread)if (packet->rhqoff >= packet->maxcnt) packet->rhqoff =0 ; -if (unlikely ((packet->numpkt & (MAX_PKT_RECV -1 )) ==0 )) { -if (thread) { -cond_resched (); - }else { - ret =RCV_PKT_LIMIT ; -this_cpu_inc (*packet->rcd->dd->rcv_limit); - } - } + ret =check_max_packet (packet, thread); packet->rhf_addr = (__le32 *)packet->rcd->rcvhdrq + packet->rhqoff + packet->rcd->dd->rhf_offset;
@@ -767,38 +809,6 @@ static inline void finish_packet(struct hfi1_packet *packet) packet->etail, rcv_intr_dynamic, packet->numpkt); } -static inline void process_rcv_qp_work (struct hfi1_packet *packet) -{ -struct hfi1_ctxtdata *rcd; -struct rvt_qp *qp, *nqp; - - rcd = packet->rcd; - rcd->head = packet->rhqoff; - -/* -* Iterate over all QPs waiting to respond. -* The list won't change since the IRQ is only run on one CPU. -*/ -list_for_each_entry_safe (qp, nqp, &rcd->qp_wait_list, rspwait) { -list_del_init (&qp->rspwait); -if (qp->r_flags &RVT_R_RSP_NAK ) { - qp->r_flags &= ~RVT_R_RSP_NAK ; -hfi1_send_rc_ack (rcd, qp,0 ); - } -if (qp->r_flags &RVT_R_RSP_SEND ) { -unsigned long flags; - - qp->r_flags &= ~RVT_R_RSP_SEND ; -spin_lock_irqsave (&qp->s_lock, flags); -if (ib_rvt_state_ops[qp->state] & -RVT_PROCESS_OR_FLUSH_SEND ) -hfi1_schedule_send (qp); -spin_unlock_irqrestore (&qp->s_lock, flags); - } -rvt_put_qp (qp); - } -} -/* * Handle receive interrupts when using the no dma rtail option. */
@@ -826,7 +836,8 @@ int handle_receive_interrupt_nodma_rtail(struct hfi1_ctxtdata *rcd, int thread) last =RCV_PKT_DONE ;process_rcv_update (last, &packet); } -process_rcv_qp_work (&packet); +process_rcv_qp_work (rcd); + rcd->head = packet.rhqoff; bail:finish_packet (&packet);return last;
@@ -854,7 +865,8 @@ int handle_receive_interrupt_dma_rtail(struct hfi1_ctxtdata *rcd, int thread) last =RCV_PKT_DONE ;process_rcv_update (last, &packet); } -process_rcv_qp_work (&packet); +process_rcv_qp_work (rcd); + rcd->head = packet.rhqoff; bail:finish_packet (&packet);return last;
@@ -1024,7 +1036,8 @@ int handle_receive_interrupt(struct hfi1_ctxtdata *rcd, int thread)process_rcv_update (last, &packet); } -process_rcv_qp_work (&packet); +process_rcv_qp_work (rcd); + rcd->head = packet.rhqoff; bail:/*
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
PATCH 3/27 IB/hfi1: Ensure read of producer s_head is correct
In reply to a message not in the archiveFrom: Mike Marciniszyn <[email protected]>
The read of s_head in the hfi1_make_rc_req() and qib_make_rc_req() lack the necesary barrier instuctions.
Correct other ACCESS_ONCE() warnings in the same file.
--- drivers/infiniband/hw/hfi1/rc.c | 7 ++++--- drivers/infiniband/hw/qib/qib_rc.c | 7 ++++--- 2 files changed, 8 insertions(+), 6 deletions(-)
drivers/infiniband/hw/hfi1/rc.c +4 -3
@@ -414,7 +414,7 @@ int hfi1_make_rc_req(struct rvt_qp *qp, struct hfi1_pkt_state *ps) goto bail;/* We are in the error state, flush the work request. */ smp_read_barrier_depends ();/* see post_one_send() */ -if (qp->s_last ==ACCESS_ONCE (qp->s_head)) +if (qp->s_last ==READ_ONCE (qp->s_head)) goto bail;/* If DMAs are in progress, we can't flush immediately. */ if (iowait_sdma_pending (&priv->s_iowait)) {
@@ -457,7 +457,8 @@ int hfi1_make_rc_req(struct rvt_qp *qp, struct hfi1_pkt_state *ps) newreq =0 ;if (qp->s_cur == qp->s_tail) {/* Check if send work queue is empty. */ -if (qp->s_tail == qp->s_head) { +smp_read_barrier_depends ();/* see post_one_send() */ +if (qp->s_tail ==READ_ONCE (qp->s_head)) {clear_ahg (qp); goto bail; }
@@ -1590,7 +1591,7 @@ static void rc_rcv_resp(struct hfi1_ibport *ibp,/* Ignore invalid responses. */ smp_read_barrier_depends ();/* see post_one_send */ -if (cmp_psn (psn,ACCESS_ONCE (qp->s_next_psn)) >=0 ) +if (cmp_psn (psn,READ_ONCE (qp->s_next_psn)) >=0 ) goto ack_done;/* Ignore duplicate responses. */
drivers/infiniband/hw/qib/qib_rc.c +4 -3
@@ -257,7 +257,7 @@ int qib_make_rc_req(struct rvt_qp *qp, unsigned long *flags) goto bail;/* We are in the error state, flush the work request. */ smp_read_barrier_depends ();/* see post_one_send() */ -if (qp->s_last ==ACCESS_ONCE (qp->s_head)) +if (qp->s_last ==READ_ONCE (qp->s_head)) goto bail;/* If DMAs are in progress, we can't flush immediately. */ if (atomic_read (&priv->s_dma_busy)) {
@@ -303,7 +303,8 @@ int qib_make_rc_req(struct rvt_qp *qp, unsigned long *flags) newreq =0 ;if (qp->s_cur == qp->s_tail) {/* Check if send work queue is empty. */ -if (qp->s_tail == qp->s_head) +smp_read_barrier_depends ();/* see post_one_send() */ +if (qp->s_tail ==READ_ONCE (qp->s_head)) goto bail;/* * If a fence is requested, wait for previous
@@ -1390,7 +1391,7 @@ static void qib_rc_rcv_resp(struct qib_ibport *ibp,/* Ignore invalid responses. */ smp_read_barrier_depends ();/* see post_one_send */ -if (qib_cmp24 (psn,ACCESS_ONCE (qp->s_next_psn)) >=0 ) +if (qib_cmp24 (psn,READ_ONCE (qp->s_next_psn)) >=0 ) goto ack_done;/* Ignore duplicate responses. */
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
PATCH 4/27 IB/hfi1: Use static CTLE with Preset 6 for integrated HFIs
In reply to a message not in the archiveFrom: Easwar Hariharan <[email protected]>
After extended testing, it was found that the previous PCIe Gen 3 recipe, which used adaptive CTLE with Preset 4, could cause an NMI/Surprise Link Down in about 1 in 100 to 1 in 1000 power cycles on some platforms. New EV data combined with extensive empirical data indicates that the new recipe should use static CTLE with Preset 6 for all integrated silicon SKUs.
--- drivers/infiniband/hw/hfi1/pcie.c | 4 ++-- 1 files changed, 2 insertions(+), 2 deletions(-)
drivers/infiniband/hw/hfi1/pcie.c +2 -2
@@ -663,12 +663,12 @@ static void tune_pcie_caps(struct hfi1_devdata *dd)#define UNSET_PSET 255#define DEFAULT_DISCRETE_PSET 2/* discrete HFI */ -#define DEFAULT_MCP_PSET 4/* MCP HFI */ +#define DEFAULT_MCP_PSET 6/* MCP HFI */ static uint pcie_pset =UNSET_PSET ;module_param (pcie_pset, uint,S_IRUGO );MODULE_PARM_DESC (pcie_pset,"PCIe Eq Pset value to use, range is 0-10" ); -static uint pcie_ctle =1 ;/* discrete on, integrated off */ +static uint pcie_ctle =3 ;/* discrete on, integrated on */ module_param (pcie_ctle, uint,S_IRUGO );MODULE_PARM_DESC (pcie_ctle,"PCIe static CTLE mode, bit 0 - discrete on/off, bit 1 - integrated on/off" );
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
PATCH 5/27 IB/hfi1: Correct error calldown locking
In reply to a message not in the archiveFrom: Mike Marciniszyn <[email protected]>
The resource specific wait locking missed correcting the lock for the notify_error_qp() calldown.
The code is fixed to correctly use the iowait lock field to protect the head that is protected by that lock.
--- drivers/infiniband/hw/hfi1/qp.c | 19 +++++++++++-------- 1 files changed, 11 insertions(+), 8 deletions(-)
drivers/infiniband/hw/hfi1/qp.c +11 -8
@@ -961,17 +961,20 @@ int get_pmtu_from_attr(struct rvt_dev_info *rdi, struct rvt_qp *qp,void notify_error_qp (struct rvt_qp *qp) { -struct hfi1_ibdev *dev =to_idev (qp->ibqp.device);struct hfi1_qp_priv *priv = qp->priv; +seqlock_t *lock = priv->s_iowait.lock; -write_seqlock (&dev->iowait_lock); -if (!list_empty (&priv->s_iowait.list) && !(qp->s_flags &RVT_S_BUSY )) { - qp->s_flags &= ~RVT_S_ANY_WAIT_IO ; -list_del_init (&priv->s_iowait.list); - priv->s_iowait.lock =NULL ; -rvt_put_qp (qp); +if (lock) { +write_seqlock (lock); +if (!list_empty (&priv->s_iowait.list) && + !(qp->s_flags &RVT_S_BUSY )) { + qp->s_flags &= ~RVT_S_ANY_WAIT_IO ; +list_del_init (&priv->s_iowait.list); + priv->s_iowait.lock =NULL ; +rvt_put_qp (qp); + } +write_sequnlock (lock); } -write_sequnlock (&dev->iowait_lock);if (!(qp->s_flags &RVT_S_BUSY )) { qp->s_hdrwords =0 ;
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
PATCH 6/27 IB/hfi1: Access hfi1_ibport through rcd pointer
In reply to a message not in the archiveFrom: Sebastian Sanchez <[email protected]>
Receive code paths use the QP's device and port number to access the struct hfi1_ibport. When an instance of struct hfi1_ctxtdata is present, it can be used to access struct hfi1_ibport through a pointer. This makes struct hfi1_ibport lookup time faster as an array doesn't have to be indexed and access fields in other cache-lines.
--- drivers/infiniband/hw/hfi1/driver.c | 4 ++-- drivers/infiniband/hw/hfi1/hfi.h | 5 +++++ drivers/infiniband/hw/hfi1/rc.c | 10 +++++----- drivers/infiniband/hw/hfi1/uc.c | 2 +- drivers/infiniband/hw/hfi1/ud.c | 2 +- drivers/infiniband/hw/hfi1/verbs.c | 4 ++-- 6 files changed, 16 insertions(+), 11 deletions(-)
drivers/infiniband/hw/hfi1/driver.c +2 -2
@@ -284,7 +284,7 @@ static void rcv_hdrerr(struct hfi1_ctxtdata *rcd, struct hfi1_pportdata *ppd,struct ib_header *rhdr = packet->hdr;u32 rte =rhf_rcv_type_err (packet->rhf);int lnh =be16_to_cpu (rhdr->lrh[0 ]) &3 ; -struct hfi1_ibport *ibp = &ppd->ibport_data; +struct hfi1_ibport *ibp =rcd_to_iport (rcd);struct hfi1_devdata *dd = ppd->dd;struct rvt_dev_info *rdi = &dd->verbs_dev.rdi;
@@ -599,7 +599,7 @@ static void __prescan_rxq(struct hfi1_packet *packet)while (1 ) {struct hfi1_devdata *dd = rcd->dd; -struct hfi1_ibport *ibp = &rcd->ppd->ibport_data; +struct hfi1_ibport *ibp =rcd_to_iport (rcd);__le32 *rhf_addr = (__le32 *)rcd->rcvhdrq + mdata.ps_head + dd->rhf_offset;struct rvt_qp *qp;
drivers/infiniband/hw/hfi1/hfi.h +5 -0
@@ -1584,6 +1584,11 @@ static inline int valid_opa_max_mtu(unsigned int mtu)return &dd->pport[pidx].ibport_data; } +static inline struct hfi1_ibport *rcd_to_iport (struct hfi1_ctxtdata *rcd) +{ +return &rcd->ppd->ibport_data; +} +void hfi1_process_ecn_slowpath (struct rvt_qp *qp,struct hfi1_packet *pkt,bool do_cnp); static inline bool process_ecn(struct rvt_qp *qp, struct hfi1_packet *pkt,
drivers/infiniband/hw/hfi1/rc.c +5 -5
@@ -841,7 +841,7 @@ int hfi1_make_rc_req(struct rvt_qp *qp, struct hfi1_pkt_state *ps)void hfi1_send_rc_ack (struct hfi1_ctxtdata *rcd,struct rvt_qp *qp,int is_fecn) { -struct hfi1_ibport *ibp =to_iport (qp->ibqp.device, qp->port_num); +struct hfi1_ibport *ibp =rcd_to_iport (rcd);struct hfi1_pportdata *ppd =ppd_from_ibp (ibp);u64 pbc, pbc_flags =0 ;u16 lrh0;
@@ -1326,7 +1326,7 @@ static int do_rc_ack(struct rvt_qp *qp, u32 aeth, u32 psn, int opcode,if (aeth >>29 ) ack_psn--; wqe =rvt_get_swqe_ptr (qp, qp->s_acked); - ibp =to_iport (qp->ibqp.device, qp->port_num); + ibp =rcd_to_iport (rcd);/* * The MSN might be for a later WQE than the PSN indicates so
@@ -1791,7 +1791,7 @@ static noinline int rc_rcv_error(struct ib_other_headers *ohdr, void *data, struct rvt_qp *qp, u32 opcode, u32 psn, int diff, struct hfi1_ctxtdata *rcd) { - struct hfi1_ibport *ibp = to_iport(qp->ibqp.device, qp->port_num); + struct hfi1_ibport *ibp = rcd_to_iport(rcd);struct rvt_ack_entry *e;unsigned long flags;u8 i, prev;
@@ -2100,7 +2100,7 @@ void hfi1_rc_rcv(struct hfi1_packet *packet)void *data = packet->ebuf;u32 tlen = packet->tlen;struct rvt_qp *qp = packet->qp; -struct hfi1_ibport *ibp =to_iport (qp->ibqp.device, qp->port_num); +struct hfi1_ibport *ibp =rcd_to_iport (rcd);struct ib_other_headers *ohdr = packet->ohdr;u32 bth0, opcode;u32 hdrsize = packet->hlen;
@@ -2552,7 +2552,7 @@ void hfi1_rc_hdrerr( {int has_grh = rcv_flags &HFI1_HAS_GRH ;struct ib_other_headers *ohdr; -struct hfi1_ibport *ibp =to_iport (qp->ibqp.device, qp->port_num); +struct hfi1_ibport *ibp =rcd_to_iport (rcd);int diff;u32 opcode;u32 psn, bth0;
drivers/infiniband/hw/hfi1/uc.c +1 -1
@@ -296,7 +296,7 @@ int hfi1_make_uc_req(struct rvt_qp *qp, struct hfi1_pkt_state *ps)*/ void hfi1_uc_rcv (struct hfi1_packet *packet) { -struct hfi1_ibport *ibp = &packet->rcd->ppd->ibport_data; +struct hfi1_ibport *ibp =rcd_to_iport (packet->rcd);struct ib_header *hdr = packet->hdr;u32 rcv_flags = packet->rcv_flags;void *data = packet->ebuf;
drivers/infiniband/hw/hfi1/ud.c +1 -1
@@ -672,7 +672,7 @@ void hfi1_ud_rcv(struct hfi1_packet *packet)u32 src_qp;u16 dlid, pkey;int mgmt_pkey_idx =-1 ; -struct hfi1_ibport *ibp = &packet->rcd->ppd->ibport_data; +struct hfi1_ibport *ibp =rcd_to_iport (packet->rcd);struct hfi1_pportdata *ppd =ppd_from_ibp (ibp);struct ib_header *hdr = packet->hdr;u32 rcv_flags = packet->rcv_flags;
drivers/infiniband/hw/hfi1/verbs.c +2 -2
@@ -576,7 +576,7 @@ void hfi1_ib_rcv(struct hfi1_packet *packet)struct ib_header *hdr = packet->hdr;u32 tlen = packet->tlen;struct hfi1_pportdata *ppd = rcd->ppd; -struct hfi1_ibport *ibp = &ppd->ibport_data; +struct hfi1_ibport *ibp =rcd_to_iport (rcd);struct rvt_dev_info *rdi = &ppd->dd->verbs_dev.rdi;opcode_handler packet_handler;unsigned long flags;
@@ -1910,7 +1910,7 @@ void hfi1_unregister_ib_device(struct hfi1_devdata *dd)void hfi1_cnp_rcv (struct hfi1_packet *packet) { -struct hfi1_ibport *ibp = &packet->rcd->ppd->ibport_data; +struct hfi1_ibport *ibp =rcd_to_iport (packet->rcd);struct hfi1_pportdata *ppd =ppd_from_ibp (ibp);struct ib_header *hdr = packet->hdr;struct rvt_qp *qp = packet->qp;
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
PATCH 7/27 IB/rdmavt: Use per-CPU reference count for MRs
In reply to a message not in the archiveFrom: Sebastian Sanchez <[email protected]>
Having per-CPU reference count for each MR prevents cache-line bouncing across the system. Thus, it prevents bottlenecks. Use per-CPU reference counts per MR.
The per-CPU reference count for FMRs is used in atomic mode to allow accurate testing of the busy state. Other MR types run in per-CPU mode MR until they're freed.
--- drivers/infiniband/sw/rdmavt/mr.c | 59 ++++++++++++++++++++++++------------- include/rdma/rdmavt_mr.h | 10 +++--- 2 files changed, 43 insertions(+), 26 deletions(-)
drivers/infiniband/sw/rdmavt/mr.c +38 -21
@@ -120,10 +120,19 @@ static void rvt_deinit_mregion(struct rvt_mregion *mr) mr->mapsz =0 ;while (i)kfree (mr->map[--i]); +percpu_ref_exit (&mr->refcount); +} + +static void __rvt_mregion_complete (struct percpu_ref *ref) +{ + struct rvt_mregion *mr = container_of(ref, struct rvt_mregion, + refcount); + +complete (&mr->comp); }static int rvt_init_mregion (struct rvt_mregion *mr,struct ib_pd *pd, -int count) +int count,unsigned int percpu_flags) {int m, i =0 ;struct rvt_dev_info *dev =ib_to_rvt (pd->device);
@@ -133,19 +142,23 @@ static int rvt_init_mregion(struct rvt_mregion *mr, struct ib_pd *pd,for (; i < m; i++) { mr->map[i] =kzalloc_node (sizeof (*mr->map[0 ]),GFP_KERNEL , dev->dparms.node); -if (!mr->map[i]) { -rvt_deinit_mregion (mr); -return -ENOMEM ; - } +if (!mr->map[i]) + goto bail; mr->mapsz++; }init_completion (&mr->comp);/* count returning the ptr to user */ -atomic_set (&mr->refcount,1 ); +if (percpu_ref_init (&mr->refcount, &__rvt_mregion_complete, + percpu_flags,GFP_KERNEL )) + goto bail; +atomic_set (&mr->lkey_invalid,0 ); mr->pd = pd; mr->max_segs = count;return 0 ; +bail: +rvt_deinit_mregion (mr); +return -ENOMEM ; }/**
@@ -180,8 +193,7 @@ static int rvt_alloc_lkey(struct rvt_mregion *mr, int dma_region)if (!tmr) {rcu_assign_pointer (dev->dma_mr, mr); mr->lkey_published =1 ; - }else { -rvt_put_mr (mr); +rvt_get_mr (mr); } goto success; }
@@ -239,11 +251,14 @@ static void rvt_free_lkey(struct rvt_mregion *mr)int freed =0 ;spin_lock_irqsave (&rkt->lock, flags); -if (!mr->lkey_published) - goto out; -if (lkey ==0 ) { -RCU_INIT_POINTER (dev->dma_mr,NULL ); +if (!lkey) { +if (mr->lkey_published) { +RCU_INIT_POINTER (dev->dma_mr,NULL ); +rvt_put_mr (mr); + } }else { +if (!mr->lkey_published) + goto out; r = lkey >> (32 - dev->dparms.lkey_table_size);RCU_INIT_POINTER (rkt->table[r],NULL ); }
@@ -253,7 +268,7 @@ static void rvt_free_lkey(struct rvt_mregion *mr)spin_unlock_irqrestore (&rkt->lock, flags);if (freed) {synchronize_rcu (); -rvt_put_mr (mr); +percpu_ref_kill (&mr->refcount); } }
@@ -269,7 +284,7 @@ static void rvt_free_lkey(struct rvt_mregion *mr)if (!mr) goto bail; - rval =rvt_init_mregion (&mr->mr, pd, count); + rval =rvt_init_mregion (&mr->mr, pd, count,0 );if (rval) goto bail;/*
@@ -294,8 +309,8 @@ static void rvt_free_lkey(struct rvt_mregion *mr)static void __rvt_free_mr (struct rvt_mr *mr) { -rvt_deinit_mregion (&mr->mr);rvt_free_lkey (&mr->mr); +rvt_deinit_mregion (&mr->mr);kfree (mr); }
@@ -323,7 +338,7 @@ struct ib_mr *rvt_get_dma_mr(struct ib_pd *pd, int acc) goto bail; } - rval =rvt_init_mregion (&mr->mr, pd,0 ); + rval =rvt_init_mregion (&mr->mr, pd,0 ,0 );if (rval) { ret =ERR_PTR (rval); goto bail;
@@ -445,8 +460,8 @@ int rvt_dereg_mr(struct ib_mr *ibmr) timeout =wait_for_completion_timeout (&mr->mr.comp,5 *HZ );if (!timeout) {rvt_pr_err (rdi, -"rvt_dereg_mr timeout mr %p pd %p refcount %u\n" , - mr, mr->mr.pd,atomic_read (&mr->mr.refcount)); +"rvt_dereg_mr timeout mr %p pd %p\n" , + mr, mr->mr.pd);rvt_get_mr (&mr->mr); ret = -EBUSY ; goto out;
@@ -623,7 +638,8 @@ struct ib_fmr *rvt_alloc_fmr(struct ib_pd *pd, int mr_access_flags,if (!fmr) goto bail; - rval =rvt_init_mregion (&fmr->mr, pd, fmr_attr->max_pages); + rval =rvt_init_mregion (&fmr->mr, pd, fmr_attr->max_pages, +PERCPU_REF_INIT_ATOMIC );if (rval) goto bail;
@@ -674,11 +690,12 @@ int rvt_map_phys_fmr(struct ib_fmr *ibfmr, u64 *page_list,struct rvt_fmr *fmr =to_ifmr (ibfmr);struct rvt_lkey_table *rkt;unsigned long flags; -int m, n, i; +int m, n; +unsigned long i;u32 ps;struct rvt_dev_info *rdi =ib_to_rvt (ibfmr->device); - i =atomic_read (&fmr->mr.refcount); + i =atomic_long_read (&fmr->mr.refcount.count);if (i >2 )return -EBUSY ;
include/rdma/rdmavt_mr.h +5 -5
@@ -52,6 +52,7 @@* For Memory Regions. This stuff should probably be moved into rdmavt/mr.h once * drivers no longer need access to the MR directly. */ +#include <linux/percpu-refcount.h> /* * A segment is a linear region of low physical memory.
@@ -79,11 +80,11 @@ struct rvt_mregion {int access_flags;u32 max_segs;/* number of rvt_segs in all the arrays */ u32 mapsz;/* size of the map array */ +atomic_t lkey_invalid;/* true if current lkey is invalid */ u8 page_shift;/* 0 - non unform/non powerof2 sizes */ u8 lkey_published;/* in global table */ -atomic_t lkey_invalid;/* true if current lkey is invalid */ +struct percpu_ref refcount;struct completion comp;/* complete when refcount goes to zero */ -atomic_t refcount;struct rvt_segarray *map[0 ];/* the segments */ };
@@ -123,13 +124,12 @@ struct rvt_sge_state {static inline void rvt_put_mr (struct rvt_mregion *mr) { -if (unlikely (atomic_dec_and_test (&mr->refcount))) -complete (&mr->comp); +percpu_ref_put (&mr->refcount); }static inline void rvt_get_mr (struct rvt_mregion *mr) { -atomic_inc (&mr->refcount); +percpu_ref_get (&mr->refcount); }static inline void rvt_put_ss (struct rvt_sge_state *ss)
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
PATCH 8/27 IB/hfi1: Allocate context data on memory node
In reply to a message not in the archiveFrom: Sebastian Sanchez <[email protected]>
There are some memory allocation calls in hfi1_create_ctxtdata() that do not use the numa function parameter. This
can cause cache lines to be filled over QPI.
--- drivers/infiniband/hw/hfi1/init.c | 17 +++++++++-------- 1 files changed, 9 insertions(+), 8 deletions(-)
drivers/infiniband/hw/hfi1/init.c +9 -8
@@ -297,14 +297,15 @@ struct hfi1_ctxtdata *hfi1_create_ctxtdata(struct hfi1_pportdata *ppd, u32 ctxt,* The resulting value will be rounded down to the closest * multiple of dd->rcv_entries.group_size. */ - rcd->egrbufs.buffers =kcalloc (rcd->egrbufs.count, -sizeof (*rcd->egrbufs.buffers), -GFP_KERNEL ); + rcd->egrbufs.buffers =kzalloc_node ( + rcd->egrbufs.count *sizeof (*rcd->egrbufs.buffers), +GFP_KERNEL , numa);if (!rcd->egrbufs.buffers) goto bail; - rcd->egrbufs.rcvtids =kcalloc (rcd->egrbufs.count, -sizeof (*rcd->egrbufs.rcvtids), -GFP_KERNEL ); + rcd->egrbufs.rcvtids =kzalloc_node ( + rcd->egrbufs.count * +sizeof (*rcd->egrbufs.rcvtids), +GFP_KERNEL , numa);if (!rcd->egrbufs.rcvtids) goto bail; rcd->egrbufs.size = eager_buffer_size;
@@ -322,8 +323,8 @@ struct hfi1_ctxtdata *hfi1_create_ctxtdata(struct hfi1_pportdata *ppd, u32 ctxt, rcd->egrbufs.rcvtid_size =HFI1_MAX_EAGER_BUFFER_SIZE ;if (ctxt < dd->first_user_ctxt) {/* N/A for PSM contexts */ - rcd->opstats =kzalloc (sizeof (*rcd->opstats), -GFP_KERNEL ); + rcd->opstats =kzalloc_node (sizeof (*rcd->opstats), +GFP_KERNEL , numa);if (!rcd->opstats) goto bail; }
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
PATCH 9/27 IB/hfi1: Add additional fields to qp_stats
In reply to a message not in the archiveFrom: Mike Marciniszyn <[email protected]>
The r_psn and s_rnr_retry are missing.
Add with this patch.
--- drivers/infiniband/hw/hfi1/qp.c | 4 +++- 1 files changed, 3 insertions(+), 1 deletions(-)
drivers/infiniband/hw/hfi1/qp.c +3 -1
@@ -744,7 +744,7 @@ void qp_iter_print(struct seq_file *s, struct qp_iter *iter) wqe =rvt_get_swqe_ptr (qp, qp->s_last); send_context =qp_to_send_context (qp, priv->s_sc); seq_printf(s, - "N %d %s QP %x R %u %s %u %u %u f=%x %u %u %u %u %u %u PSN %x %x %x %x %x (%u %u %u %u %u %u %u) RQP %x LID %x SL %u MTU %u %u %u %u SDE %p,%u SC %p,%u SCQ %u %u PID %d\n", + "N %d %s QP %x R %u %s %u %u %u f=%x %u %u %u %u %u %u SPSN %x %x %x %x %x RPSN %x (%u %u %u %u %u %u %u) RQP %x LID %x SL %u MTU %u %u %u %u %u SDE %p,%u SC %p,%u SCQ %u %u PID %d\n", iter->n, qp_idle(qp) ? "I" : "B", qp->ibqp.qp_num,
@@ -763,6 +763,7 @@ void qp_iter_print(struct seq_file *s, struct qp_iter *iter) qp->s_last_psn, qp->s_psn, qp->s_next_psn, qp->s_sending_psn, qp->s_sending_hpsn, + qp->r_psn, qp->s_last, qp->s_acked, qp->s_cur, qp->s_tail, qp->s_head, qp->s_size, qp->s_avail,
@@ -773,6 +774,7 @@ void qp_iter_print(struct seq_file *s, struct qp_iter *iter) qp->s_retry, qp->s_retry_cnt, qp->s_rnr_retry_cnt, + qp->s_rnr_retry, sde, sde ? sde->this_idx :0 , send_context,
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
PATCH 10/27 IB/hfi1: Reduce oversized fields in struct hfi1_packet
In reply to a message not in the archiveFrom: Sebastian Sanchez <[email protected]>
Some fields in struct hfi1_packet are oversized.
Reduce them.
--- drivers/infiniband/hw/hfi1/hfi.h | 7 +++---- 1 files changed, 3 insertions(+), 4 deletions(-)
drivers/infiniband/hw/hfi1/hfi.h +3 -4
@@ -356,12 +356,11 @@ struct hfi1_packet {u64 rhf;u32 maxcnt;u32 rhqoff; -u32 hdrqtail; -int numpkt;u16 tlen; -u16 hlen;s16 etail; -u16 rsize; +u8 hlen; +u8 numpkt; +u8 rsize;u8 updegr;u8 rcv_flags;u8 etype;
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
PATCH 11/27 IB/hfi1: Check upper-case EFI variables
In reply to a message not in the archiveFrom: Sebastian Sanchez <[email protected]>
The EFI variable that provides board ID is named by the PCI address of the device, which is published in upper-case, while the HFI1 driver reads the EFI variable in lower-case.
This prevents returning the correct board id when queried through sysfs. Read EFI variables in upper-case if the lower-case read fails.
--- drivers/infiniband/hw/hfi1/efivar.c | 26 ++++++++++++++++++++++---- 1 files changed, 22 insertions(+), 4 deletions(-)
drivers/infiniband/hw/hfi1/efivar.c +22 -4
@@ -45,6 +45,7 @@* */ +#include <linux/ctype.h> #include "efivar.h" /* GUID for HFI1 variables in EFI */
@@ -150,15 +151,32 @@ static int read_efi_var(const char *name, unsigned long *size,int read_hfi1_efi_var (struct hfi1_devdata *dd,const char *kind,unsigned long *size,void **return_data) { +char prefix_name[64 ];char name[64 ]; +int result; +int i;/* create a common prefix */ -snprintf (name,sizeof (name),"%04x:%02x:%02x.%x-%s" , +snprintf (prefix_name,sizeof (prefix_name),"%04x:%02x:%02x.%x" ,pci_domain_nr (dd->pcidev->bus), dd->pcidev->bus->number,PCI_SLOT (dd->pcidev->devfn), -PCI_FUNC (dd->pcidev->devfn), - kind); +PCI_FUNC (dd->pcidev->devfn)); +snprintf (name,sizeof (name),"%s-%s" , prefix_name, kind); + result =read_efi_var (name, size, return_data); + +/* +* If reading the lowercase EFI variable fail, read the uppercase +* variable. +*/ +if (result) { +/* Converting to uppercase */ +for (i =0 ; prefix_name[i]; i++) +if (isalpha (prefix_name[i])) + prefix_name[i] =toupper (prefix_name[i]); +snprintf (name,sizeof (name),"%s-%s" , prefix_name, kind); + result =read_efi_var (name, size, return_data); + } -return read_efi_var (name, size, return_data); +return result; }
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
PATCH 12/27 IB/hfi1, qib, rdmavt: Move two IB event functions into rdmavt
In reply to a message not in the archiveFrom: Brian Welty <[email protected]>
Add rvt_rc_error() and rvt_comm_est() as shared functions in rdmavt, moved from hfi1/qib logic.
--- drivers/infiniband/hw/hfi1/qp.c | 13 ----------- drivers/infiniband/hw/hfi1/qp.h | 6 ----- drivers/infiniband/hw/hfi1/rc.c | 27 +++-------------------- drivers/infiniband/hw/hfi1/ruc.c | 2 +- drivers/infiniband/hw/hfi1/uc.c | 4 ++- drivers/infiniband/hw/hfi1/ud.c | 4 ++- drivers/infiniband/hw/hfi1/verbs.h | 2 -- drivers/infiniband/hw/qib/qib_rc.c | 38 ++++----------------------------- drivers/infiniband/hw/qib/qib_ruc.c | 2 +- drivers/infiniband/hw/qib/qib_uc.c | 15 +++---------- drivers/infiniband/hw/qib/qib_ud.c | 4 ++- drivers/infiniband/hw/qib/qib_verbs.h | 2 -- drivers/infiniband/sw/rdmavt/qp.c | 38 +++++++++++++++++++++++++++++++++ include/rdma/rdmavt_qp.h | 2 ++ 14 files changed, 60 insertions(+), 99 deletions(-)
drivers/infiniband/hw/hfi1/qp.c +0 -13
@@ -784,19 +784,6 @@ void qp_iter_print(struct seq_file *s, struct qp_iter *iter) qp->pid); } -void qp_comm_est (struct rvt_qp *qp) -{ - qp->r_flags |=RVT_R_COMM_EST ; -if (qp->ibqp.event_handler) { -struct ib_event ev; - - ev.device = qp->ibqp.device; - ev.element.qp = &qp->ibqp; - ev.event =IB_EVENT_COMM_EST ; - qp->ibqp.event_handler (&ev, qp->ibqp.qp_context); - } -} -void *qp_priv_alloc (struct rvt_dev_info *rdi,struct rvt_qp *qp,gfp_t gfp) {
drivers/infiniband/hw/hfi1/qp.h +0 -6
@@ -131,12 +131,6 @@ struct ib_qp *hfi1_create_qp(struct ib_pd *ibpd,*/ void qp_iter_print (struct seq_file *s,struct qp_iter *iter); -/** -* qp_comm_est - handle trap with QP established -* @qp: the QP -*/ -void qp_comm_est (struct rvt_qp *qp); -void _hfi1_schedule_send (struct rvt_qp *qp);void hfi1_schedule_send (struct rvt_qp *qp);
drivers/infiniband/hw/hfi1/rc.c +4 -23
@@ -1966,25 +1966,6 @@ static noinline int rc_rcv_error(struct ib_other_headers *ohdr, void *data,return 0 ; } -void hfi1_rc_error (struct rvt_qp *qp,enum ib_wc_status err) -{ -unsigned long flags; -int lastwqe; - -spin_lock_irqsave (&qp->s_lock, flags); - lastwqe =rvt_error_qp (qp, err); -spin_unlock_irqrestore (&qp->s_lock, flags); - -if (lastwqe) { -struct ib_event ev; - - ev.device = qp->ibqp.device; - ev.element.qp = &qp->ibqp; - ev.event =IB_EVENT_QP_LAST_WQE_REACHED ; - qp->ibqp.event_handler (&ev, qp->ibqp.qp_context); - } -} -static inline void update_ack_queue (struct rvt_qp *qp,unsigned n) {unsigned next;
@@ -2185,7 +2166,7 @@ void hfi1_rc_rcv(struct hfi1_packet *packet) }if (qp->state ==IB_QPS_RTR && !(qp->r_flags &RVT_R_COMM_EST )) -qp_comm_est (qp); +rvt_comm_est (qp);/* OK, process the packet. */ switch (opcode) {
@@ -2517,7 +2498,7 @@ void hfi1_rc_rcv(struct hfi1_packet *packet)return ; nack_op_err: -hfi1_rc_error (qp,IB_WC_LOC_QP_OP_ERR ); +rvt_rc_error (qp,IB_WC_LOC_QP_OP_ERR ); qp->r_nak_state =IB_NAK_REMOTE_OPERATIONAL_ERROR ; qp->r_ack_psn = qp->r_psn;/* Queue NAK for later */
@@ -2527,7 +2508,7 @@ void hfi1_rc_rcv(struct hfi1_packet *packet) nack_inv_unlck:spin_unlock_irqrestore (&qp->s_lock, flags); nack_inv: -hfi1_rc_error (qp,IB_WC_LOC_QP_OP_ERR ); +rvt_rc_error (qp,IB_WC_LOC_QP_OP_ERR ); qp->r_nak_state =IB_NAK_INVALID_REQUEST ; qp->r_ack_psn = qp->r_psn;/* Queue NAK for later */
@@ -2537,7 +2518,7 @@ void hfi1_rc_rcv(struct hfi1_packet *packet) nack_acc_unlck:spin_unlock_irqrestore (&qp->s_lock, flags); nack_acc: -hfi1_rc_error (qp,IB_WC_LOC_PROT_ERR ); +rvt_rc_error (qp,IB_WC_LOC_PROT_ERR ); qp->r_nak_state =IB_NAK_REMOTE_ACCESS_ERROR ; qp->r_ack_psn = qp->r_psn; send_ack:
drivers/infiniband/hw/hfi1/ruc.c +1 -1
@@ -637,7 +637,7 @@ static void ruc_loopback(struct rvt_qp *sqp) wc.status =IB_WC_LOC_PROT_ERR ; err:/* responder goes to error state */ -hfi1_rc_error (qp, wc.status); +rvt_rc_error (qp, wc.status); serr:spin_lock_irqsave (&sqp->s_lock, flags);
drivers/infiniband/hw/hfi1/uc.c +2 -2
@@ -384,7 +384,7 @@ void hfi1_uc_rcv(struct hfi1_packet *packet) }if (qp->state ==IB_QPS_RTR && !(qp->r_flags &RVT_R_COMM_EST )) -qp_comm_est (qp); +rvt_comm_est (qp);/* OK, process the packet. */ switch (opcode) {
@@ -584,5 +584,5 @@ void hfi1_uc_rcv(struct hfi1_packet *packet)return ; op_err: -hfi1_rc_error (qp,IB_WC_LOC_QP_OP_ERR ); +rvt_rc_error (qp,IB_WC_LOC_QP_OP_ERR ); }
drivers/infiniband/hw/hfi1/ud.c +2 -2
@@ -167,7 +167,7 @@ static void ud_loopback(struct rvt_qp *sqp, struct rvt_swqe *swqe) ret =hfi1_rvt_get_rwqe (qp,0 );if (ret <0 ) { -hfi1_rc_error (qp,IB_WC_LOC_QP_OP_ERR ); +rvt_rc_error (qp,IB_WC_LOC_QP_OP_ERR ); goto bail_unlock; }if (!ret) {
@@ -796,7 +796,7 @@ void hfi1_ud_rcv(struct hfi1_packet *packet) ret =hfi1_rvt_get_rwqe (qp,0 );if (ret <0 ) { -hfi1_rc_error (qp,IB_WC_LOC_QP_OP_ERR ); +rvt_rc_error (qp,IB_WC_LOC_QP_OP_ERR );return ; }if (!ret) {
drivers/infiniband/hw/hfi1/verbs.h +0 -2
@@ -327,8 +327,6 @@ void hfi1_rc_hdrerr(void hfi1_rc_send_complete (struct rvt_qp *qp,struct ib_header *hdr); -void hfi1_rc_error (struct rvt_qp *qp,enum ib_wc_status err); -void hfi1_ud_rcv (struct hfi1_packet *packet);int hfi1_lookup_pkey_idx (struct hfi1_ibport *ibp,u16 pkey);
drivers/infiniband/hw/qib/qib_rc.c +5 -33
@@ -1765,25 +1765,6 @@ static int qib_rc_rcv_error(struct ib_other_headers *ohdr,return 0 ; } -void qib_rc_error (struct rvt_qp *qp,enum ib_wc_status err) -{ -unsigned long flags; -int lastwqe; - -spin_lock_irqsave (&qp->s_lock, flags); - lastwqe =rvt_error_qp (qp, err); -spin_unlock_irqrestore (&qp->s_lock, flags); - -if (lastwqe) { -struct ib_event ev; - - ev.device = qp->ibqp.device; - ev.element.qp = &qp->ibqp; - ev.event =IB_EVENT_QP_LAST_WQE_REACHED ; - qp->ibqp.event_handler (&ev, qp->ibqp.qp_context); - } -} -static inline void qib_update_ack_queue (struct rvt_qp *qp,unsigned n) {unsigned next;
@@ -1895,17 +1876,8 @@ void qib_rc_rcv(struct qib_ctxtdata *rcd, struct ib_header *hdr,break ; } -if (qp->state ==IB_QPS_RTR && !(qp->r_flags &RVT_R_COMM_EST )) { - qp->r_flags |=RVT_R_COMM_EST ; -if (qp->ibqp.event_handler) { -struct ib_event ev; - - ev.device = qp->ibqp.device; - ev.element.qp = &qp->ibqp; - ev.event =IB_EVENT_COMM_EST ; - qp->ibqp.event_handler (&ev, qp->ibqp.qp_context); - } - } +if (qp->state ==IB_QPS_RTR && !(qp->r_flags &RVT_R_COMM_EST )) +rvt_comm_est (qp);/* OK, process the packet. */ switch (opcode) {
@@ -2197,7 +2169,7 @@ void qib_rc_rcv(struct qib_ctxtdata *rcd, struct ib_header *hdr,return ; nack_op_err: -qib_rc_error (qp,IB_WC_LOC_QP_OP_ERR ); +rvt_rc_error (qp,IB_WC_LOC_QP_OP_ERR ); qp->r_nak_state =IB_NAK_REMOTE_OPERATIONAL_ERROR ; qp->r_ack_psn = qp->r_psn;/* Queue NAK for later */
@@ -2211,7 +2183,7 @@ void qib_rc_rcv(struct qib_ctxtdata *rcd, struct ib_header *hdr, nack_inv_unlck:spin_unlock_irqrestore (&qp->s_lock, flags); nack_inv: -qib_rc_error (qp,IB_WC_LOC_QP_OP_ERR ); +rvt_rc_error (qp,IB_WC_LOC_QP_OP_ERR ); qp->r_nak_state =IB_NAK_INVALID_REQUEST ; qp->r_ack_psn = qp->r_psn;/* Queue NAK for later */
@@ -2225,7 +2197,7 @@ void qib_rc_rcv(struct qib_ctxtdata *rcd, struct ib_header *hdr, nack_acc_unlck:spin_unlock_irqrestore (&qp->s_lock, flags); nack_acc: -qib_rc_error (qp,IB_WC_LOC_PROT_ERR ); +rvt_rc_error (qp,IB_WC_LOC_PROT_ERR ); qp->r_nak_state =IB_NAK_REMOTE_ACCESS_ERROR ; qp->r_ack_psn = qp->r_psn; send_ack:
drivers/infiniband/hw/qib/qib_ruc.c +1 -1
@@ -621,7 +621,7 @@ static void qib_ruc_loopback(struct rvt_qp *sqp) wc.status =IB_WC_LOC_PROT_ERR ; err:/* responder goes to error state */ -qib_rc_error (qp, wc.status); +rvt_rc_error (qp, wc.status); serr:spin_lock_irqsave (&sqp->s_lock, flags);
drivers/infiniband/hw/qib/qib_uc.c +3 -12
@@ -325,17 +325,8 @@ void qib_uc_rcv(struct qib_ibport *ibp, struct ib_header *hdr, goto inv; } -if (qp->state ==IB_QPS_RTR && !(qp->r_flags &RVT_R_COMM_EST )) { - qp->r_flags |=RVT_R_COMM_EST ; -if (qp->ibqp.event_handler) { -struct ib_event ev; - - ev.device = qp->ibqp.device; - ev.element.qp = &qp->ibqp; - ev.event =IB_EVENT_COMM_EST ; - qp->ibqp.event_handler (&ev, qp->ibqp.qp_context); - } - } +if (qp->state ==IB_QPS_RTR && !(qp->r_flags &RVT_R_COMM_EST )) +rvt_comm_est (qp);/* OK, process the packet. */ switch (opcode) {
@@ -527,7 +518,7 @@ void qib_uc_rcv(struct qib_ibport *ibp, struct ib_header *hdr,return ; op_err: -qib_rc_error (qp,IB_WC_LOC_QP_OP_ERR ); +rvt_rc_error (qp,IB_WC_LOC_QP_OP_ERR );return ; }
drivers/infiniband/hw/qib/qib_ud.c +2 -2
@@ -152,7 +152,7 @@ static void qib_ud_loopback(struct rvt_qp *sqp, struct rvt_swqe *swqe) ret =qib_get_rwqe (qp,0 );if (ret <0 ) { -qib_rc_error (qp,IB_WC_LOC_QP_OP_ERR ); +rvt_rc_error (qp,IB_WC_LOC_QP_OP_ERR ); goto bail_unlock; }if (!ret) {
@@ -548,7 +548,7 @@ void qib_ud_rcv(struct qib_ibport *ibp, struct ib_header *hdr, ret =qib_get_rwqe (qp,0 );if (ret <0 ) { -qib_rc_error (qp,IB_WC_LOC_QP_OP_ERR ); +rvt_rc_error (qp,IB_WC_LOC_QP_OP_ERR );return ; }if (!ret) {
drivers/infiniband/hw/qib/qib_verbs.h +0 -2
@@ -326,8 +326,6 @@ void qib_rc_rcv(struct qib_ctxtdata *rcd, struct ib_header *hdr,void qib_rc_send_complete (struct rvt_qp *qp,struct ib_header *hdr); -void qib_rc_error (struct rvt_qp *qp,enum ib_wc_status err); -int qib_post_ud_send (struct rvt_qp *qp,struct ib_send_wr *wr); void qib_ud_rcv(struct qib_ibport *ibp, struct ib_header *hdr,
drivers/infiniband/sw/rdmavt/qp.c +38 -0
@@ -1868,3 +1868,41 @@ int rvt_post_srq_recv(struct ib_srq *ibsrq, struct ib_recv_wr *wr, }return 0 ; } + +/** +* qp_comm_est - handle trap with QP established +* @qp: the QP +*/ +void rvt_comm_est (struct rvt_qp *qp) +{ + qp->r_flags |=RVT_R_COMM_EST ; +if (qp->ibqp.event_handler) { +struct ib_event ev; + + ev.device = qp->ibqp.device; + ev.element.qp = &qp->ibqp; + ev.event =IB_EVENT_COMM_EST ; + qp->ibqp.event_handler (&ev, qp->ibqp.qp_context); + } +} +EXPORT_SYMBOL (rvt_comm_est); + +void rvt_rc_error (struct rvt_qp *qp,enum ib_wc_status err) +{ +unsigned long flags; +int lastwqe; + +spin_lock_irqsave (&qp->s_lock, flags); + lastwqe =rvt_error_qp (qp, err); +spin_unlock_irqrestore (&qp->s_lock, flags); + +if (lastwqe) { +struct ib_event ev; + + ev.device = qp->ibqp.device; + ev.element.qp = &qp->ibqp; + ev.event =IB_EVENT_QP_LAST_WQE_REACHED ; + qp->ibqp.event_handler (&ev, qp->ibqp.qp_context); + } +} +EXPORT_SYMBOL (rvt_rc_error);
include/rdma/rdmavt_qp.h +2 -0
@@ -607,6 +607,8 @@ static inline u32 rvt_div_mtu(struct rvt_qp *qp, u32 len)extern const int ib_rvt_state_ops[];struct rvt_dev_info ; +void rvt_comm_est (struct rvt_qp *qp);int rvt_error_qp (struct rvt_qp *qp,enum ib_wc_status err); +void rvt_rc_error (struct rvt_qp *qp,enum ib_wc_status err);#endif /* DEF_RDMAVT_INCQP_H */
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
PATCH 13/27 IB/hfi1, qib, rdmavt: Move AETH credit functions into rdmavt
In reply to a message not in the archiveFrom: Brian Welty <[email protected]>
Add rvt_compute_aeth() and rvt_get_credit() as shared functions in rdmavt, moved from hfi1/qib logic.
--- drivers/infiniband/hw/hfi1/common.h | 4 - drivers/infiniband/hw/hfi1/qp.c | 137 ----------------------- drivers/infiniband/hw/hfi1/qp.h | 16 --- drivers/infiniband/hw/hfi1/rc.c | 40 +++---- drivers/infiniband/hw/hfi1/trace.c | 4 - drivers/infiniband/hw/hfi1/verbs.h | 9 -- drivers/infiniband/hw/qib/qib_common.h | 4 - drivers/infiniband/hw/qib/qib_qp.c | 134 ----------------------- drivers/infiniband/hw/qib/qib_rc.c | 40 +++---- drivers/infiniband/hw/qib/qib_verbs.h | 4 - drivers/infiniband/sw/rdmavt/Makefile | 4 - drivers/infiniband/sw/rdmavt/rc.c | 190 ++++++++++++++++++++++++++++++++ include/rdma/rdmavt_qp.h | 31 +++++ 13 files changed, 265 insertions(+), 352 deletions(-) create mode 100644 drivers/infiniband/sw/rdmavt/rc.c
drivers/infiniband/hw/hfi1/common.h +0 -4
@@ -331,10 +331,6 @@ struct diag_pkt {#define FULL_MGMT_P_KEY 0xFFFF#define DEFAULT_P_KEY LIM_MGMT_P_KEY -#define HFI1_AETH_CREDIT_SHIFT 24 -#define HFI1_AETH_CREDIT_MASK 0x1F -#define HFI1_AETH_CREDIT_INVAL 0x1F -#define HFI1_MSN_MASK 0xFFFFFF#define HFI1_FECN_SHIFT 31#define HFI1_FECN_MASK 1#define HFI1_FECN_SMASK BIT(HFI1_FECN_SHIFT)
drivers/infiniband/hw/hfi1/qp.c +0 -137
155 lines not on this page · raw message
drivers/infiniband/hw/hfi1/qp.h +0 -16
@@ -71,14 +71,6 @@ static inline void clear_ahg(struct rvt_qp *qp) }/** -* hfi1_compute_aeth - compute the AETH (syndrome + MSN) -* @qp: the queue pair to compute the AETH for -* -* Returns the AETH. -*/ -__be32 hfi1_compute_aeth (struct rvt_qp *qp); - -/** * hfi1_create_qp - create a queue pair for a device * @ibpd: the protection domain who's device we create the queue pair for * @init_attr: the attributes of the queue pair
@@ -91,14 +83,6 @@ static inline void clear_ahg(struct rvt_qp *qp)struct ib_qp *hfi1_create_qp (struct ib_pd *ibpd,struct ib_qp_init_attr *init_attr,struct ib_udata *udata); -/** -* hfi1_get_credit - flush the send work queue of a QP -* @qp: the qp who's send work queue to flush -* @aeth: the Acknowledge Extended Transport Header -* -* The QP s_lock should be held. -*/ -void hfi1_get_credit (struct rvt_qp *qp,u32 aeth);/** * hfi1_qp_wakeup - wake up on the indicated event
drivers/infiniband/hw/hfi1/rc.c +20 -20
@@ -284,7 +284,7 @@ static int make_rc_ack(struct hfi1_ibdev *dev, struct rvt_qp *qp, qp->s_ack_state =OP (RDMA_READ_RESPONSE_ONLY ); e->sent =1 ; } - ohdr->u.aeth =hfi1_compute_aeth (qp); + ohdr->u.aeth =rvt_compute_aeth (qp); hwords++; qp->s_ack_rdma_psn = e->psn; bth2 =mask_psn (qp->s_ack_rdma_psn++);
@@ -293,7 +293,7 @@ static int make_rc_ack(struct hfi1_ibdev *dev, struct rvt_qp *qp, ps->s_txreq->ss =NULL ; len =0 ; qp->s_ack_state =OP (ATOMIC_ACKNOWLEDGE ); - ohdr->u.at.aeth =hfi1_compute_aeth (qp); + ohdr->u.at.aeth =rvt_compute_aeth (qp);ib_u64_put (e->atomic_data, &ohdr->u.at.atomic_ack_eth); hwords +=sizeof (ohdr->u.at) /sizeof (u32); bth2 =mask_psn (e->psn);
@@ -315,7 +315,7 @@ static int make_rc_ack(struct hfi1_ibdev *dev, struct rvt_qp *qp, len = pmtu; middle =HFI1_CAP_IS_KSET (SDMA_AHG ); }else { - ohdr->u.aeth =hfi1_compute_aeth (qp); + ohdr->u.aeth =rvt_compute_aeth (qp); hwords++; qp->s_ack_state =OP (RDMA_READ_RESPONSE_LAST ); e = &qp->s_ack_queue[qp->s_tail_ack_queue];
@@ -338,11 +338,11 @@ static int make_rc_ack(struct hfi1_ibdev *dev, struct rvt_qp *qp, ps->s_txreq->ss =NULL ;if (qp->s_nak_state) ohdr->u.aeth = -cpu_to_be32 ((qp->r_msn &HFI1_MSN_MASK ) | +cpu_to_be32 ((qp->r_msn &RVT_MSN_MASK ) | (qp->s_nak_state << -HFI1_AETH_CREDIT_SHIFT )); +RVT_AETH_CREDIT_SHIFT ));else - ohdr->u.aeth =hfi1_compute_aeth (qp); + ohdr->u.aeth =rvt_compute_aeth (qp); hwords++; len =0 ; bth0 =OP (ACKNOWLEDGE ) <<24 ;
@@ -519,7 +519,7 @@ int hfi1_make_rc_req(struct rvt_qp *qp, struct hfi1_pkt_state *ps)case IB_WR_SEND_WITH_INV :/* If no credit, return. */ if (!(qp->s_flags &RVT_S_UNLIMITED_CREDIT ) && -cmp_msn (wqe->ssn, qp->s_lsn +1 ) >0 ) { +rvt_cmp_msn (wqe->ssn, qp->s_lsn +1 ) >0 ) { qp->s_flags |=RVT_S_WAIT_SSN_CREDIT ; goto bail; }
@@ -556,7 +556,7 @@ int hfi1_make_rc_req(struct rvt_qp *qp, struct hfi1_pkt_state *ps)case IB_WR_RDMA_WRITE_WITH_IMM :/* If no credit, return. */ if (!(qp->s_flags &RVT_S_UNLIMITED_CREDIT ) && -cmp_msn (wqe->ssn, qp->s_lsn +1 ) >0 ) { +rvt_cmp_msn (wqe->ssn, qp->s_lsn +1 ) >0 ) { qp->s_flags |=RVT_S_WAIT_SSN_CREDIT ; goto bail; }
@@ -885,11 +885,11 @@ void hfi1_send_rc_ack(struct hfi1_ctxtdata *rcd, struct rvt_qp *qp,if (qp->s_mig_state ==IB_MIG_MIGRATED ) bth0 |=IB_BTH_MIG_REQ ;if (qp->r_nak_state) - ohdr->u.aeth =cpu_to_be32 ((qp->r_msn &HFI1_MSN_MASK ) | + ohdr->u.aeth =cpu_to_be32 ((qp->r_msn &RVT_MSN_MASK ) | (qp->r_nak_state << -HFI1_AETH_CREDIT_SHIFT )); +RVT_AETH_CREDIT_SHIFT ));else - ohdr->u.aeth =hfi1_compute_aeth (qp); + ohdr->u.aeth =rvt_compute_aeth (qp); sc5 = ibp->sl_to_sc[qp->remote_ah_attr.sl];/* set PBC_DC_INFO bit (aka SC[4]) in pbc_flags */ pbc_flags |= ((!!(sc5 &0x10 )) <<PBC_DC_INFO_SHIFT );
@@ -1323,7 +1323,7 @@ static int do_rc_ack(struct rvt_qp *qp, u32 aeth, u32 psn, int opcode,* request but will include an ACK'ed request(s). */ ack_psn = psn; -if (aeth >>29 ) +if (aeth >>RVT_AETH_NAK_SHIFT ) ack_psn--; wqe =rvt_get_swqe_ptr (qp, qp->s_acked); ibp =rcd_to_iport (rcd);
@@ -1403,7 +1403,7 @@ static int do_rc_ack(struct rvt_qp *qp, u32 aeth, u32 psn, int opcode,break ; } -switch (aeth >>29 ) { +switch (aeth >>RVT_AETH_NAK_SHIFT ) {case 0 :/* ACK */ this_cpu_inc (*ibp->rvp.rc_acks);if (qp->s_acked != qp->s_tail) {
@@ -1430,7 +1430,7 @@ static int do_rc_ack(struct rvt_qp *qp, u32 aeth, u32 psn, int opcode, qp->s_flags &= ~RVT_S_WAIT_ACK ;hfi1_schedule_send (qp); } -hfi1_get_credit (qp, aeth); +rvt_get_credit (qp, aeth); qp->s_rnr_retry = qp->s_rnr_retry_cnt; qp->s_retry = qp->s_retry_cnt;update_last_psn (qp, psn);
@@ -1459,8 +1459,8 @@ static int do_rc_ack(struct rvt_qp *qp, u32 aeth, u32 psn, int opcode, qp->s_flags &= ~(RVT_S_WAIT_SSN_CREDIT |RVT_S_WAIT_ACK );hfi1_stop_rc_timers (qp); to = - ib_hfi1_rnr_table[(aeth >>HFI1_AETH_CREDIT_SHIFT ) & -HFI1_AETH_CREDIT_MASK ]; + ib_hfi1_rnr_table[(aeth >>RVT_AETH_CREDIT_SHIFT ) & +RVT_AETH_CREDIT_MASK ];hfi1_add_rnr_timer (qp, to);return 0 ;
@@ -1469,8 +1469,8 @@ static int do_rc_ack(struct rvt_qp *qp, u32 aeth, u32 psn, int opcode, goto bail_stop;/* The last valid PSN is the previous PSN. */ update_last_psn (qp, psn -1 ); -switch ((aeth >>HFI1_AETH_CREDIT_SHIFT ) & -HFI1_AETH_CREDIT_MASK ) { +switch ((aeth >>RVT_AETH_CREDIT_SHIFT ) & +RVT_AETH_CREDIT_MASK ) {case 0 :/* PSN sequence error */ ibp->rvp.n_seq_naks++;/*
@@ -1600,8 +1600,8 @@ static void rc_rcv_resp(struct hfi1_ibport *ibp,/* Update credits for "ghost" ACKs */ if (diff ==0 && opcode ==OP (ACKNOWLEDGE )) { aeth =be32_to_cpu (ohdr->u.aeth); -if ((aeth >>29 ) ==0 ) -hfi1_get_credit (qp, aeth); +if ((aeth >>RVT_AETH_NAK_SHIFT ) ==0 ) +rvt_get_credit (qp, aeth); } goto ack_done; }
drivers/infiniband/hw/hfi1/trace.c +2 -2
@@ -130,14 +130,14 @@ u8 ibhdr_exhdr_len(struct ib_header *hdr)case OP (RC ,ACKNOWLEDGE ):trace_seq_printf (p,AETH_PRN ,be32_to_cpu (eh->aeth) >>24 ,parse_syndrome (be32_to_cpu (eh->aeth) >>24 ), -be32_to_cpu (eh->aeth) &HFI1_MSN_MASK ); +be32_to_cpu (eh->aeth) &RVT_MSN_MASK );break ;/* aeth + atomicacketh */ case OP (RC ,ATOMIC_ACKNOWLEDGE ):trace_seq_printf (p,AETH_PRN " " ATOMICACKETH_PRN ,be32_to_cpu (eh->at.aeth) >>24 ,parse_syndrome (be32_to_cpu (eh->at.aeth) >>24 ), -be32_to_cpu (eh->at.aeth) &HFI1_MSN_MASK , +be32_to_cpu (eh->at.aeth) &RVT_MSN_MASK ,ib_u64_get (&eh->at.atomic_ack_eth));break ;/* atomiceth */
drivers/infiniband/hw/hfi1/verbs.h +0 -9
@@ -260,15 +260,6 @@ int hfi1_process_mad(struct ib_device *ibdev, int mad_flags, u8 port,#define PSN_MODIFY_MASK 0xFFFFFF/* -* Compare the lower 24 bits of the msn values. -* Returns an integer <, ==, or > than zero. -*/ -static inline int cmp_msn (u32 a,u32 b) -{ -return (((int )a) - ((int )b)) <<8 ; -} - -/* * Compare two PSNs * Returns an integer <, ==, or > than zero. */
drivers/infiniband/hw/qib/qib_common.h +0 -4
@@ -742,11 +742,7 @@ struct qib_tid_session_member {#define SIZE_OF_CRC 1#define QIB_DEFAULT_P_KEY 0xFFFF -#define QIB_AETH_CREDIT_SHIFT 24 -#define QIB_AETH_CREDIT_MASK 0x1F -#define QIB_AETH_CREDIT_INVAL 0x1F#define QIB_PSN_MASK 0xFFFFFF -#define QIB_MSN_MASK 0xFFFFFF#define QIB_EAGER_TID_ID QLOGIC_IB_I_TID_MASK#define QIB_MULTICAST_QPN 0xFFFFFF
drivers/infiniband/hw/qib/qib_qp.c +0 -134
152 lines not on this page · raw message
drivers/infiniband/hw/qib/qib_rc.c +20 -20
@@ -144,7 +144,7 @@ static int qib_make_rc_ack(struct qib_ibdev *dev, struct rvt_qp *qp, qp->s_ack_state =OP (RDMA_READ_RESPONSE_ONLY ); e->sent =1 ; } - ohdr->u.aeth =qib_compute_aeth (qp); + ohdr->u.aeth =rvt_compute_aeth (qp); hwords++; qp->s_ack_rdma_psn = e->psn; bth2 = qp->s_ack_rdma_psn++ &QIB_PSN_MASK ;
@@ -153,7 +153,7 @@ static int qib_make_rc_ack(struct qib_ibdev *dev, struct rvt_qp *qp, qp->s_cur_sge =NULL ; len =0 ; qp->s_ack_state =OP (ATOMIC_ACKNOWLEDGE ); - ohdr->u.at.aeth =qib_compute_aeth (qp); + ohdr->u.at.aeth =rvt_compute_aeth (qp);ib_u64_put (e->atomic_data, &ohdr->u.at.atomic_ack_eth); hwords +=sizeof (ohdr->u.at) /sizeof (u32); bth2 = e->psn &QIB_PSN_MASK ;
@@ -174,7 +174,7 @@ static int qib_make_rc_ack(struct qib_ibdev *dev, struct rvt_qp *qp,if (len > pmtu) len = pmtu;else { - ohdr->u.aeth =qib_compute_aeth (qp); + ohdr->u.aeth =rvt_compute_aeth (qp); hwords++; qp->s_ack_state =OP (RDMA_READ_RESPONSE_LAST ); e = &qp->s_ack_queue[qp->s_tail_ack_queue];
@@ -197,11 +197,11 @@ static int qib_make_rc_ack(struct qib_ibdev *dev, struct rvt_qp *qp, qp->s_cur_sge =NULL ;if (qp->s_nak_state) ohdr->u.aeth = -cpu_to_be32 ((qp->r_msn &QIB_MSN_MASK ) | +cpu_to_be32 ((qp->r_msn &RVT_MSN_MASK ) | (qp->s_nak_state << -QIB_AETH_CREDIT_SHIFT )); +RVT_AETH_CREDIT_SHIFT ));else - ohdr->u.aeth =qib_compute_aeth (qp); + ohdr->u.aeth =rvt_compute_aeth (qp); hwords++; len =0 ; bth0 =OP (ACKNOWLEDGE ) <<24 ;
@@ -331,7 +331,7 @@ int qib_make_rc_req(struct rvt_qp *qp, unsigned long *flags)case IB_WR_SEND_WITH_IMM :/* If no credit, return. */ if (!(qp->s_flags &RVT_S_UNLIMITED_CREDIT ) && -qib_cmp24 (wqe->ssn, qp->s_lsn +1 ) >0 ) { +rvt_cmp_msn (wqe->ssn, qp->s_lsn +1 ) >0 ) { qp->s_flags |=RVT_S_WAIT_SSN_CREDIT ; goto bail; }
@@ -362,7 +362,7 @@ int qib_make_rc_req(struct rvt_qp *qp, unsigned long *flags)case IB_WR_RDMA_WRITE_WITH_IMM :/* If no credit, return. */ if (!(qp->s_flags &RVT_S_UNLIMITED_CREDIT ) && -qib_cmp24 (wqe->ssn, qp->s_lsn +1 ) >0 ) { +rvt_cmp_msn (wqe->ssn, qp->s_lsn +1 ) >0 ) { qp->s_flags |=RVT_S_WAIT_SSN_CREDIT ; goto bail; }
@@ -658,11 +658,11 @@ void qib_send_rc_ack(struct rvt_qp *qp)if (qp->s_mig_state ==IB_MIG_MIGRATED ) bth0 |=IB_BTH_MIG_REQ ;if (qp->r_nak_state) - ohdr->u.aeth =cpu_to_be32 ((qp->r_msn &QIB_MSN_MASK ) | + ohdr->u.aeth =cpu_to_be32 ((qp->r_msn &RVT_MSN_MASK ) | (qp->r_nak_state << -QIB_AETH_CREDIT_SHIFT )); +RVT_AETH_CREDIT_SHIFT ));else - ohdr->u.aeth =qib_compute_aeth (qp); + ohdr->u.aeth =rvt_compute_aeth (qp); lrh0 |= ibp->sl_to_vl[qp->remote_ah_attr.sl] <<12 | qp->remote_ah_attr.sl <<4 ; hdr.lrh[0 ] =cpu_to_be16 (lrh0);
@@ -1098,7 +1098,7 @@ static int do_rc_ack(struct rvt_qp *qp, u32 aeth, u32 psn, int opcode,* request but will include an ACK'ed request(s). */ ack_psn = psn; -if (aeth >>29 ) +if (aeth >>RVT_AETH_NAK_SHIFT ) ack_psn--; wqe =rvt_get_swqe_ptr (qp, qp->s_acked); ibp =to_iport (qp->ibqp.device, qp->port_num);
@@ -1178,7 +1178,7 @@ static int do_rc_ack(struct rvt_qp *qp, u32 aeth, u32 psn, int opcode,break ; } -switch (aeth >>29 ) { +switch (aeth >>RVT_AETH_NAK_SHIFT ) {case 0 :/* ACK */ this_cpu_inc (*ibp->rvp.rc_acks);if (qp->s_acked != qp->s_tail) {
@@ -1201,7 +1201,7 @@ static int do_rc_ack(struct rvt_qp *qp, u32 aeth, u32 psn, int opcode, qp->s_flags &= ~RVT_S_WAIT_ACK ;qib_schedule_send (qp); } -qib_get_credit (qp, aeth); +rvt_get_credit (qp, aeth); qp->s_rnr_retry = qp->s_rnr_retry_cnt; qp->s_retry = qp->s_retry_cnt;update_last_psn (qp, psn);
@@ -1232,8 +1232,8 @@ static int do_rc_ack(struct rvt_qp *qp, u32 aeth, u32 psn, int opcode, qp->s_flags |=RVT_S_WAIT_RNR ; qp->s_timer.function = qib_rc_rnr_retry; qp->s_timer.expires = jiffies +usecs_to_jiffies ( - ib_qib_rnr_table[(aeth >>QIB_AETH_CREDIT_SHIFT ) & -QIB_AETH_CREDIT_MASK ]); + ib_qib_rnr_table[(aeth >>RVT_AETH_CREDIT_SHIFT ) & +RVT_AETH_CREDIT_MASK ]);add_timer (&qp->s_timer); goto bail;
@@ -1242,8 +1242,8 @@ static int do_rc_ack(struct rvt_qp *qp, u32 aeth, u32 psn, int opcode, goto bail;/* The last valid PSN is the previous PSN. */ update_last_psn (qp, psn -1 ); -switch ((aeth >>QIB_AETH_CREDIT_SHIFT ) & -QIB_AETH_CREDIT_MASK ) { +switch ((aeth >>RVT_AETH_CREDIT_SHIFT ) & +RVT_AETH_CREDIT_MASK ) {case 0 :/* PSN sequence error */ ibp->rvp.n_seq_naks++;/*
@@ -1400,8 +1400,8 @@ static void qib_rc_rcv_resp(struct qib_ibport *ibp,/* Update credits for "ghost" ACKs */ if (diff ==0 && opcode ==OP (ACKNOWLEDGE )) { aeth =be32_to_cpu (ohdr->u.aeth); -if ((aeth >>29 ) ==0 ) -qib_get_credit (qp, aeth); +if ((aeth >>RVT_AETH_NAK_SHIFT ) ==0 ) +rvt_get_credit (qp, aeth); } goto ack_done; }
drivers/infiniband/hw/qib/qib_verbs.h +0 -4
@@ -270,8 +270,6 @@ int qib_snapshot_counters(struct qib_pportdata *ppd, u64 *swords,int qib_get_counters (struct qib_pportdata *ppd,struct qib_verbs_counters *cntrs); -__be32 qib_compute_aeth (struct rvt_qp *qp); -/* * Functions provided by qib driver for rdmavt to use */
@@ -294,8 +292,6 @@ int qib_alloc_qpn(struct rvt_dev_info *rdi, struct rvt_qpn_table *qpt,#endif -void qib_get_credit (struct rvt_qp *qp,u32 aeth); -unsigned qib_pkt_delay (u32 plen,u8 snd_mult,u8 rcv_mult);void qib_verbs_sdma_desc_avail (struct qib_pportdata *ppd,unsigned avail);
drivers/infiniband/sw/rdmavt/Makefile +2 -2
@@ -7,7 +7,7 @@ # obj-$(CONFIG_INFINIBAND_RDMAVT) += rdmavt.o -rdmavt-y := vt.o ah.o cq.o dma.o mad.o mcast.o mmap.o mr.o pd.o qp.o srq.o \ - trace.o +rdmavt-y := vt.o ah.o cq.o dma.o mad.o mcast.o mmap.o mr.o pd.o qp.o \ + rc.o srq.o trace.o CFLAGS_trace.o = -I$(src)
drivers/infiniband/sw/rdmavt/rc.c new file +190 -0
190 lines not on this page · raw message
include/rdma/rdmavt_qp.h +31 -0
@@ -582,6 +582,37 @@ static inline void rvt_qp_swqe_complete( } } +#define RVT_AETH_CREDIT_SHIFT 24 +#define RVT_AETH_CREDIT_MASK 0x1F +#define RVT_AETH_NAK_SHIFT 29 +#define RVT_MSN_MASK 0xFFFFFF + +/* +* Compare the lower 24 bits of the msn values. +* Returns an integer <, ==, or > than zero. +*/ +static inline int rvt_cmp_msn (u32 a,u32 b) +{ +return (((int )a) - ((int )b)) <<8 ; +} + +/** +* rvt_compute_aeth - compute the AETH (syndrome + MSN) +* @qp: the queue pair to compute the AETH for +* +* Returns the AETH. +*/ +__be32 rvt_compute_aeth (struct rvt_qp *qp); + +/** +* rvt_get_credit - flush the send work queue of a QP +* @qp: the qp who's send work queue to flush +* @aeth: the Acknowledge Extended Transport Header +* +* The QP s_lock should be held. +*/ +void rvt_get_credit (struct rvt_qp *qp,u32 aeth); +/** * @qp - the qp pair * @len - the length
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
PATCH 14/27 IB/rdmavt: Adding timer logic to rdmavt
In reply to a message not in the archiveFrom: Venkata Sandeep Dhanalakota <[email protected]>
To move common code across target to rdmavt for code reuse.
--- drivers/infiniband/sw/rdmavt/qp.c | 183 +++++++++++++++++++++++++++++++++++++ include/rdma/rdma_vt.h | 19 ++++ include/rdma/rdmavt_qp.h | 6 + 3 files changed, 206 insertions(+), 2 deletions(-)
drivers/infiniband/sw/rdmavt/qp.c +181 -2
@@ -55,6 +55,46 @@#include "vt.h" #include "trace.h" +static void rvt_rc_timeout (unsigned long arg); + +/* +* Convert the AETH RNR timeout code into the number of microseconds. +*/ +static const u32 ib_rvt_rnr_table[32 ] = { +655360 ,/* 00: 655.36 */ +10 ,/* 01: .01 */ +20 ,/* 02 .02 */ +30 ,/* 03: .03 */ +40 ,/* 04: .04 */ +60 ,/* 05: .06 */ +80 ,/* 06: .08 */ +120 ,/* 07: .12 */ +160 ,/* 08: .16 */ +240 ,/* 09: .24 */ +320 ,/* 0A: .32 */ +480 ,/* 0B: .48 */ +640 ,/* 0C: .64 */ +960 ,/* 0D: .96 */ +1280 ,/* 0E: 1.28 */ +1920 ,/* 0F: 1.92 */ +2560 ,/* 10: 2.56 */ +3840 ,/* 11: 3.84 */ +5120 ,/* 12: 5.12 */ +7680 ,/* 13: 7.68 */ +10240 ,/* 14: 10.24 */ +15360 ,/* 15: 15.36 */ +20480 ,/* 16: 20.48 */ +30720 ,/* 17: 30.72 */ +40960 ,/* 18: 40.96 */ +61440 ,/* 19: 61.44 */ +81920 ,/* 1A: 81.92 */ +122880 ,/* 1B: 122.88 */ +163840 ,/* 1C: 163.84 */ +245760 ,/* 1D: 245.76 */ +327680 ,/* 1E: 327.68 */ +491520 /* 1F: 491.52 */ +}; +/* * Note that it is OK to post send work requests in the SQE and ERR * states; rvt_do_send() will process them and generate error
@@ -200,7 +240,8 @@ int rvt_driver_qp_init(struct rvt_dev_info *rdi)if (!rdi->driver_f.free_all_qps || !rdi->driver_f.qp_priv_alloc || !rdi->driver_f.qp_priv_free || - !rdi->driver_f.notify_qp_reset) + !rdi->driver_f.notify_qp_reset || + !rdi->driver_f.notify_restart_rc)return -EINVAL ;/* allocate parent object */
@@ -587,6 +628,7 @@ static void rvt_reset_qp(struct rvt_dev_info *rdi, struct rvt_qp *qp,/* Let drivers flush their waitlist */ rdi->driver_f.flush_qp_waiters (qp); +rvt_stop_rc_timers (qp); qp->s_flags &= ~(RVT_S_TIMER |RVT_S_ANY_WAIT );spin_unlock (&qp->s_lock);spin_unlock (&qp->s_hlock);
@@ -594,7 +636,7 @@ static void rvt_reset_qp(struct rvt_dev_info *rdi, struct rvt_qp *qp,/* Stop the send queue and the retry timer */ rdi->driver_f.stop_send_queue (qp); - +rvt_del_timers_sync (qp);/* Wait for things to stop */ rdi->driver_f.quiesce_qp (qp);
@@ -730,6 +772,11 @@ struct ib_qp *rvt_create_qp(struct ib_pd *ibpd,if (!qp->s_ack_queue) goto bail_qp; } +/* initialize timers needed for rc qp */ +setup_timer (&qp->s_timer, rvt_rc_timeout, (unsigned long )qp); +hrtimer_init (&qp->s_rnr_timer,CLOCK_MONOTONIC , +HRTIMER_MODE_REL ); + qp->s_rnr_timer.function = rvt_rc_rnr_retry;/* * Driver needs to set up it's private QP structure and do any
@@ -1906,3 +1953,135 @@ void rvt_rc_error(struct rvt_qp *qp, enum ib_wc_status err) } }EXPORT_SYMBOL (rvt_rc_error); + +static inline unsigned long rvt_aeth_to_usec (u32 aeth) +{ +return ib_rvt_rnr_table[(aeth >>RVT_AETH_CREDIT_SHIFT ) & +RVT_AETH_CREDIT_MASK ]; +} + +/* +* rvt_add_retry_timer - add/start a retry timer +* @qp - the QP +* add a retry timer on the QP +*/ +void rvt_add_retry_timer (struct rvt_qp *qp) +{ +struct ib_qp *ibqp = &qp->ibqp; +struct rvt_dev_info *rdi =ib_to_rvt (ibqp->device); + +lockdep_assert_held (&qp->s_lock); + qp->s_flags |=RVT_S_TIMER ; +/* 4.096 usec. * (1 << qp->timeout) */ + qp->s_timer.expires = jiffies + qp->timeout_jiffies + + rdi->busy_jiffies; +add_timer (&qp->s_timer); +} +EXPORT_SYMBOL (rvt_add_retry_timer); + +/** +* rvt_add_rnr_timer - add/start an rnr timer +* @qp - the QP +* @aeth - aeth of RNR timeout, simulated aeth for loopback +* add an rnr timer on the QP +*/ +void rvt_add_rnr_timer (struct rvt_qp *qp,u32 aeth) +{ +u32 to; + +lockdep_assert_held (&qp->s_lock); + qp->s_flags |=RVT_S_WAIT_RNR ; + to =rvt_aeth_to_usec (aeth); +hrtimer_start (&qp->s_rnr_timer, +ns_to_ktime (1000 * to),HRTIMER_MODE_REL ); +} +EXPORT_SYMBOL (rvt_add_rnr_timer); + +/** +* rvt_stop_rc_timers - stop all timers +* @qp - the QP +* stop any pending timers +*/ +void rvt_stop_rc_timers (struct rvt_qp *qp) +{ +lockdep_assert_held (&qp->s_lock); +/* Remove QP from all timers */ +if (qp->s_flags & (RVT_S_TIMER |RVT_S_WAIT_RNR )) { + qp->s_flags &= ~(RVT_S_TIMER |RVT_S_WAIT_RNR ); +del_timer (&qp->s_timer); +hrtimer_try_to_cancel (&qp->s_rnr_timer); + } +} +EXPORT_SYMBOL (rvt_stop_rc_timers); + +/** +* rvt_stop_rnr_timer - stop an rnr timer +* @qp - the QP +* +* stop an rnr timer and return if the timer +* had been pending. +*/ +static int rvt_stop_rnr_timer (struct rvt_qp *qp) +{ +int rval =0 ; + +lockdep_assert_held (&qp->s_lock); +/* Remove QP from rnr timer */ +if (qp->s_flags &RVT_S_WAIT_RNR ) { + qp->s_flags &= ~RVT_S_WAIT_RNR ; + rval =hrtimer_try_to_cancel (&qp->s_rnr_timer); + } +return rval; +} + +/** +* rvt_del_timers_sync - wait for any timeout routines to exit +* @qp - the QP +*/ +void rvt_del_timers_sync (struct rvt_qp *qp) +{ +del_timer_sync (&qp->s_timer); +hrtimer_cancel (&qp->s_rnr_timer); +} +EXPORT_SYMBOL (rvt_del_timers_sync); + +/** +* This is called from s_timer for missing responses. +*/ +static void rvt_rc_timeout (unsigned long arg) +{ +struct rvt_qp *qp = (struct rvt_qp *)arg; +struct rvt_dev_info *rdi =ib_to_rvt (qp->ibqp.device); +unsigned long flags; + +spin_lock_irqsave (&qp->r_lock, flags); +spin_lock (&qp->s_lock); +if (qp->s_flags &RVT_S_TIMER ) { + qp->s_flags &= ~RVT_S_TIMER ; +del_timer (&qp->s_timer); +if (rdi->driver_f.notify_restart_rc) + rdi->driver_f.notify_restart_rc (qp, + qp->s_last_psn +1 , +1 ); + rdi->driver_f.schedule_send (qp); + } +spin_unlock (&qp->s_lock); +spin_unlock_irqrestore (&qp->r_lock, flags); +} + +/* +* This is called from s_timer for RNR timeouts. +*/ +enum hrtimer_restart rvt_rc_rnr_retry (struct hrtimer *t) +{ + struct rvt_qp *qp = container_of(t, struct rvt_qp, s_rnr_timer); +struct rvt_dev_info *rdi =ib_to_rvt (qp->ibqp.device); +unsigned long flags; + +spin_lock_irqsave (&qp->s_lock, flags); +rvt_stop_rnr_timer (qp); + rdi->driver_f.schedule_send (qp); +spin_unlock_irqrestore (&qp->s_lock, flags); +return HRTIMER_NORESTART ; +} +EXPORT_SYMBOL (rvt_rc_rnr_retry);
include/rdma/rdma_vt.h +19 -0
@@ -335,6 +335,8 @@ struct rvt_driver_provided {/* Notify driver a mad agent has been removed */ void (*notify_free_mad_agent)(struct rvt_dev_info *rdi,int port_idx); +/* Notify driver to restart rc */ +void (*notify_restart_rc)(struct rvt_qp *qp,u32 psn,int wait); };struct rvt_dev_info {
@@ -483,6 +485,23 @@ static inline u16 rvt_get_pkey(struct rvt_dev_info *rdi,return qp; } +/** +* rvt_mod_retry_timer - mod a retry timer +* @qp - the QP +* Modify a potentially already running retry timer +*/ +static inline void rvt_mod_retry_timer (struct rvt_qp *qp) +{ +struct ib_qp *ibqp = &qp->ibqp; +struct rvt_dev_info *rdi =ib_to_rvt (ibqp->device); + +lockdep_assert_held (&qp->s_lock); + qp->s_flags |=RVT_S_TIMER ; +/* 4.096 usec. * (1 << qp->timeout) */ +mod_timer (&qp->s_timer, jiffies + qp->timeout_jiffies + + rdi->busy_jiffies); +} +struct rvt_dev_info *rvt_alloc_device (size_t size,int nports);void rvt_dealloc_device (struct rvt_dev_info *rdi);int rvt_register_device (struct rvt_dev_info *rvd);
include/rdma/rdmavt_qp.h +6 -0
@@ -370,6 +370,7 @@ struct rvt_qp {struct rvt_sge_state s_ack_rdma_sge;struct timer_list s_timer; +struct hrtimer s_rnr_timer;atomic_t local_ops_pending;/* number of fast_reg/local_inv reqs */
@@ -641,5 +642,10 @@ static inline u32 rvt_div_mtu(struct rvt_qp *qp, u32 len)void rvt_comm_est (struct rvt_qp *qp);int rvt_error_qp (struct rvt_qp *qp,enum ib_wc_status err);void rvt_rc_error (struct rvt_qp *qp,enum ib_wc_status err); +enum hrtimer_restart rvt_rc_rnr_retry (struct hrtimer *t); +void rvt_add_rnr_timer (struct rvt_qp *qp,u32 aeth); +void rvt_del_timers_sync (struct rvt_qp *qp); +void rvt_stop_rc_timers (struct rvt_qp *qp); +void rvt_add_retry_timer (struct rvt_qp *qp);#endif /* DEF_RDMAVT_INCQP_H */
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
A message that is not in the archive (sent off-list or not yet received): 20170208132712.16442.57028.stgit-9QXIwq+3FY+1XWohqUldA0EOCMrvLtNR@public.gmane.org
Re: [PATCH 14/27] IB/rdmavt: Adding timer logic to rdmavt
Quoted 105 lines · Dennis Dalessandro
From: Venkata Sandeep Dhanalakota <[email protected]>
To move common code across target to rdmavt for code reuse.
Reviewed-by Mike Marciniszyn <[email protected]>Reviewed-by Brian Welty <[email protected]>Signed-off-by Venkata Sandeep Dhanalakota <[email protected]>Signed-off-by Dennis Dalessandro <[email protected]>--- drivers/infiniband/sw/rdmavt/qp.c | 183 +++++++++++++++++++++++++++++++++++++ include/rdma/rdma_vt.h | 19 ++++ include/rdma/rdmavt_qp.h | 6 + 3 files changed, 206 insertions(+), 2 deletions(-)drivers/infiniband/sw/rdmavt/qp.c +49
-2@@ -55,6 +55,46 @@#include "vt.h" #include "trace.h" +static void rvt_rc_timeout (unsigned long arg); + +/* +* Convert the AETH RNR timeout code into the number of microseconds. +*/ +static const u32 ib_rvt_rnr_table[32 ] = { +655360 ,/* 00: 655.36 */ +10 ,/* 01: .01 */ +20 ,/* 02 .02 */ +30 ,/* 03: .03 */ +40 ,/* 04: .04 */ +60 ,/* 05: .06 */ +80 ,/* 06: .08 */ +120 ,/* 07: .12 */ +160 ,/* 08: .16 */ +240 ,/* 09: .24 */ +320 ,/* 0A: .32 */ +480 ,/* 0B: .48 */ +640 ,/* 0C: .64 */ +960 ,/* 0D: .96 */ +1280 ,/* 0E: 1.28 */ +1920 ,/* 0F: 1.92 */ +2560 ,/* 10: 2.56 */ +3840 ,/* 11: 3.84 */ +5120 ,/* 12: 5.12 */ +7680 ,/* 13: 7.68 */ +10240 ,/* 14: 10.24 */ +15360 ,/* 15: 15.36 */ +20480 ,/* 16: 20.48 */ +30720 ,/* 17: 30.72 */ +40960 ,/* 18: 40.96 */ +61440 ,/* 19: 61.44 */ +81920 ,/* 1A: 81.92 */ +122880 ,/* 1B: 122.88 */ +163840 ,/* 1C: 163.84 */ +245760 ,/* 1D: 245.76 */ +327680 ,/* 1E: 327.68 */ +491520 /* 1F: 491.52 */ +}; +/* * Note that it is OK to post send work requests in the SQE and ERR * states; rvt_do_send() will process them and generate error @@ -200,7 +240,8 @@ int rvt_driver_qp_init(struct rvt_dev_info *rdi)if (!rdi->driver_f.free_all_qps || !rdi->driver_f.qp_priv_alloc || !rdi->driver_f.qp_priv_free || - !rdi->driver_f.notify_qp_reset) + !rdi->driver_f.notify_qp_reset || + !rdi->driver_f.notify_restart_rc)return -EINVAL ;/* allocate parent object */ @@ -587,6 +628,7 @@ static void rvt_reset_qp(struct rvt_dev_info *rdi, struct rvt_qp *qp,/* Let drivers flush their waitlist */ rdi->driver_f.flush_qp_waiters (qp); +rvt_stop_rc_timers (qp); qp->s_flags &= ~(RVT_S_TIMER |RVT_S_ANY_WAIT );spin_unlock (&qp->s_lock);spin_unlock (&qp->s_hlock);@@ -594,7 +636,7 @@ static void rvt_reset_qp(struct rvt_dev_info *rdi, struct rvt_qp *qp,/* Stop the send queue and the retry timer */ rdi->driver_f.stop_send_queue (qp); - +rvt_del_timers_sync (qp);/* Wait for things to stop */ rdi->driver_f.quiesce_qp (qp);@@ -730,6 +772,11 @@ struct ib_qp *rvt_create_qp(struct ib_pd *ibpd,if (!qp->s_ack_queue) goto bail_qp; } +/* initialize timers needed for rc qp */ +setup_timer (&qp->s_timer, rvt_rc_timeout, (unsigned long )qp); +hrtimer_init (&qp->s_rnr_timer,CLOCK_MONOTONIC , +HRTIMER_MODE_REL ); + qp->s_rnr_timer.function = rvt_rc_rnr_retry;/* * Driver needs to set up it's private QP structure and do any
drivers/infiniband/sw/rdmavt/qp.c void rvt_rc_error(struct rvt_qp *qp, enum ib_wc_status err) · line 1961
Quoted 9 lines
drivers/infiniband/sw/rdmavt/qp.c +6
-0@@ -1906,3 +1953,135 @@ void rvt_rc_error(struct rvt_qp *qp, enum ib_wc_status err) } }EXPORT_SYMBOL (rvt_rc_error); + +static inline unsigned long rvt_aeth_to_usec (u32 aeth) +{ +return ib_rvt_rnr_table[(aeth >>RVT_AETH_CREDIT_SHIFT ) & +RVT_AETH_CREDIT_MASK ]; +}
No need to inline this function.
http://lxr.free-electrons.com/source/Documentation/CodingStyle#L861
Quoted 191 lines
drivers/infiniband/sw/rdmavt/qp.c +126
-0+ +/* +* rvt_add_retry_timer - add/start a retry timer +* @qp - the QP +* add a retry timer on the QP +*/ +void rvt_add_retry_timer (struct rvt_qp *qp) +{ +struct ib_qp *ibqp = &qp->ibqp; +struct rvt_dev_info *rdi =ib_to_rvt (ibqp->device); + +lockdep_assert_held (&qp->s_lock); + qp->s_flags |=RVT_S_TIMER ; +/* 4.096 usec. * (1 << qp->timeout) */ + qp->s_timer.expires = jiffies + qp->timeout_jiffies + + rdi->busy_jiffies; +add_timer (&qp->s_timer); +} +EXPORT_SYMBOL (rvt_add_retry_timer); + +/** +* rvt_add_rnr_timer - add/start an rnr timer +* @qp - the QP +* @aeth - aeth of RNR timeout, simulated aeth for loopback +* add an rnr timer on the QP +*/ +void rvt_add_rnr_timer (struct rvt_qp *qp,u32 aeth) +{ +u32 to; + +lockdep_assert_held (&qp->s_lock); + qp->s_flags |=RVT_S_WAIT_RNR ; + to =rvt_aeth_to_usec (aeth); +hrtimer_start (&qp->s_rnr_timer, +ns_to_ktime (1000 * to),HRTIMER_MODE_REL ); +} +EXPORT_SYMBOL (rvt_add_rnr_timer); + +/** +* rvt_stop_rc_timers - stop all timers +* @qp - the QP +* stop any pending timers +*/ +void rvt_stop_rc_timers (struct rvt_qp *qp) +{ +lockdep_assert_held (&qp->s_lock); +/* Remove QP from all timers */ +if (qp->s_flags & (RVT_S_TIMER |RVT_S_WAIT_RNR )) { + qp->s_flags &= ~(RVT_S_TIMER |RVT_S_WAIT_RNR ); +del_timer (&qp->s_timer); +hrtimer_try_to_cancel (&qp->s_rnr_timer); + } +} +EXPORT_SYMBOL (rvt_stop_rc_timers); + +/** +* rvt_stop_rnr_timer - stop an rnr timer +* @qp - the QP +* +* stop an rnr timer and return if the timer +* had been pending. +*/ +static int rvt_stop_rnr_timer (struct rvt_qp *qp) +{ +int rval =0 ; + +lockdep_assert_held (&qp->s_lock); +/* Remove QP from rnr timer */ +if (qp->s_flags &RVT_S_WAIT_RNR ) { + qp->s_flags &= ~RVT_S_WAIT_RNR ; + rval =hrtimer_try_to_cancel (&qp->s_rnr_timer); + } +return rval; +} + +/** +* rvt_del_timers_sync - wait for any timeout routines to exit +* @qp - the QP +*/ +void rvt_del_timers_sync (struct rvt_qp *qp) +{ +del_timer_sync (&qp->s_timer); +hrtimer_cancel (&qp->s_rnr_timer); +} +EXPORT_SYMBOL (rvt_del_timers_sync); + +/** +* This is called from s_timer for missing responses. +*/ +static void rvt_rc_timeout (unsigned long arg) +{ +struct rvt_qp *qp = (struct rvt_qp *)arg; +struct rvt_dev_info *rdi =ib_to_rvt (qp->ibqp.device); +unsigned long flags; + +spin_lock_irqsave (&qp->r_lock, flags); +spin_lock (&qp->s_lock); +if (qp->s_flags &RVT_S_TIMER ) { + qp->s_flags &= ~RVT_S_TIMER ; +del_timer (&qp->s_timer); +if (rdi->driver_f.notify_restart_rc) + rdi->driver_f.notify_restart_rc (qp, + qp->s_last_psn +1 , +1 ); + rdi->driver_f.schedule_send (qp); + } +spin_unlock (&qp->s_lock); +spin_unlock_irqrestore (&qp->r_lock, flags); +} + +/* +* This is called from s_timer for RNR timeouts. +*/ +enum hrtimer_restart rvt_rc_rnr_retry (struct hrtimer *t) +{ + struct rvt_qp *qp = container_of(t, struct rvt_qp, s_rnr_timer); +struct rvt_dev_info *rdi =ib_to_rvt (qp->ibqp.device); +unsigned long flags; + +spin_lock_irqsave (&qp->s_lock, flags); +rvt_stop_rnr_timer (qp); + rdi->driver_f.schedule_send (qp); +spin_unlock_irqrestore (&qp->s_lock, flags); +return HRTIMER_NORESTART ; +} +EXPORT_SYMBOL (rvt_rc_rnr_retry);include/rdma/rdma_vt.h +19
-0@@ -335,6 +335,8 @@ struct rvt_driver_provided {/* Notify driver a mad agent has been removed */ void (*notify_free_mad_agent)(struct rvt_dev_info *rdi,int port_idx); +/* Notify driver to restart rc */ +void (*notify_restart_rc)(struct rvt_qp *qp,u32 psn,int wait); };struct rvt_dev_info {@@ -483,6 +485,23 @@ static inline u16 rvt_get_pkey(struct rvt_dev_info *rdi,return qp; } +/** +* rvt_mod_retry_timer - mod a retry timer +* @qp - the QP +* Modify a potentially already running retry timer +*/ +static inline void rvt_mod_retry_timer (struct rvt_qp *qp) +{ +struct ib_qp *ibqp = &qp->ibqp; +struct rvt_dev_info *rdi =ib_to_rvt (ibqp->device); + +lockdep_assert_held (&qp->s_lock); + qp->s_flags |=RVT_S_TIMER ; +/* 4.096 usec. * (1 << qp->timeout) */ +mod_timer (&qp->s_timer, jiffies + qp->timeout_jiffies + + rdi->busy_jiffies); +} +struct rvt_dev_info *rvt_alloc_device (size_t size,int nports);void rvt_dealloc_device (struct rvt_dev_info *rdi);int rvt_register_device (struct rvt_dev_info *rvd);include/rdma/rdmavt_qp.h +6
-1@@ -370,6 +370,7 @@ struct rvt_qp {struct rvt_sge_state s_ack_rdma_sge;struct timer_list s_timer; +struct hrtimer s_rnr_timer;atomic_t local_ops_pending;/* number of fast_reg/local_inv reqs */ @@ -641,5 +642,10 @@ static inline u32 rvt_div_mtu(struct rvt_qp *qp, u32 len)void rvt_comm_est (struct rvt_qp *qp);int rvt_error_qp (struct rvt_qp *qp,enum ib_wc_status err);void rvt_rc_error (struct rvt_qp *qp,enum ib_wc_status err); +enum hrtimer_restart rvt_rc_rnr_retry (struct hrtimer *t); +void rvt_add_rnr_timer (struct rvt_qp *qp,u32 aeth); +void rvt_del_timers_sync (struct rvt_qp *qp); +void rvt_stop_rc_timers (struct rvt_qp *qp); +void rvt_add_retry_timer (struct rvt_qp *qp);#endif /* DEF_RDMAVT_INCQP_H */ --To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
PATCH 15/27 IB/hfi1: Use new rdmavt timers
In reply to a message not in the archiveFrom: Venkata Sandeep Dhanalakota <[email protected]>
Reduce hfi1 code footprint by using the rdmavt timers.
--- drivers/infiniband/hw/hfi1/qp.c | 4 - drivers/infiniband/hw/hfi1/rc.c | 194 ++---------------------------------- drivers/infiniband/hw/hfi1/ruc.c | 43 -------- drivers/infiniband/hw/hfi1/verbs.c | 1 drivers/infiniband/hw/hfi1/verbs.h | 9 -- 5 files changed, 16 insertions(+), 235 deletions(-)
drivers/infiniband/hw/hfi1/qp.c +0 -4
@@ -671,8 +671,6 @@ void qp_iter_print(struct seq_file *s, struct qp_iter *iter) iowait_sleep, iowait_wakeup, iowait_sdma_drained); -setup_timer (&priv->s_rnr_timer, hfi1_rc_rnr_retry, (unsigned long )qp); - qp->s_timer.function = hfi1_rc_timeout;return priv; }
@@ -713,7 +711,6 @@ void flush_qp_waiters(struct rvt_qp *qp) {lockdep_assert_held (&qp->s_lock);flush_iowait (qp); -hfi1_stop_rc_timers (qp); }void stop_send_queue (struct rvt_qp *qp)
@@ -721,7 +718,6 @@ void stop_send_queue(struct rvt_qp *qp)struct hfi1_qp_priv *priv = qp->priv;cancel_work_sync (&priv->s_iowait.iowork); -hfi1_del_timers_sync (qp); }void quiesce_qp (struct rvt_qp *qp)
drivers/infiniband/hw/hfi1/rc.c +12 -182
278 lines not on this page · raw message
drivers/infiniband/hw/hfi1/ruc.c +2 -41
61 lines not on this page · raw message
drivers/infiniband/hw/hfi1/verbs.c +1 -0
@@ -1845,6 +1845,7 @@ int hfi1_register_ib_device(struct hfi1_devdata *dd) dd->verbs_dev.rdi.driver_f.mtu_to_path_mtu = mtu_to_path_mtu; dd->verbs_dev.rdi.driver_f.check_modify_qp = hfi1_check_modify_qp; dd->verbs_dev.rdi.driver_f.modify_qp = hfi1_modify_qp; + dd->verbs_dev.rdi.driver_f.notify_restart_rc = hfi1_restart_rc; dd->verbs_dev.rdi.driver_f.check_send_wqe = hfi1_check_send_wqe;/* completeion queue */
drivers/infiniband/hw/hfi1/verbs.h +1 -8
@@ -127,7 +127,6 @@ struct hfi1_qp_priv {u8 s_sc;/* SC[0..4] for next packet */ u8 r_adefered;/* number of acks defered */ struct iowait s_iowait; -struct timer_list s_rnr_timer;struct rvt_qp *owner; };
@@ -310,12 +309,6 @@ void hfi1_rc_hdrerr(struct ib_ah *hfi1_create_qp0_ah (struct hfi1_ibport *ibp,u16 dlid); -void hfi1_rc_rnr_retry (unsigned long arg); -void hfi1_add_rnr_timer (struct rvt_qp *qp,u32 to); -void hfi1_rc_timeout (unsigned long arg); -void hfi1_del_timers_sync (struct rvt_qp *qp); -void hfi1_stop_rc_timers (struct rvt_qp *qp); -void hfi1_rc_send_complete (struct rvt_qp *qp,struct ib_header *hdr);void hfi1_ud_rcv (struct hfi1_packet *packet);
@@ -331,7 +324,7 @@ int hfi1_check_modify_qp(struct rvt_qp *qp, struct ib_qp_attr *attr,void hfi1_modify_qp (struct rvt_qp *qp,struct ib_qp_attr *attr,int attr_mask,struct ib_udata *udata); - +void hfi1_restart_rc (struct rvt_qp *qp,u32 psn,int wait);int hfi1_check_send_wqe (struct rvt_qp *qp,struct rvt_swqe *wqe);extern const u32 rc_only_opcode;
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
PATCH 16/27 IB/qib: Use new rdmavt timers
In reply to a message not in the archiveFrom: Venkata Sandeep Dhanalakota <[email protected]>
Reduce qib code footprint by using the rdmavt timers.
--- drivers/infiniband/hw/qib/qib_qp.c | 1 drivers/infiniband/hw/qib/qib_rc.c | 94 ++++++--------------------------- drivers/infiniband/hw/qib/qib_ruc.c | 45 +--------------- drivers/infiniband/hw/qib/qib_verbs.c | 1 drivers/infiniband/hw/qib/qib_verbs.h | 2 - 5 files changed, 21 insertions(+), 122 deletions(-)
drivers/infiniband/hw/qib/qib_qp.c +0 -1
@@ -351,7 +351,6 @@ void qib_stop_send_queue(struct rvt_qp *qp)struct qib_qp_priv *priv = qp->priv;cancel_work_sync (&priv->s_work); -del_timer_sync (&qp->s_timer); }void qib_quiesce_qp (struct rvt_qp *qp)
drivers/infiniband/hw/qib/qib_rc.c +17 -77
172 lines not on this page · raw message
drivers/infiniband/hw/qib/qib_ruc.c +2 -43
57 lines not on this page · raw message
drivers/infiniband/hw/qib/qib_verbs.c +1 -0
@@ -1659,6 +1659,7 @@ int qib_register_ib_device(struct qib_devdata *dd) dd->verbs_dev.rdi.driver_f.stop_send_queue = qib_stop_send_queue; dd->verbs_dev.rdi.driver_f.flush_qp_waiters = qib_flush_qp_waiters; dd->verbs_dev.rdi.driver_f.notify_error_qp = qib_notify_error_qp; + dd->verbs_dev.rdi.driver_f.notify_restart_rc = qib_restart_rc; dd->verbs_dev.rdi.driver_f.mtu_to_path_mtu = qib_mtu_to_path_mtu; dd->verbs_dev.rdi.driver_f.mtu_from_qp = qib_mtu_from_qp; dd->verbs_dev.rdi.driver_f.get_pmtu_from_attr = qib_get_pmtu_from_attr;
drivers/infiniband/hw/qib/qib_verbs.h +1 -1
@@ -279,7 +279,7 @@ int qib_get_counters(struct qib_pportdata *ppd,void qib_notify_qp_reset (struct rvt_qp *qp);int qib_alloc_qpn (struct rvt_dev_info *rdi,struct rvt_qpn_table *qpt,enum ib_qp_type type,u8 port,gfp_t gfp); - +void qib_restart_rc (struct rvt_qp *qp,u32 psn,int wait);#ifdef CONFIG_DEBUG_FS struct qib_qp_iter ;
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
PATCH 17/27 IB/hfi1, rdmavt: Update copy_sge to use boolean arguments
In reply to a message not in the archiveFrom: Brian Welty <[email protected]>
Convert copy_sge and related SGE state functions to use boolean. For determining if QP is in user mode, add helper function in rdmavt_qp.h. This is used to determine if QP needs the last byte ordering. While here, change rvt_pd.user to a boolean.
--- drivers/infiniband/hw/hfi1/rc.c | 16 ++++++++-------- drivers/infiniband/hw/hfi1/ruc.c | 10 +++++----- drivers/infiniband/hw/hfi1/uc.c | 10 +++++----- drivers/infiniband/hw/hfi1/ud.c | 12 ++++++------ drivers/infiniband/hw/hfi1/verbs.c | 21 +++++++++++---------- drivers/infiniband/hw/hfi1/verbs.h | 4 ++-- drivers/infiniband/sw/rdmavt/pd.c | 2 +- include/rdma/rdma_vt.h | 2 +- include/rdma/rdmavt_qp.h | 9 +++++++++ 9 files changed, 48 insertions(+), 38 deletions(-)
drivers/infiniband/hw/hfi1/rc.c +8 -8
@@ -67,7 +67,7 @@ static u32 restart_sge(struct rvt_sge_state *ss, struct rvt_swqe *wqe, ss->sg_list = wqe->sg_list +1 ; ss->num_sge = wqe->wr.num_sge; ss->total_len = wqe->length; -hfi1_skip_sge (ss, len,0 ); +hfi1_skip_sge (ss, len, false);return wqe->length - len; }
@@ -1508,7 +1508,7 @@ static void rc_rcv_resp(struct hfi1_ibport *ibp, qp->s_rdma_read_len -= pmtu;update_last_psn (qp, psn);spin_unlock_irqrestore (&qp->s_lock, flags); -hfi1_copy_sge (&qp->s_rdma_read_sge, data, pmtu,0 ,0 ); +hfi1_copy_sge (&qp->s_rdma_read_sge, data, pmtu, false, false); goto bail;case OP (RDMA_READ_RESPONSE_ONLY ):
@@ -1552,7 +1552,7 @@ static void rc_rcv_resp(struct hfi1_ibport *ibp,if (unlikely (tlen != qp->s_rdma_read_len)) goto ack_len_err; aeth =be32_to_cpu (ohdr->u.aeth); -hfi1_copy_sge (&qp->s_rdma_read_sge, data, tlen,0 ,0 ); +hfi1_copy_sge (&qp->s_rdma_read_sge, data, tlen, false, false);WARN_ON (qp->s_rdma_read_sge.num_sge); (void )do_rc_ack (qp, aeth, psn,OP (RDMA_READ_RESPONSE_LAST ),0 , rcd);
@@ -1923,7 +1923,7 @@ void hfi1_rc_rcv(struct hfi1_packet *packet)struct ib_reth *reth;unsigned long flags;int ret, is_fecn =0 ; -int copy_last =0 ; +bool copy_last = false;u32 rkey;lockdep_assert_held (&qp->r_lock);
@@ -2017,7 +2017,7 @@ void hfi1_rc_rcv(struct hfi1_packet *packet) qp->r_rcv_len += pmtu;if (unlikely (qp->r_rcv_len > qp->r_len)) goto nack_inv; -hfi1_copy_sge (&qp->r_sge, data, pmtu,1 ,0 ); +hfi1_copy_sge (&qp->r_sge, data, pmtu, true, false);break ;case OP (RDMA_WRITE_LAST_WITH_IMMEDIATE ):
@@ -2057,7 +2057,7 @@ void hfi1_rc_rcv(struct hfi1_packet *packet) wc.wc_flags =IB_WC_WITH_INVALIDATE ; goto send_last;case OP (RDMA_WRITE_LAST ): - copy_last =ibpd_to_rvtpd (qp->ibqp.pd)->user; + copy_last =rvt_is_user_qp (qp);/* fall through */ case OP (SEND_LAST ): no_immediate_data:
@@ -2075,7 +2075,7 @@ void hfi1_rc_rcv(struct hfi1_packet *packet) wc.byte_len = tlen + qp->r_rcv_len;if (unlikely (wc.byte_len > qp->r_len)) goto nack_inv; -hfi1_copy_sge (&qp->r_sge, data, tlen,1 , copy_last); +hfi1_copy_sge (&qp->r_sge, data, tlen, true, copy_last);rvt_put_ss (&qp->r_sge); qp->r_msn++;if (!__test_and_clear_bit (RVT_R_WRID_VALID , &qp->r_aflags))
@@ -2113,7 +2113,7 @@ void hfi1_rc_rcv(struct hfi1_packet *packet)break ;case OP (RDMA_WRITE_ONLY ): - copy_last =1 ; + copy_last =rvt_is_user_qp (qp);/* fall through */ case OP (RDMA_WRITE_FIRST ):case OP (RDMA_WRITE_ONLY_WITH_IMMEDIATE ):
drivers/infiniband/hw/hfi1/ruc.c +5 -5
@@ -320,9 +320,9 @@ static void ruc_loopback(struct rvt_qp *sqp)u64 sdata;atomic64_t *maddr;enum ib_wc_status send_status; -int release; +bool release;int ret; -int copy_last =0 ; +bool copy_last = false;int local_ops =0 ;rcu_read_lock ();
@@ -386,7 +386,7 @@ static void ruc_loopback(struct rvt_qp *sqp)memset (&wc,0 ,sizeof (wc)); send_status =IB_WC_SUCCESS ; - release =1 ; + release = true; sqp->s_sge.sge = wqe->sg_list[0 ]; sqp->s_sge.sg_list = wqe->sg_list +1 ; sqp->s_sge.num_sge = wqe->wr.num_sge;
@@ -437,7 +437,7 @@ static void ruc_loopback(struct rvt_qp *sqp)/* skip copy_last set and qp_access_flags recheck */ goto do_write;case IB_WR_RDMA_WRITE : - copy_last =ibpd_to_rvtpd (qp->ibqp.pd)->user; + copy_last =rvt_is_user_qp (qp);if (unlikely (!(qp->qp_access_flags &IB_ACCESS_REMOTE_WRITE ))) goto inv_err; do_write:
@@ -461,7 +461,7 @@ static void ruc_loopback(struct rvt_qp *sqp) wqe->rdma_wr.rkey, IB_ACCESS_REMOTE_READ))) goto acc_err; - release =0 ; + release = false; sqp->s_sge.sg_list =NULL ; sqp->s_sge.num_sge =1 ; qp->r_sge.sge = wqe->sg_list[0 ];
drivers/infiniband/hw/hfi1/uc.c +5 -5
@@ -419,7 +419,7 @@ void hfi1_uc_rcv(struct hfi1_packet *packet) qp->r_rcv_len += pmtu;if (unlikely (qp->r_rcv_len > qp->r_len)) goto rewind; -hfi1_copy_sge (&qp->r_sge, data, pmtu,0 ,0 ); +hfi1_copy_sge (&qp->r_sge, data, pmtu, false, false);break ;case OP (SEND_LAST_WITH_IMMEDIATE ):
@@ -444,7 +444,7 @@ void hfi1_uc_rcv(struct hfi1_packet *packet)if (unlikely (wc.byte_len > qp->r_len)) goto rewind; wc.opcode =IB_WC_RECV ; -hfi1_copy_sge (&qp->r_sge, data, tlen,0 ,0 ); +hfi1_copy_sge (&qp->r_sge, data, tlen, false, false);rvt_put_ss (&qp->s_rdma_read_sge); last_imm: wc.wr_id = qp->r_wr_id;
@@ -519,7 +519,7 @@ void hfi1_uc_rcv(struct hfi1_packet *packet) qp->r_rcv_len += pmtu;if (unlikely (qp->r_rcv_len > qp->r_len)) goto drop; -hfi1_copy_sge (&qp->r_sge, data, pmtu,1 ,0 ); +hfi1_copy_sge (&qp->r_sge, data, pmtu, true, false);break ;case OP (RDMA_WRITE_LAST_WITH_IMMEDIATE ):
@@ -548,7 +548,7 @@ void hfi1_uc_rcv(struct hfi1_packet *packet) } wc.byte_len = qp->r_len; wc.opcode =IB_WC_RECV_RDMA_WITH_IMM ; -hfi1_copy_sge (&qp->r_sge, data, tlen,1 ,0 ); +hfi1_copy_sge (&qp->r_sge, data, tlen, true, false);rvt_put_ss (&qp->r_sge); goto last_imm;
@@ -564,7 +564,7 @@ void hfi1_uc_rcv(struct hfi1_packet *packet) tlen -= (hdrsize + pad +4 );if (unlikely (tlen + qp->r_rcv_len != qp->r_len)) goto drop; -hfi1_copy_sge (&qp->r_sge, data, tlen,1 ,0 ); +hfi1_copy_sge (&qp->r_sge, data, tlen, true, false);rvt_put_ss (&qp->r_sge);break ;
drivers/infiniband/hw/hfi1/ud.c +6 -6
@@ -189,10 +189,10 @@ static void ud_loopback(struct rvt_qp *sqp, struct rvt_swqe *swqe)hfi1_make_grh (ibp, &grh, &grd,0 ,0 );hfi1_copy_sge (&qp->r_sge, &grh, -sizeof (grh),1 ,0 ); +sizeof (grh), true, false); wc.wc_flags |=IB_WC_GRH ; }else { -hfi1_skip_sge (&qp->r_sge,sizeof (struct ib_grh ),1 ); +hfi1_skip_sge (&qp->r_sge,sizeof (struct ib_grh ), true); } ssge.sg_list = swqe->sg_list +1 ; ssge.sge = *swqe->sg_list;
@@ -206,7 +206,7 @@ static void ud_loopback(struct rvt_qp *sqp, struct rvt_swqe *swqe)if (len > sge->sge_length) len = sge->sge_length;WARN_ON_ONCE (len ==0 ); -hfi1_copy_sge (&qp->r_sge, sge->vaddr, len,1 ,0 ); +hfi1_copy_sge (&qp->r_sge, sge->vaddr, len, true, false); sge->vaddr += len; sge->length -= len; sge->sge_length -= len;
@@ -812,13 +812,13 @@ void hfi1_ud_rcv(struct hfi1_packet *packet) }if (has_grh) {hfi1_copy_sge (&qp->r_sge, &hdr->u.l.grh, -sizeof (struct ib_grh ),1 ,0 ); +sizeof (struct ib_grh ), true, false); wc.wc_flags |=IB_WC_GRH ; }else { -hfi1_skip_sge (&qp->r_sge,sizeof (struct ib_grh ),1 ); +hfi1_skip_sge (&qp->r_sge,sizeof (struct ib_grh ), true); }hfi1_copy_sge (&qp->r_sge, data, wc.byte_len -sizeof (struct ib_grh ), -1 ,0 ); + true, false);rvt_put_ss (&qp->r_sge);if (!test_and_clear_bit (RVT_R_WRID_VALID , &qp->r_aflags))return ;
drivers/infiniband/hw/hfi1/verbs.c +11 -10
@@ -291,7 +291,7 @@ static void wss_insert(void *address)/* * Is the working set larger than the threshold? */ -static inline int wss_exceeds_threshold (void ) +static inline bool wss_exceeds_threshold (void ) {return atomic_read (&wss.total_count) >= wss.threshold; }
@@ -419,18 +419,19 @@ static inline int wss_exceeds_threshold(void)* @ss: the SGE state * @data: the data to copy * @length: the length of the data +* @release: boolean to release MR * @copy_last: do a separate copy of the last 8 bytes */ void hfi1_copy_sge (struct rvt_sge_state *ss,void *data,u32 length, -int release, -int copy_last) +bool release, +bool copy_last) {struct rvt_sge *sge = &ss->sge; -int in_last =0 ;int i; -int cacheless_copy =0 ; +bool in_last = false; +bool cacheless_copy = false;if (sge_copy_mode ==COPY_CACHELESS ) { cacheless_copy = length >=PAGE_SIZE ;
@@ -454,8 +455,8 @@ void hfi1_copy_sge(if (length >8 ) { length -=8 ; }else { - copy_last =0 ; - in_last =1 ; + copy_last = false; + in_last = true; } }
@@ -501,8 +502,8 @@ void hfi1_copy_sge( }if (copy_last) { - copy_last =0 ; - in_last =1 ; + copy_last = false; + in_last = true; length =8 ; goto again; }
@@ -513,7 +514,7 @@ void hfi1_copy_sge(* @ss: the SGE state * @length: the number of bytes to skip */ -void hfi1_skip_sge (struct rvt_sge_state *ss,u32 length,int release) +void hfi1_skip_sge (struct rvt_sge_state *ss,u32 length,bool release) {struct rvt_sge *sge = &ss->sge;
drivers/infiniband/hw/hfi1/verbs.h +2 -2
@@ -289,9 +289,9 @@ static inline u32 delta_psn(u32 a, u32 b)int hfi1_verbs_send (struct rvt_qp *qp,struct hfi1_pkt_state *ps);void hfi1_copy_sge (struct rvt_sge_state *ss,void *data,u32 length, -int release,int copy_last); +bool release,bool copy_last); -void hfi1_skip_sge (struct rvt_sge_state *ss,u32 length,int release); +void hfi1_skip_sge (struct rvt_sge_state *ss,u32 length,bool release);void hfi1_cnp_rcv (struct hfi1_packet *packet);
drivers/infiniband/sw/rdmavt/pd.c +1 -1
@@ -90,7 +90,7 @@ struct ib_pd *rvt_alloc_pd(struct ib_device *ibdev,spin_unlock (&dev->n_pds_lock);/* ib_alloc_pd() will initialize pd->ibpd. */ - pd->user = udata ?1 :0 ; + pd->user = !!udata; ret = &pd->ibpd;
include/rdma/rdma_vt.h +1 -1
@@ -164,7 +164,7 @@ struct rvt_driver_params {/* Protection domain */ struct rvt_pd {struct ib_pd ibpd; -int user;/* non-zero if created from user space */ +bool user; };/* Address handle */
include/rdma/rdmavt_qp.h +9 -0
@@ -468,6 +468,15 @@ struct rvt_mcast { }/** +* rvt_is_user_qp - return if this is user mode QP +* @qp - the target QP +*/ +static inline bool rvt_is_user_qp (struct rvt_qp *qp) +{ +return !!qp->pid; +} + +/** * rvt_get_qp - get a QP reference * @qp - the QP to hold */
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
PATCH 18/27 IB/hfi1, rdmavt: Move SGE state helper routines into rdmavt
In reply to a message not in the archiveFrom: Brian Welty <[email protected]>
To improve code reuse, add small SGE state helper routines to rdmavt_mr.h. Leverage these in hfi1, including refactoring of hfi1_copy_sge.
--- drivers/infiniband/hw/hfi1/hfi.h | 2 - drivers/infiniband/hw/hfi1/rc.c | 2 - drivers/infiniband/hw/hfi1/ud.c | 4 +- drivers/infiniband/hw/hfi1/verbs.c | 91 ++---------------------------------- drivers/infiniband/hw/hfi1/verbs.h | 2 - include/rdma/rdmavt_mr.h | 50 ++++++++++++++++++++ 6 files changed, 57 insertions(+), 94 deletions(-)
drivers/infiniband/hw/hfi1/hfi.h +0 -2
@@ -1797,8 +1797,6 @@ static inline void flush_wc(void)int kdeth_process_eager (struct hfi1_packet *packet);int process_receive_invalid (struct hfi1_packet *packet); -void update_sge (struct rvt_sge_state *ss,u32 length); -/* global module parameter variables */ extern unsigned int hfi1_max_mtu;extern unsigned int hfi1_cu;
drivers/infiniband/hw/hfi1/rc.c +1 -1
@@ -67,7 +67,7 @@ static u32 restart_sge(struct rvt_sge_state *ss, struct rvt_swqe *wqe, ss->sg_list = wqe->sg_list +1 ; ss->num_sge = wqe->wr.num_sge; ss->total_len = wqe->length; -hfi1_skip_sge (ss, len, false); +rvt_skip_sge (ss, len, false);return wqe->length - len; }
drivers/infiniband/hw/hfi1/ud.c +2 -2
@@ -192,7 +192,7 @@ static void ud_loopback(struct rvt_qp *sqp, struct rvt_swqe *swqe) sizeof(grh), true, false); wc.wc_flags |=IB_WC_GRH ; }else { -hfi1_skip_sge (&qp->r_sge,sizeof (struct ib_grh ), true); +rvt_skip_sge (&qp->r_sge,sizeof (struct ib_grh ), true); } ssge.sg_list = swqe->sg_list +1 ; ssge.sge = *swqe->sg_list;
@@ -815,7 +815,7 @@ void hfi1_ud_rcv(struct hfi1_packet *packet) sizeof(struct ib_grh), true, false); wc.wc_flags |=IB_WC_GRH ; }else { -hfi1_skip_sge (&qp->r_sge,sizeof (struct ib_grh ), true); +rvt_skip_sge (&qp->r_sge,sizeof (struct ib_grh ), true); }hfi1_copy_sge (&qp->r_sge, data, wc.byte_len -sizeof (struct ib_grh ), true, false);
drivers/infiniband/hw/hfi1/verbs.c +4 -87
128 lines not on this page · raw message
drivers/infiniband/hw/hfi1/verbs.h +0 -2
@@ -291,8 +291,6 @@ static inline u32 delta_psn(u32 a, u32 b)void hfi1_copy_sge (struct rvt_sge_state *ss,void *data,u32 length,bool release,bool copy_last); -void hfi1_skip_sge (struct rvt_sge_state *ss,u32 length,bool release); -void hfi1_cnp_rcv (struct hfi1_packet *packet);void hfi1_uc_rcv (struct hfi1_packet *packet);
include/rdma/rdmavt_mr.h +50 -0
54 lines not on this page · raw message
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
PATCH 19/27 IB/qib: Updates to use rdmavt's SGE helper routines
In reply to a message not in the archiveFrom: Brian Welty <[email protected]>
--- drivers/infiniband/hw/qib/qib_rc.c | 2 - drivers/infiniband/hw/qib/qib_ud.c | 4 + drivers/infiniband/hw/qib/qib_verbs.c | 95 ++------------------------------- drivers/infiniband/hw/qib/qib_verbs.h | 2 - 4 files changed, 9 insertions(+), 94 deletions(-)
drivers/infiniband/hw/qib/qib_rc.c +1 -1
@@ -49,7 +49,7 @@ static u32 restart_sge(struct rvt_sge_state *ss, struct rvt_swqe *wqe, ss->sg_list = wqe->sg_list +1 ; ss->num_sge = wqe->wr.num_sge; ss->total_len = wqe->length; -qib_skip_sge (ss, len,0 ); +rvt_skip_sge (ss, len, false);return wqe->length - len; }
drivers/infiniband/hw/qib/qib_ud.c +2 -2
@@ -177,7 +177,7 @@ static void qib_ud_loopback(struct rvt_qp *sqp, struct rvt_swqe *swqe) sizeof(grh), 1); wc.wc_flags |=IB_WC_GRH ; }else -qib_skip_sge (&qp->r_sge,sizeof (struct ib_grh ),1 ); +rvt_skip_sge (&qp->r_sge,sizeof (struct ib_grh ), true); ssge.sg_list = swqe->sg_list +1 ; ssge.sge = *swqe->sg_list; ssge.num_sge = swqe->wr.num_sge;
@@ -567,7 +567,7 @@ void qib_ud_rcv(struct qib_ibport *ibp, struct ib_header *hdr, sizeof(struct ib_grh), 1); wc.wc_flags |=IB_WC_GRH ; }else -qib_skip_sge (&qp->r_sge,sizeof (struct ib_grh ),1 ); +rvt_skip_sge (&qp->r_sge,sizeof (struct ib_grh ), true);qib_copy_sge (&qp->r_sge, data, wc.byte_len -sizeof (struct ib_grh ),1 );rvt_put_ss (&qp->r_sge);if (!test_and_clear_bit (RVT_R_WRID_VALID , &qp->r_aflags))
drivers/infiniband/hw/qib/qib_verbs.c +6 -89
129 lines not on this page · raw message
drivers/infiniband/hw/qib/qib_verbs.h +0 -2
@@ -304,8 +304,6 @@ int qib_verbs_send(struct rvt_qp *qp, struct ib_header *hdr,void qib_copy_sge (struct rvt_sge_state *ss,void *data,u32 length,int release); -void qib_skip_sge (struct rvt_sge_state *ss,u32 length,int release); -void qib_uc_rcv (struct qib_ibport *ibp,struct ib_header *hdr,int has_grh,void *data,u32 tlen,struct rvt_qp *qp);
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
PATCH 20/27 IB/rdmavt, IB/hfi1, IB/qib: Correct ack count for passive (RTR) QPs
In reply to a message not in the archiveFrom: Mike Marciniszyn <[email protected]>
The send complete for RC QPs mismanages the ack count when the responder side is only in RTR.
A QP in that state cannot send requests, but it can be the target for operations that elicit responses.
Adjust the RC completion logic to correct the count maintenance by reflecting RECV_OK in a new state test.
--- drivers/infiniband/hw/hfi1/rc.c | 2 +- drivers/infiniband/hw/qib/qib_rc.c | 2 +- include/rdma/rdmavt_qp.h | 2 ++ 3 files changed, 4 insertions(+), 2 deletions(-)
drivers/infiniband/hw/hfi1/rc.c +1 -1
@@ -990,7 +990,7 @@ void hfi1_rc_send_complete(struct rvt_qp *qp, struct ib_header *hdr)u32 psn;lockdep_assert_held (&qp->s_lock); -if (!(ib_rvt_state_ops[qp->state] &RVT_PROCESS_OR_FLUSH_SEND )) +if (!(ib_rvt_state_ops[qp->state] &RVT_SEND_OR_FLUSH_OR_RECV_OK ))return ;/* Find out where the BTH is */
drivers/infiniband/hw/qib/qib_rc.c +1 -1
@@ -895,7 +895,7 @@ void qib_rc_send_complete(struct rvt_qp *qp, struct ib_header *hdr)u32 opcode;u32 psn; -if (!(ib_rvt_state_ops[qp->state] &RVT_PROCESS_OR_FLUSH_SEND )) +if (!(ib_rvt_state_ops[qp->state] &RVT_SEND_OR_FLUSH_OR_RECV_OK ))return ;/* Find out where the BTH is */
include/rdma/rdmavt_qp.h +2 -0
@@ -144,6 +144,8 @@#define RVT_FLUSH_RECV 0x40#define RVT_PROCESS_OR_FLUSH_SEND \ (RVT_PROCESS_SEND_OK | RVT_FLUSH_SEND) +#define RVT_SEND_OR_FLUSH_OR_RECV_OK \ + (RVT_PROCESS_SEND_OK | RVT_FLUSH_SEND | RVT_PROCESS_RECV_OK)/* * Internal send flags
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
PATCH 21/27 IB/hfi1: Modify logging frequency of DCC errors
In reply to a message not in the archiveFrom: Jakub Byczkowski <[email protected]>
Use rate-limit state to limit number of messages logged to kernel message buffer for DCC errors. Add new macro dd_dev_info_ratelimited for that propose. Replace all dd_dev_info calls in handle_dcc_err function with rate-limited version.
--- drivers/infiniband/hw/hfi1/chip.c | 20 +++++++++++--------- drivers/infiniband/hw/hfi1/hfi.h | 4 ++++ 2 files changed, 15 insertions(+), 9 deletions(-)
drivers/infiniband/hw/hfi1/chip.c +11 -9
@@ -7827,7 +7827,8 @@ static void handle_dcc_err(struct hfi1_devdata *dd, u32 unused, u64 reg) }/* just report this */ -dd_dev_info (dd,"DCC Error: fmconfig error: %s\n" , extra); +dd_dev_info_ratelimited (dd,"DCC Error: fmconfig error: %s\n" , + extra); reg &= ~DCC_ERR_FLG_FMCONFIG_ERR_SMASK ; }
@@ -7878,34 +7879,35 @@ static void handle_dcc_err(struct hfi1_devdata *dd, u32 unused, u64 reg) }/* just report this */ -dd_dev_info (dd,"DCC Error: PortRcv error: %s\n" , extra); -dd_dev_info (dd," hdr0 0x%llx, hdr1 0x%llx\n" , - hdr0, hdr1); +dd_dev_info_ratelimited (dd,"DCC Error: PortRcv error: %s\n" +" hdr0 0x%llx, hdr1 0x%llx\n" , + extra, hdr0, hdr1); reg &= ~DCC_ERR_FLG_RCVPORT_ERR_SMASK ; }if (reg &DCC_ERR_FLG_EN_CSR_ACCESS_BLOCKED_UC_SMASK ) {/* informative only */ -dd_dev_info (dd,"8051 access to LCB blocked\n" ); +dd_dev_info_ratelimited (dd,"8051 access to LCB blocked\n" ); reg &= ~DCC_ERR_FLG_EN_CSR_ACCESS_BLOCKED_UC_SMASK ; }if (reg &DCC_ERR_FLG_EN_CSR_ACCESS_BLOCKED_HOST_SMASK ) {/* informative only */ -dd_dev_info (dd,"host access to LCB blocked\n" ); +dd_dev_info_ratelimited (dd,"host access to LCB blocked\n" ); reg &= ~DCC_ERR_FLG_EN_CSR_ACCESS_BLOCKED_HOST_SMASK ; }/* report any remaining errors */ if (reg) -dd_dev_info (dd,"DCC Error: %s\n" , -dcc_err_string (buf,sizeof (buf), reg)); +dd_dev_info_ratelimited (dd,"DCC Error: %s\n" , +dcc_err_string (buf,sizeof (buf), reg));if (lcl_reason ==0 ) lcl_reason =OPA_LINKDOWN_REASON_UNKNOWN ;if (do_bounce) { -dd_dev_info (dd,"%s: PortErrorAction bounce\n" , __func__); +dd_dev_info_ratelimited (dd,"%s: PortErrorAction bounce\n" , + __func__);set_link_down_reason (ppd, lcl_reason,0 , lcl_reason);queue_work (ppd->hfi1_wq, &ppd->link_bounce_work); }
drivers/infiniband/hw/hfi1/hfi.h +4 -0
@@ -1942,6 +1942,10 @@ static inline u64 hfi1_pkt_base_sdma_integrity(struct hfi1_devdata *dd)dev_info (&(dd)->pcidev->dev,"%s: " fmt, \ get_unit_name((dd)->unit), ##__VA_ARGS__) +#define dd_dev_info_ratelimited (dd, fmt, ...) \ + dev_info_ratelimited(&(dd)->pcidev->dev, "%s: " fmt, \ + get_unit_name((dd)->unit), ##__VA_ARGS__) +#define dd_dev_dbg (dd, fmt, ...) \ dev_dbg(&(dd)->pcidev->dev, "%s: " fmt, \ get_unit_name((dd)->unit), ##__VA_ARGS__)
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
PATCH 22/27 IB/hfi1: Add receive fault injection feature
In reply to a message not in the archiveFrom: Mike Marciniszyn <[email protected]>
Add fault injection capability:
- Drop packets unconditionally (fault_by_packet)
- Drop packets based on opcode (fault_by_opcode)
This feature is disabled by default and can be
enabled using the HFI1_FAULT_INJECTION Kconfig.
The faulting traces have been added:
- misc/fault_opcode
- misc/fault_packet
See 'Documentation/fault-injection/fault-injection.txt' for details.
Examples:
- Dropping packets by opcode:
/sys/kernel/debug/hfi1/hfi1_X/fault_opcode
# Enable fault
echo Y > fault_by_opcode
# Setprobability of dropping (0-100%)
# echo 25 > probability
# Set opcode
echo 0x64 > opcode
# Number of times to fault
echo 3 > times
# An optional mask allows you to fault
# a range of opcodes
echo 0xf0 > mask
/sys/kernel/debug/hfi1/hfi1_X/fault_stats
contains a value in parentheses to indicate
number of each opcode dropped.
- Dropping packets unconditionally /sys/kernel/debug/hfi1/hfi1_X/fault_packet
# Enable fault
echo Y > fault_by_packet
/sys/kernel/debug/hfi1/hfi1_X/fault_packet/fault_stats contains the number of packets dropped.
--- drivers/infiniband/hw/hfi1/Kconfig | 6 + drivers/infiniband/hw/hfi1/debugfs.c | 222 +++++++++++++++++++++++++++++++ drivers/infiniband/hw/hfi1/debugfs.h | 35 +++++ drivers/infiniband/hw/hfi1/driver.c | 8 + drivers/infiniband/hw/hfi1/trace_misc.h | 48 +++++++ drivers/infiniband/hw/hfi1/verbs.c | 6 + drivers/infiniband/hw/hfi1/verbs.h | 4 + 7 files changed, 329 insertions(+), 0 deletions(-)
drivers/infiniband/hw/hfi1/Kconfig +6 -0
@@ -27,3 +27,9 @@ config SDMA_VERBOSITY ---help--- This is a configuration flag to enable verbose SDMA debug +config HFI1_FAULT_INJECTION + bool "Fault-injection capability" + depends on FAULT_INJECTION && FAULT_INJECTION_DEBUG_FS + default n + ---help--- + Provide fault-injection capability for HFI1 driver.
drivers/infiniband/hw/hfi1/debugfs.c +222 -0
248 lines not on this page · raw message
drivers/infiniband/hw/hfi1/debugfs.h +35 -0
@@ -53,6 +53,41 @@void hfi1_dbg_ibdev_exit (struct hfi1_ibdev *ibd);void hfi1_dbg_init (void );void hfi1_dbg_exit (void ); + +#ifdef CONFIG_HFI1_FAULT_INJECTION +#include <linux/fault-inject.h> +struct fault_opcode { +struct fault_attr attr; +struct dentry *dir; +bool fault_by_opcode; +u64 n_rxfaults[256 ]; +u64 n_txfaults[256 ]; +u8 opcode; +u8 mask; +}; + +struct fault_packet { +struct fault_attr attr; +struct dentry *dir; +bool fault_by_packet; +u64 n_faults; +}; + +bool hfi1_dbg_fault_opcode (struct rvt_qp *qp,u32 opcode,bool rx); +bool hfi1_dbg_fault_packet (struct hfi1_packet *packet); +#else +static inline bool hfi1_dbg_fault_packet (struct hfi1_packet *packet) +{ +return false; +} + +static inline bool hfi1_dbg_fault_opcode (struct rvt_qp *qp, +u32 opcode,bool rx) +{ +return false; +} +#endif +#else static inline void hfi1_dbg_ibdev_init (struct hfi1_ibdev *ibd) {
drivers/infiniband/hw/hfi1/driver.c +8 -0
@@ -59,6 +59,7 @@#include "trace.h" #include "qp.h" #include "sdma.h" +#include "debugfs.h" #undef pr_fmt#define pr_fmt (fmt) DRIVER_NAME ": " fmt
@@ -1354,6 +1355,9 @@ void handle_eflags(struct hfi1_packet *packet)*/ int process_receive_ib (struct hfi1_packet *packet) { +if (unlikely (hfi1_dbg_fault_packet (packet))) +return RHF_RCV_CONTINUE ; + trace_hfi1_rcvhdr(packet->rcd->ppd->dd, packet->rcd->ctxt, rhf_err_flags(packet->rhf),
@@ -1409,6 +1413,8 @@ int process_receive_error(struct hfi1_packet *packet)int kdeth_process_expected (struct hfi1_packet *packet) { +if (unlikely (hfi1_dbg_fault_packet (packet))) +return RHF_RCV_CONTINUE ;if (unlikely (rhf_err_flags (packet->rhf)))handle_eflags (packet);
@@ -1421,6 +1427,8 @@ int kdeth_process_eager(struct hfi1_packet *packet) {if (unlikely (rhf_err_flags (packet->rhf)))handle_eflags (packet); +if (unlikely (hfi1_dbg_fault_packet (packet))) +return RHF_RCV_CONTINUE ;dd_dev_err (packet->rcd->dd,"Unhandled eager packet received. Dropping.\n" );
drivers/infiniband/hw/hfi1/trace_misc.h +48 -0
54 lines not on this page · raw message
drivers/infiniband/hw/hfi1/verbs.c +6 -0
@@ -60,6 +60,7 @@#include "trace.h" #include "qp.h" #include "verbs_txreq.h" +#include "debugfs.h" static unsigned int hfi1_lkey_table_size =16 ; module_param_named(lkey_table_size, hfi1_lkey_table_size, uint,
@@ -583,6 +584,11 @@ void hfi1_ib_rcv(struct hfi1_packet *packet)rcu_read_unlock (); goto drop; } +if (unlikely (hfi1_dbg_fault_opcode (packet->qp, opcode, + true))) { +rcu_read_unlock (); + goto drop; + }spin_lock_irqsave (&packet->qp->r_lock, flags); packet_handler =qp_ok (opcode, packet);if (likely (packet_handler))
drivers/infiniband/hw/hfi1/verbs.h +4 -0
@@ -195,6 +195,10 @@ struct hfi1_ibdev {struct dentry *hfi1_ibdev_dbg;/* per HFI symlinks to above */ struct dentry *hfi1_ibdev_link; +#ifdef CONFIG_HFI1_FAULT_INJECTION +struct fault_opcode *fault_opcode; +struct fault_packet *fault_packet; +#endif #endif };
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
A message that is not in the archive (sent off-list or not yet received): 20170208132800.16442.94549.stgit-9QXIwq+3FY+1XWohqUldA0EOCMrvLtNR@public.gmane.org
Re: [PATCH 22/27] IB/hfi1: Add receive fault injection feature
Quoted 59 lines · Dennis Dalessandro
From: Mike Marciniszyn <[email protected]>
Add fault injection capability:
- Drop packets unconditionally (fault_by_packet)
- Drop packets based on opcode (fault_by_opcode)This feature is disabled by default and can be
enabled using theHFI1_FAULT_INJECTIONKconfig.The faulting traces have been added:
- misc/fault_opcode
- misc/fault_packetSee '
Documentation/fault-injection/fault-injection.txt' for details.Examples:
- Dropping packets by opcode:
/sys/kernel/debug/hfi1/hfi1_X/fault_opcode
# Enable fault
echo Y >fault_by_opcode
# Setprobability of dropping (0-100%)
# echo 25 > probability
# Set opcode
echo0x64> opcode
# Number of times to fault
echo 3 > times
# An optional mask allows you to fault
# a range of opcodes
echo0xf0> mask
/sys/kernel/debug/hfi1/hfi1_X/fault_stats
contains a value in parentheses to indicate
number of each opcode dropped.- Dropping packets unconditionally
/sys/kernel/debug/hfi1/hfi1_X/fault_packet
# Enable fault
echo Y >fault_by_packet
/sys/kernel/debug/hfi1/hfi1_X/fault_packet/fault_statscontains the number of packets dropped.Signed-off-by Mike Marciniszyn <[email protected]>Signed-off-by Don Hiatt <[email protected]>Signed-off-by Dennis Dalessandro <[email protected]>--- drivers/infiniband/hw/hfi1/Kconfig | 6 + drivers/infiniband/hw/hfi1/debugfs.c | 222 +++++++++++++++++++++++++++++++ drivers/infiniband/hw/hfi1/debugfs.h | 35 +++++ drivers/infiniband/hw/hfi1/driver.c | 8 + drivers/infiniband/hw/hfi1/trace_misc.h | 48 +++++++ drivers/infiniband/hw/hfi1/verbs.c | 6 + drivers/infiniband/hw/hfi1/verbs.h | 4 + 7 files changed, 329 insertions(+), 0 deletions(-)drivers/infiniband/hw/hfi1/Kconfig +0
-0
drivers/infiniband/hw/hfi1/Kconfig config SDMA_VERBOSITY · line 33
Quoted 7 lines
drivers/infiniband/hw/hfi1/Kconfig +4
-0@@ -27,3 +27,9 @@ config SDMA_VERBOSITY ---help--- This is a configuration flag to enable verbose SDMA debug +config HFI1_FAULT_INJECTION + bool "Fault-injection capability" + depends on FAULT_INJECTION && FAULT_INJECTION_DEBUG_FS + default n
First no need to set default to "n", it is already default. Second, we got NAK from Doug while we proposed to add new Kconfig to our driver (mlx5) to improve debug.
Doug,
I would like to know if your statement do not add new Kconfigs to IB is still relevant.
Thanks
A message that is not in the archive (sent off-list or not yet received): 20170212174205.GI14015-U/[email protected]
Re: [PATCH 22/27] IB/hfi1: Add receive fault injection feature
Quoted 76 lines · Leon Romanovsky
Quoted 65 lines · Dennis Dalessandro
From: Mike Marciniszyn <[email protected]>
Add fault injection capability:
- Drop packets unconditionally (fault_by_packet)
- Drop packets based on opcode (fault_by_opcode)This feature is disabled by default and can be
enabled using theHFI1_FAULT_INJECTIONKconfig.The faulting traces have been added:
- misc/fault_opcode
- misc/fault_packetSee '
Documentation/fault-injection/fault-injection.txt' for details.Examples:
- Dropping packets by opcode:
/sys/kernel/debug/hfi1/hfi1_X/fault_opcode
# Enable fault
echo Y >fault_by_opcode
# Setprobability of dropping (0-100%)
# echo 25 > probability
# Set opcode
echo0x64> opcode
# Number of times to fault
echo 3 > times
# An optional mask allows you to fault
# a range of opcodes
echo0xf0> mask
/sys/kernel/debug/hfi1/hfi1_X/fault_stats
contains a value in parentheses to indicate
number of each opcode dropped.- Dropping packets unconditionally
/sys/kernel/debug/hfi1/hfi1_X/fault_packet
# Enable fault
echo Y >fault_by_packet
/sys/kernel/debug/hfi1/hfi1_X/fault_packet/fault_stats contains the number of packets dropped.Signed-off-by Mike Marciniszyn <[email protected]>Signed-off-by Don Hiatt <[email protected]>Signed-off-by Dennis Dalessandro <[email protected]>---
drivers/infiniband/hw/hfi1/Kconfig | 6 + drivers/infiniband/hw/hfi1/debugfs.c | 222
+++++++++++++++++++++++++++++++
drivers/infiniband/hw/hfi1/debugfs.h | 35 +++++ drivers/infiniband/hw/hfi1/driver.c | 8 +drivers/infiniband/hw/hfi1/trace_misc.h| 48 +++++++ drivers/infiniband/hw/hfi1/verbs.c | 6 + drivers/infiniband/hw/hfi1/verbs.h | 4 +7 files changed, 329 insertions(+), 0 deletions(-)drivers/infiniband/hw/hfi1/Kconfig +0
-0@@ -27,3 +27,9 @@ config SDMA_VERBOSITY---help---
This is a configuration flag to enable verbose SDMA debugdrivers/infiniband/hw/hfi1/Kconfig config SDMA_VERBOSITY · line 33
Quoted 4 lines
drivers/infiniband/hw/hfi1/Kconfig +4
-0+config HFI1_FAULT_INJECTION + bool "Fault-injection capability" + depends on FAULT_INJECTION && FAULT_INJECTION_DEBUG_FS + default nFirst no need to set default to "n", it is already default. Second, we got NAK from Doug while we proposed to add new
Kconfigto our
driver (mlx5) to improve debug.
Quoted 4 lines
Doug,
I would like to know if your statement do not add new Kconfigs to IB is
still relevant.
As much as is possible, yes. Kconfig bloat is something I very much want to avoid. I would be more willing to have a single, global CONFIG_INFINIBAND_DEBUG option that turned on debugging stack wide than I would for individual drivers to each have their own debug configs.
Signature · 4 lines
Doug Ledford <[email protected]> GPG KeyID: B826A3330E572FDD Key fingerprint = AE6B 1BDA 122B 23B4 265B 1274 B826 A333 0E57 2FDD
A message that is not in the archive (sent off-list or not yet received): 1487109065.86943.86.camel-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org
Re: [PATCH 22/27] IB/hfi1: Add receive fault injection feature
Quoted 13 lines · Doug Ledford
Quoted 11 lines · Leon Romanovsky
On Wed, Feb 08, 2017 at 05:28:01AM -0800, Dennis Dalessandro wrote:
First no need to set default to "n", it is already default. Second, we got NAK from Doug while we proposed to add new
Kconfigto our
driver (mlx5) to improve debug.Doug,
I would like to know if your statement do not add new Kconfigs to IB is
still relevant.
Quoted 4 lines
As much as is possible, yes.
Kconfigbloat is something I very much want to avoid. I would be more willing to have a single, globalCONFIG_INFINIBAND_DEBUGoption that turned on debugging stack wide than I would for individual drivers to each have their own debug configs.
Been talking among ourselves here and for this patch and the other one we can just rely on the standard CONFIG_FAULT_INJECTION. I will resubmit those with my next set of patches.
In the future though it may be nice rather than having a single debug variable, but a small set of them that are used stack wide. So no mlx or hfi1, etc specific ones, but just for instance a general reference counting flag for mrs and qps. If we come up with some small sane list that we can all agree on would that be acceptable? Yeah we are growing the kconfig but it's at least controlled and thought out.
-Denny
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
PATCH 23/27 IB/hfi1: Add transmit fault injection feature
In reply to a message not in the archiveFrom: Don Hiatt <[email protected]>
Add ability to fault packets on transmit by opcode. Dropping by packet can be achived by setting the mask to 0.
In order to drop non-verbs traffic we set PbcInsertHrc to NONE (0x2). The packet will still be delivered to the receiving node but a KHdrHCRCErr (KDETH packet with a bad HCRC) will be triggered and the packet will not be delivered to the correct context.
In order to drop regular verbs traffic we set the PbcTestEbp flag. The packet will still be delivered to the receiving node but a 'late ebp error' will be triggered and will be dropped.
A global toggle (/sys/kernel/debug/hfi1/hfi1_X/fault_suppress_err) has been added to suppress the error messages on the receive node when a packet was faulted on the sending node.
--- drivers/infiniband/hw/hfi1/chip.c | 4 +++ drivers/infiniband/hw/hfi1/debugfs.c | 8 ++++++ drivers/infiniband/hw/hfi1/debugfs.h | 6 ++++ drivers/infiniband/hw/hfi1/driver.c | 11 ++++++++ drivers/infiniband/hw/hfi1/verbs.c | 49 +++++++++++++++++++++++++++++----- drivers/infiniband/hw/hfi1/verbs.h | 1 + include/rdma/ib_pack.h | 2 + 7 files changed, 74 insertions(+), 7 deletions(-)
drivers/infiniband/hw/hfi1/chip.c +4 -0
@@ -64,6 +64,7 @@#include "platform.h" #include "aspm.h" #include "affinity.h" +#include "debugfs.h" #define NUM_IB_PORTS 1
@@ -7897,6 +7898,9 @@ static void handle_dcc_err(struct hfi1_devdata *dd, u32 unused, u64 reg) reg &= ~DCC_ERR_FLG_EN_CSR_ACCESS_BLOCKED_HOST_SMASK ; } +if (unlikely (hfi1_dbg_fault_suppress_err (&dd->verbs_dev))) + reg &= ~DCC_ERR_FLG_LATE_EBP_ERR_SMASK ; +/* report any remaining errors */ if (reg) dd_dev_info_ratelimited(dd, "DCC Error: %s\n",
drivers/infiniband/hw/hfi1/debugfs.c +8 -0
@@ -1257,6 +1257,11 @@ static int __init fault_init_debugfs(struct hfi1_ibdev *ibd)return ret; } +bool hfi1_dbg_fault_suppress_err (struct hfi1_ibdev *ibd) +{ +return ibd->fault_suppress_err; +} +bool hfi1_dbg_fault_opcode (struct rvt_qp *qp,u32 opcode,bool rx) {bool ret = false;
@@ -1346,6 +1351,9 @@ void hfi1_dbg_ibdev_init(struct hfi1_ibdev *ibd) }#ifdef CONFIG_HFI1_FAULT_INJECTION +debugfs_create_bool ("fault_suppress_err" ,0600 , + ibd->hfi1_ibdev_dbg, + &ibd->fault_suppress_err);fault_init_debugfs (ibd);#endif }
drivers/infiniband/hw/hfi1/debugfs.h +6 -0
@@ -75,6 +75,7 @@ struct fault_packet {bool hfi1_dbg_fault_opcode (struct rvt_qp *qp,u32 opcode,bool rx);bool hfi1_dbg_fault_packet (struct hfi1_packet *packet); +bool hfi1_dbg_fault_suppress_err (struct hfi1_ibdev *ibd);#else static inline bool hfi1_dbg_fault_packet (struct hfi1_packet *packet) {
@@ -86,6 +87,11 @@ static inline bool hfi1_dbg_fault_opcode(struct rvt_qp *qp, {return false; } + +static inline bool hfi1_dbg_fault_suppress_err (struct hfi1_ibdev *ibd) +{ +return false; +}#endif #else
drivers/infiniband/hw/hfi1/driver.c +11 -0
@@ -1367,6 +1367,11 @@ int process_receive_ib(struct hfi1_packet *packet) packet->updegr, rhf_egr_index(packet->rhf)); +if (unlikely ( + (hfi1_dbg_fault_suppress_err (&packet->rcd->dd->verbs_dev) && + (packet->rhf &RHF_DC_ERR )))) +return RHF_RCV_CONTINUE ; +if (unlikely (rhf_err_flags (packet->rhf))) {handle_eflags (packet);return RHF_RCV_CONTINUE ;
@@ -1402,6 +1407,12 @@ int process_receive_bypass(struct hfi1_packet *packet)int process_receive_error (struct hfi1_packet *packet) { +/* KHdrHCRCErr -- KDETH packet with a bad HCRC */ +if (unlikely ( +hfi1_dbg_fault_suppress_err (&packet->rcd->dd->verbs_dev) && +rhf_rcv_type_err (packet->rhf) ==3 )) +return RHF_RCV_CONTINUE ; +handle_eflags (packet);if (unlikely (rhf_err_flags (packet->rhf)))
drivers/infiniband/hw/hfi1/verbs.c +42 -7
90 lines not on this page · raw message
drivers/infiniband/hw/hfi1/verbs.h +1 -0
@@ -198,6 +198,7 @@ struct hfi1_ibdev {#ifdef CONFIG_HFI1_FAULT_INJECTION struct fault_opcode *fault_opcode;struct fault_packet *fault_packet; +bool fault_suppress_err;#endif #endif };
include/rdma/ib_pack.h +2 -0
@@ -80,6 +80,8 @@ enum {IB_OPCODE_UD =0x60 ,/* per IBTA 1.3 vol 1 Table 38, A10.3.2 */ IB_OPCODE_CNP =0x80 , +/* Manufacturer specific */ +IB_OPCODE_MSP =0xe0 ,/* operations -- just used to define real constants */ IB_OPCODE_SEND_FIRST = 0x00,
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
PATCH 24/27 IB/hfi1: Do not set physical link state if DC is in the shutdown state
In reply to a message not in the archiveFrom: Michael J. Ruhl <[email protected]>
If the DC is in shutdown state, the set link state function will return an error. Since this is not a failure in this state, make sure to only call set link state if the DC is on.
--- drivers/infiniband/hw/hfi1/chip.c | 18 ++++++++++-------- 1 files changed, 10 insertions(+), 8 deletions(-)
drivers/infiniband/hw/hfi1/chip.c +10 -8
@@ -10514,16 +10514,18 @@ int set_link_state(struct hfi1_pportdata *ppd, u32 state) ppd->remote_link_down_reason =0 ; } - ret1 =set_physical_link_state (dd,PLS_DISABLED ); -if (ret1 !=HCMD_SUCCESS ) { -dd_dev_err (dd, -"Failed to transition to Disabled link state, return 0x%x\n" , - ret1); - ret = -EINVAL ; -break ; +if (!dd->dc_shutdown) { + ret1 =set_physical_link_state (dd,PLS_DISABLED ); +if (ret1 !=HCMD_SUCCESS ) { +dd_dev_err (dd, +"Failed to transition to Disabled link state, return 0x%x\n" , + ret1); + ret = -EINVAL ; +break ; + } +dc_shutdown (dd); } ppd->host_link_state =HLS_DN_DISABLE ; -dc_shutdown (dd);break ;case HLS_DN_OFFLINE :if (ppd->host_link_state ==HLS_DN_DISABLE )
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
PATCH 25/27 IB/hfi1: Add rvt_rnr_tbl_to_usec function
In reply to a message not in the archiveFrom: Don Hiatt <[email protected]>
Return usec from an index into ib_rvt_rnr_table.
--- drivers/infiniband/sw/rdmavt/qp.c | 11 +++++++++++ include/rdma/rdmavt_qp.h | 1 + 2 files changed, 12 insertions(+), 0 deletions(-)
drivers/infiniband/sw/rdmavt/qp.c +11 -0
@@ -1954,6 +1954,17 @@ void rvt_rc_error(struct rvt_qp *qp, enum ib_wc_status err) }EXPORT_SYMBOL (rvt_rc_error); +/* +* rvt_rnr_tbl_to_usec - return index into ib_rvt_rnr_table +* @index - the index +* return usec from an index into ib_rvt_rnr_table +*/ +unsigned long rvt_rnr_tbl_to_usec (u32 index) +{ +return ib_rvt_rnr_table[(index &RVT_AETH_CREDIT_MASK )]; +} +EXPORT_SYMBOL (rvt_rnr_tbl_to_usec); +static inline unsigned long rvt_aeth_to_usec (u32 aeth) { return ib_rvt_rnr_table[(aeth >> RVT_AETH_CREDIT_SHIFT) &
include/rdma/rdmavt_qp.h +1 -0
@@ -653,6 +653,7 @@ static inline u32 rvt_div_mtu(struct rvt_qp *qp, u32 len)void rvt_comm_est (struct rvt_qp *qp);int rvt_error_qp (struct rvt_qp *qp,enum ib_wc_status err);void rvt_rc_error (struct rvt_qp *qp,enum ib_wc_status err); +unsigned long rvt_rnr_tbl_to_usec (u32 index);enum hrtimer_restart rvt_rc_rnr_retry (struct hrtimer *t);void rvt_add_rnr_timer (struct rvt_qp *qp,u32 aeth);void rvt_del_timers_sync (struct rvt_qp *qp);
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
A message that is not in the archive (sent off-list or not yet received): 20170208132818.16442.38634.stgit-9QXIwq+3FY+1XWohqUldA0EOCMrvLtNR@public.gmane.org
Re: [PATCH 25/27] IB/hfi1: Add rvt_rnr_tbl_to_usec function
Quoted 21 lines · Dennis Dalessandro
From: Don Hiatt <[email protected]>
Return usec from an index into
ib_rvt_rnr_table.Reviewed-by Mike Marciniszyn <[email protected]>Signed-off-by Don Hiatt <[email protected]>Signed-off-by Dennis Dalessandro <[email protected]>--- drivers/infiniband/sw/rdmavt/qp.c | 11 +++++++++++ include/rdma/rdmavt_qp.h | 1 + 2 files changed, 12 insertions(+), 0 deletions(-)drivers/infiniband/sw/rdmavt/qp.c +1
-0@@ -1954,6 +1954,17 @@ void rvt_rc_error(struct rvt_qp *qp, enum ib_wc_status err) }EXPORT_SYMBOL (rvt_rc_error); +/*
drivers/infiniband/sw/rdmavt/qp.c void rvt_rc_error(struct rvt_qp *qp, enum ib_wc_status err) · line 1967
Quoted 10 lines
drivers/infiniband/sw/rdmavt/qp.c +10
-0+* rvt_rnr_tbl_to_usec - return index into ib_rvt_rnr_table +* @index - the index +* return usec from an index into ib_rvt_rnr_table +*/ +unsigned long rvt_rnr_tbl_to_usec (u32 index) +{ +return ib_rvt_rnr_table[(index &RVT_AETH_CREDIT_MASK )]; +} +EXPORT_SYMBOL (rvt_rnr_tbl_to_usec); +
Do we have a usage of this new exported function?
Quoted 20 lines
drivers/infiniband/sw/rdmavt/qp.c +0
-0static inline unsigned long rvt_aeth_to_usec (u32 aeth) { return ib_rvt_rnr_table[(aeth >> RVT_AETH_CREDIT_SHIFT) &include/rdma/rdmavt_qp.h +1
-1@@ -653,6 +653,7 @@ static inline u32 rvt_div_mtu(struct rvt_qp *qp, u32 len)void rvt_comm_est (struct rvt_qp *qp);int rvt_error_qp (struct rvt_qp *qp,enum ib_wc_status err);void rvt_rc_error (struct rvt_qp *qp,enum ib_wc_status err); +unsigned long rvt_rnr_tbl_to_usec (u32 index);enum hrtimer_restart rvt_rc_rnr_retry (struct hrtimer *t);void rvt_add_rnr_timer (struct rvt_qp *qp,u32 aeth);void rvt_del_timers_sync (struct rvt_qp *qp); --To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
PATCH 26/27 IB/hfi1, qib, rdmavt: Move AETH defines to rdma/ib_hdrs.h
In reply to a message not in the archiveFrom: Don Hiatt <[email protected]>
Rename RVT AETH defines and export in rdma/ib_hdrs.h
--- drivers/infiniband/hw/hfi1/rc.c | 18 +++++++++--------- drivers/infiniband/hw/hfi1/ruc.c | 2 +- drivers/infiniband/hw/hfi1/trace.c | 4 ++-- drivers/infiniband/hw/qib/qib_rc.c | 18 +++++++++--------- drivers/infiniband/hw/qib/qib_ruc.c | 2 +- drivers/infiniband/sw/rdmavt/qp.c | 7 ++++--- drivers/infiniband/sw/rdmavt/rc.c | 15 +++++++-------- include/rdma/ib_hdrs.h | 6 ++++++ include/rdma/rdmavt_qp.h | 5 ----- 9 files changed, 39 insertions(+), 38 deletions(-)
drivers/infiniband/hw/hfi1/rc.c +9 -9
@@ -211,9 +211,9 @@ static int make_rc_ack(struct hfi1_ibdev *dev, struct rvt_qp *qp, ps->s_txreq->ss =NULL ;if (qp->s_nak_state) ohdr->u.aeth = -cpu_to_be32 ((qp->r_msn &RVT_MSN_MASK ) | +cpu_to_be32 ((qp->r_msn &IB_MSN_MASK ) | (qp->s_nak_state << -RVT_AETH_CREDIT_SHIFT )); +IB_AETH_CREDIT_SHIFT ));else ohdr->u.aeth =rvt_compute_aeth (qp); hwords++;
@@ -758,9 +758,9 @@ void hfi1_send_rc_ack(struct hfi1_ctxtdata *rcd, struct rvt_qp *qp,if (qp->s_mig_state ==IB_MIG_MIGRATED ) bth0 |=IB_BTH_MIG_REQ ;if (qp->r_nak_state) - ohdr->u.aeth =cpu_to_be32 ((qp->r_msn &RVT_MSN_MASK ) | + ohdr->u.aeth =cpu_to_be32 ((qp->r_msn &IB_MSN_MASK ) | (qp->r_nak_state << -RVT_AETH_CREDIT_SHIFT )); +IB_AETH_CREDIT_SHIFT ));else ohdr->u.aeth =rvt_compute_aeth (qp); sc5 = ibp->sl_to_sc[qp->remote_ah_attr.sl];
@@ -1157,7 +1157,7 @@ static int do_rc_ack(struct rvt_qp *qp, u32 aeth, u32 psn, int opcode,* request but will include an ACK'ed request(s). */ ack_psn = psn; -if (aeth >>RVT_AETH_NAK_SHIFT ) +if (aeth >>IB_AETH_NAK_SHIFT ) ack_psn--; wqe =rvt_get_swqe_ptr (qp, qp->s_acked); ibp =rcd_to_iport (rcd);
@@ -1237,7 +1237,7 @@ static int do_rc_ack(struct rvt_qp *qp, u32 aeth, u32 psn, int opcode,break ; } -switch (aeth >>RVT_AETH_NAK_SHIFT ) { +switch (aeth >>IB_AETH_NAK_SHIFT ) {case 0 :/* ACK */ this_cpu_inc (*ibp->rvp.rc_acks);if (qp->s_acked != qp->s_tail) {
@@ -1300,8 +1300,8 @@ static int do_rc_ack(struct rvt_qp *qp, u32 aeth, u32 psn, int opcode, goto bail_stop;/* The last valid PSN is the previous PSN. */ update_last_psn (qp, psn -1 ); -switch ((aeth >>RVT_AETH_CREDIT_SHIFT ) & -RVT_AETH_CREDIT_MASK ) { +switch ((aeth >>IB_AETH_CREDIT_SHIFT ) & +IB_AETH_CREDIT_MASK ) {case 0 :/* PSN sequence error */ ibp->rvp.n_seq_naks++;/*
@@ -1431,7 +1431,7 @@ static void rc_rcv_resp(struct hfi1_ibport *ibp,/* Update credits for "ghost" ACKs */ if (diff ==0 && opcode ==OP (ACKNOWLEDGE )) { aeth =be32_to_cpu (ohdr->u.aeth); -if ((aeth >>RVT_AETH_NAK_SHIFT ) ==0 ) +if ((aeth >>IB_AETH_NAK_SHIFT ) ==0 )rvt_get_credit (qp, aeth); } goto ack_done;
drivers/infiniband/hw/hfi1/ruc.c +1 -1
@@ -580,7 +580,7 @@ static void ruc_loopback(struct rvt_qp *sqp)if (!(ib_rvt_state_ops[sqp->state] &RVT_PROCESS_RECV_OK )) goto clr_busy;rvt_add_rnr_timer (sqp, qp->r_min_rnr_timer << -RVT_AETH_CREDIT_SHIFT ); +IB_AETH_CREDIT_SHIFT ); goto clr_busy; op_err:
drivers/infiniband/hw/hfi1/trace.c +2 -2
@@ -130,14 +130,14 @@ u8 ibhdr_exhdr_len(struct ib_header *hdr)case OP (RC ,ACKNOWLEDGE ):trace_seq_printf (p,AETH_PRN ,be32_to_cpu (eh->aeth) >>24 ,parse_syndrome (be32_to_cpu (eh->aeth) >>24 ), -be32_to_cpu (eh->aeth) &RVT_MSN_MASK ); +be32_to_cpu (eh->aeth) &IB_MSN_MASK );break ;/* aeth + atomicacketh */ case OP (RC ,ATOMIC_ACKNOWLEDGE ):trace_seq_printf (p,AETH_PRN " " ATOMICACKETH_PRN ,be32_to_cpu (eh->at.aeth) >>24 ,parse_syndrome (be32_to_cpu (eh->at.aeth) >>24 ), -be32_to_cpu (eh->at.aeth) &RVT_MSN_MASK , +be32_to_cpu (eh->at.aeth) &IB_MSN_MASK ,ib_u64_get (&eh->at.atomic_ack_eth));break ;/* atomiceth */
drivers/infiniband/hw/qib/qib_rc.c +9 -9
@@ -187,9 +187,9 @@ static int qib_make_rc_ack(struct qib_ibdev *dev, struct rvt_qp *qp, qp->s_cur_sge =NULL ;if (qp->s_nak_state) ohdr->u.aeth = -cpu_to_be32 ((qp->r_msn &RVT_MSN_MASK ) | +cpu_to_be32 ((qp->r_msn &IB_MSN_MASK ) | (qp->s_nak_state << -RVT_AETH_CREDIT_SHIFT )); +IB_AETH_CREDIT_SHIFT ));else ohdr->u.aeth =rvt_compute_aeth (qp); hwords++;
@@ -648,9 +648,9 @@ void qib_send_rc_ack(struct rvt_qp *qp)if (qp->s_mig_state ==IB_MIG_MIGRATED ) bth0 |=IB_BTH_MIG_REQ ;if (qp->r_nak_state) - ohdr->u.aeth =cpu_to_be32 ((qp->r_msn &RVT_MSN_MASK ) | + ohdr->u.aeth =cpu_to_be32 ((qp->r_msn &IB_MSN_MASK ) | (qp->r_nak_state << -RVT_AETH_CREDIT_SHIFT )); +IB_AETH_CREDIT_SHIFT ));else ohdr->u.aeth =rvt_compute_aeth (qp); lrh0 |= ibp->sl_to_vl[qp->remote_ah_attr.sl] << 12 |
@@ -1042,7 +1042,7 @@ static int do_rc_ack(struct rvt_qp *qp, u32 aeth, u32 psn, int opcode,* request but will include an ACK'ed request(s). */ ack_psn = psn; -if (aeth >>RVT_AETH_NAK_SHIFT ) +if (aeth >>IB_AETH_NAK_SHIFT ) ack_psn--; wqe =rvt_get_swqe_ptr (qp, qp->s_acked); ibp =to_iport (qp->ibqp.device, qp->port_num);
@@ -1122,7 +1122,7 @@ static int do_rc_ack(struct rvt_qp *qp, u32 aeth, u32 psn, int opcode,break ; } -switch (aeth >>RVT_AETH_NAK_SHIFT ) { +switch (aeth >>IB_AETH_NAK_SHIFT ) {case 0 :/* ACK */ this_cpu_inc (*ibp->rvp.rc_acks);if (qp->s_acked != qp->s_tail) {
@@ -1185,8 +1185,8 @@ static int do_rc_ack(struct rvt_qp *qp, u32 aeth, u32 psn, int opcode, goto bail;/* The last valid PSN is the previous PSN. */ update_last_psn (qp, psn -1 ); -switch ((aeth >>RVT_AETH_CREDIT_SHIFT ) & -RVT_AETH_CREDIT_MASK ) { +switch ((aeth >>IB_AETH_CREDIT_SHIFT ) & +IB_AETH_CREDIT_MASK ) {case 0 :/* PSN sequence error */ ibp->rvp.n_seq_naks++;/*
@@ -1341,7 +1341,7 @@ static void qib_rc_rcv_resp(struct qib_ibport *ibp,/* Update credits for "ghost" ACKs */ if (diff ==0 && opcode ==OP (ACKNOWLEDGE )) { aeth =be32_to_cpu (ohdr->u.aeth); -if ((aeth >>RVT_AETH_NAK_SHIFT ) ==0 ) +if ((aeth >>IB_AETH_NAK_SHIFT ) ==0 )rvt_get_credit (qp, aeth); } goto ack_done;
drivers/infiniband/hw/qib/qib_ruc.c +1 -1
@@ -562,7 +562,7 @@ static void qib_ruc_loopback(struct rvt_qp *sqp)if (!(ib_rvt_state_ops[sqp->state] &RVT_PROCESS_RECV_OK )) goto clr_busy;rvt_add_rnr_timer (sqp, qp->r_min_rnr_timer << -RVT_AETH_CREDIT_SHIFT ); +IB_AETH_CREDIT_SHIFT ); goto clr_busy; op_err:
drivers/infiniband/sw/rdmavt/qp.c +4 -3
@@ -51,6 +51,7 @@#include <linux/vmalloc.h> #include <linux/slab.h> #include <rdma/ib_verbs.h> +#include <rdma/ib_hdrs.h> #include "qp.h" #include "vt.h" #include "trace.h"
@@ -1961,14 +1962,14 @@ void rvt_rc_error(struct rvt_qp *qp, enum ib_wc_status err)*/ unsigned long rvt_rnr_tbl_to_usec (u32 index) { -return ib_rvt_rnr_table[(index &RVT_AETH_CREDIT_MASK )]; +return ib_rvt_rnr_table[(index &IB_AETH_CREDIT_MASK )]; }EXPORT_SYMBOL (rvt_rnr_tbl_to_usec);static inline unsigned long rvt_aeth_to_usec (u32 aeth) { -return ib_rvt_rnr_table[(aeth >>RVT_AETH_CREDIT_SHIFT ) & -RVT_AETH_CREDIT_MASK ]; +return ib_rvt_rnr_table[(aeth >>IB_AETH_CREDIT_SHIFT ) & +IB_AETH_CREDIT_MASK ]; }/*
drivers/infiniband/sw/rdmavt/rc.c +7 -8
@@ -46,8 +46,7 @@*/ #include <rdma/rdma_vt.h> - -#define RVT_AETH_CREDIT_INVAL RVT_AETH_CREDIT_MASK +#include <rdma/ib_hdrs.h> /* * Convert the AETH credit code into the number of credits.
@@ -94,14 +93,14 @@*/ __be32 rvt_compute_aeth (struct rvt_qp *qp) { -u32 aeth = qp->r_msn &RVT_MSN_MASK ; +u32 aeth = qp->r_msn &IB_MSN_MASK ;if (qp->ibqp.srq) {/* * Shared receive queues don't generate credits. * Set the credit field to the invalid value. */ - aeth |=RVT_AETH_CREDIT_INVAL <<RVT_AETH_CREDIT_SHIFT ; + aeth |=IB_AETH_CREDIT_INVAL <<IB_AETH_CREDIT_SHIFT ; }else {u32 min, max, x;u32 credits;
@@ -143,7 +142,7 @@ __be32 rvt_compute_aeth(struct rvt_qp *qp) min = x; } } - aeth |= x <<RVT_AETH_CREDIT_SHIFT ; + aeth |= x <<IB_AETH_CREDIT_SHIFT ; }return cpu_to_be32 (aeth); }
@@ -159,7 +158,7 @@ __be32 rvt_compute_aeth(struct rvt_qp *qp)void rvt_get_credit (struct rvt_qp *qp,u32 aeth) {struct rvt_dev_info *rdi =ib_to_rvt (qp->ibqp.device); -u32 credit = (aeth >>RVT_AETH_CREDIT_SHIFT ) &RVT_AETH_CREDIT_MASK ; +u32 credit = (aeth >>IB_AETH_CREDIT_SHIFT ) &IB_AETH_CREDIT_MASK ;lockdep_assert_held (&qp->s_lock);/*
@@ -167,7 +166,7 @@ void rvt_get_credit(struct rvt_qp *qp, u32 aeth)* as many packets as we like. Otherwise, we have to * honor the credit field. */ -if (credit ==RVT_AETH_CREDIT_INVAL ) { +if (credit ==IB_AETH_CREDIT_INVAL ) {if (!(qp->s_flags &RVT_S_UNLIMITED_CREDIT )) { qp->s_flags |=RVT_S_UNLIMITED_CREDIT ;if (qp->s_flags &RVT_S_WAIT_SSN_CREDIT ) {
@@ -177,7 +176,7 @@ void rvt_get_credit(struct rvt_qp *qp, u32 aeth) } }else if (!(qp->s_flags &RVT_S_UNLIMITED_CREDIT )) {/* Compute new LSN (i.e., MSN + credit) */ - credit = (aeth + credit_table[credit]) &RVT_MSN_MASK ; + credit = (aeth + credit_table[credit]) &IB_MSN_MASK ;if (rvt_cmp_msn (credit, qp->s_lsn) >0 ) { qp->s_lsn = credit;if (qp->s_flags &RVT_S_WAIT_SSN_CREDIT ) {
include/rdma/ib_hdrs.h +6 -0
@@ -75,6 +75,12 @@#define IB_GRH_FLOW_SHIFT 0#define IB_GRH_NEXT_HDR 0x1B +#define IB_AETH_CREDIT_SHIFT 24 +#define IB_AETH_CREDIT_MASK 0x1F +#define IB_AETH_CREDIT_INVAL 0x1F +#define IB_AETH_NAK_SHIFT 29 +#define IB_MSN_MASK 0xFFFFFF +struct ib_reth {__be64 vaddr;/* potentially unaligned */ __be32 rkey;
include/rdma/rdmavt_qp.h +0 -5
@@ -594,11 +594,6 @@ static inline void rvt_qp_swqe_complete( } } -#define RVT_AETH_CREDIT_SHIFT 24 -#define RVT_AETH_CREDIT_MASK 0x1F -#define RVT_AETH_NAK_SHIFT 29 -#define RVT_MSN_MASK 0xFFFFFF -/* * Compare the lower 24 bits of the msn values. * Returns an integer <, ==, or > than zero.
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
A message that is not in the archive (sent off-list or not yet received): 20170208132824.16442.61753.stgit-9QXIwq+3FY+1XWohqUldA0EOCMrvLtNR@public.gmane.org
Re: [PATCH 26/27] IB/hfi1, qib, rdmavt: Move AETH defines to rdma/ib_hdrs.h
Quoted 3 lines · Dennis Dalessandro
From: Don Hiatt <[email protected]>
Rename RVT AETH defines and export in
rdma/ib_hdrs.h
Sorry, for my silly question.
Why do we need this patch?
Thanks
PATCH 27/27 IB/hfi1: Code reuse with memdup_copy
In reply to a message not in the archiveFrom: Michael J. Ruhl <[email protected]>
Update several usages of kmalloc/user_copy to memdup_copy and memdup_copy_nul.
--- drivers/infiniband/hw/hfi1/debugfs.c | 39 ++++++++--------------------- drivers/infiniband/hw/hfi1/user_exp_rcv.c | 17 ++++--------- drivers/infiniband/hw/hfi1/user_sdma.c | 17 ++++++------- 3 files changed, 25 insertions(+), 48 deletions(-)
drivers/infiniband/hw/hfi1/debugfs.c +11 -28
@@ -53,6 +53,7 @@#include <linux/types.h> #include <linux/ratelimit.h> #include <linux/fault-inject.h> +#include <linux/string.h> #include "hfi.h" #include "trace.h"
@@ -507,18 +508,11 @@ static ssize_t asic_flags_write(struct file *file, const char __user *buf, ppd =private2ppd (file); dd = ppd->dd; - buff =kmalloc (count +1 ,GFP_KERNEL ); -if (!buff) -return -ENOMEM ; - - ret =copy_from_user (buff, buf, count); -if (ret >0 ) { - ret = -EFAULT ; - goto do_free; - } -/* zero terminate and read the expected integer */ - buff[count] =0 ; + buff =memdup_user_nul (buf, count); +if (IS_ERR (buff)) +return PTR_ERR (buff); + ret =kstrtoull (buff,0 , &value);if (ret) goto do_free;
@@ -696,15 +690,9 @@ static ssize_t __i2c_debugfs_write(struct file *file, const char __user *buf,if (i2c_addr ==0 )return -EINVAL ; - buff =kmalloc (count,GFP_KERNEL ); -if (!buff) -return -ENOMEM ; - - ret =copy_from_user (buff, buf, count); -if (ret >0 ) { - ret = -EFAULT ; - goto _free; - } + buff =memdup_user (buf, count); +if (IS_ERR (buff)) +return PTR_ERR (buff); total_written =i2c_write (ppd, target, i2c_addr, offset, buff, count);if (total_written <0 ) {
@@ -809,15 +797,10 @@ static ssize_t __qsfp_debugfs_write(struct file *file, const char __user *buf, ppd =private2ppd (file); - buff =kmalloc (count,GFP_KERNEL ); -if (!buff) -return -ENOMEM ; + buff =memdup_user (buf, count); +if (IS_ERR (buff)) +return PTR_ERR (buff); - ret =copy_from_user (buff, buf, count); -if (ret >0 ) { - ret = -EFAULT ; - goto _free; - } total_written =qsfp_write (ppd, target, *ppos, buff, count);if (total_written <0 ) { ret = total_written;
drivers/infiniband/hw/hfi1/user_exp_rcv.c +6 -11
@@ -45,6 +45,7 @@* */ #include <asm/page.h> +#include <linux/string.h> #include "user_exp_rcv.h" #include "trace.h"
@@ -577,16 +578,10 @@ int hfi1_user_exp_rcv_clear(struct file *fp, struct hfi1_tid_info *tinfo)u32 *tidinfo;unsigned tididx; - tidinfo =kcalloc (tinfo->tidcnt,sizeof (*tidinfo),GFP_KERNEL ); -if (!tidinfo) -return -ENOMEM ; - -if (copy_from_user (tidinfo, (void __user *)(unsigned long ) - tinfo->tidlist,sizeof (tidinfo[0 ]) * - tinfo->tidcnt)) { - ret = -EFAULT ; - goto done; - } + tidinfo =memdup_user ((void __user *)(unsigned long )tinfo->tidlist, +sizeof (tidinfo[0 ]) * tinfo->tidcnt); +if (IS_ERR (tidinfo)) +return PTR_ERR (tidinfo);mutex_lock (&uctxt->exp_lock);for (tididx =0 ; tididx < tinfo->tidcnt; tididx++) {
@@ -602,7 +597,7 @@ int hfi1_user_exp_rcv_clear(struct file *fp, struct hfi1_tid_info *tinfo)spin_unlock (&fd->tid_lock); tinfo->tidcnt = tididx;mutex_unlock (&uctxt->exp_lock); -done: +kfree (tidinfo);return ret; }
drivers/infiniband/hw/hfi1/user_sdma.c +8 -9
@@ -60,6 +60,7 @@#include <linux/mmu_context.h> #include <linux/module.h> #include <linux/vmalloc.h> +#include <linux/string.h> #include "hfi.h" #include "sdma.h"
@@ -725,30 +726,28 @@ int hfi1_user_sdma_process_request(struct file *fp, struct iovec *iovec,*/ if (req_opcode (req->info.ctrl) ==EXPECTED ) {u16 ntids = iovec[idx].iov_len /sizeof (*req->tids); +u32 *tmp;if (!ntids || ntids >MAX_TID_PAIR_ENTRIES ) { ret = -EINVAL ; goto free_req; } - req->tids =kcalloc (ntids,sizeof (*req->tids),GFP_KERNEL ); -if (!req->tids) { - ret = -ENOMEM ; - goto free_req; - } +/* * We have to copy all of the tids because they may vary * in size and, therefore, the TID count might not be * equal to the pkt count. However, there is no way to * tell at this point. */ - ret =copy_from_user (req->tids, iovec[idx].iov_base, - ntids *sizeof (*req->tids)); -if (ret) { + tmp =memdup_user (iovec[idx].iov_base, + ntids *sizeof (*req->tids)); +if (IS_ERR (tmp)) { + ret =PTR_ERR (tmp);SDMA_DBG (req,"Failed to copy %d TIDs (%d)" , ntids, ret); - ret = -EFAULT ; goto free_req; } + req->tids = tmp; req->n_tids = ntids; idx++; }
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
Quoted 11 lines · Dennis Dalessandro
Doug,
Here is a set of patches for our drivers for 4.11. This is a collection of bug fixes, perf improvements, and clean ups.
This does not include the port reordering patch series or vnic as those are still undergoing discussion on the list.
Patches apply to your for-4.11 branch and can also be found in my GitHub repo at:
https://github.com/ddalessa/kernel/tree/for-4.11
Hi Denny,
I took everything except patches 22 and 23 because they add a Kconfig and I'm trying not to do that. I'd be more than happy if you and Leon worked out a joint Kconfig for debugging that didn't have all the compile time options and instead you have one compile time option, then the remainder are runtime options, and make that a global RDMA stack debugging option. You both have debug features you want to enable, but I want the Kconfig to remain simple and sane.
Signature · 4 lines
Doug Ledford <[email protected]> GPG KeyID: B826A3330E572FDD Key fingerprint = AE6B 1BDA 122B 23B4 265B 1274 B826 A333 0E57 2FDD