IB/rdmavt/qib/hfi1/core: First round for 4.9
Subsystems (3)
cover IB/rdmavt/qib/hfi1/core: First round for 4.9
Doug,
Here are a handful of fixes that we didn't think were fitting for the RC release so we are submitting these for 4.9. Note there is a change to the core here but it is pretty harmless. There are also some improvements to rdmavt.
This applies on the last set of fixes I sent out for RC and can be seen in my git tree:
https://github.com/ddalessa/kernel/tree/for-4.9
---
Dean Luick (1):
IB/hfi1: Move serdes tune inside link start function
Harish Chegondi (1):
IB/hfi1: Fix the count of user packets submitted to an SDMA engine
Mike Marciniszyn (9):
IB/rdmavt: Add functions to get and release QP references IB/rdmavt,IB/qib,IB/hfi1: Use new QP put get routines IB/core: Add ib headers for general use IB/qib,IB/hfi: Use core common header file IB/rdmavt: Correct sparse annotation
IB/hfi1: Move iowait_init() to priv allocate IB/rdmavt: Move reset calldown to reset path IB/rdmavt: Add qp init function IB/rdmavt,IB/hfi1: Add lockdep asserts for lock debug
Tadeusz Struk (1):
IB/hfi1: Fix locking scheme for affinity settings
Tymoteusz Kielan (1):
IB/hfi1: Fix user-space buffers mapping with IOMMU enabled
drivers/infiniband/hw/hfi1/affinity.c | 91 ++++++++------- drivers/infiniband/hw/hfi1/affinity.h | 3 drivers/infiniband/hw/hfi1/chip.c | 40 +++---- drivers/infiniband/hw/hfi1/chip.h | 2 drivers/infiniband/hw/hfi1/common.h | 8 - drivers/infiniband/hw/hfi1/driver.c | 35 +++--- drivers/infiniband/hw/hfi1/file_ops.c | 41 +++++-- drivers/infiniband/hw/hfi1/hfi.h | 17 +-- drivers/infiniband/hw/hfi1/init.c | 44 ++++--- drivers/infiniband/hw/hfi1/mad.c | 1 drivers/infiniband/hw/hfi1/pio.c | 20 ++- drivers/infiniband/hw/hfi1/pio.h | 2 drivers/infiniband/hw/hfi1/qp.c | 33 +++-- drivers/infiniband/hw/hfi1/rc.c | 113 ++++++++++-------- drivers/infiniband/hw/hfi1/ruc.c | 4 - drivers/infiniband/hw/hfi1/sdma.c | 24 ++-- drivers/infiniband/hw/hfi1/sdma.h | 3 drivers/infiniband/hw/hfi1/trace.c | 27 +--- drivers/infiniband/hw/hfi1/trace_ctxts.h | 13 +- drivers/infiniband/hw/hfi1/trace_ibhdrs.h | 14 +- drivers/infiniband/hw/hfi1/trace_rx.h | 4 - drivers/infiniband/hw/hfi1/uc.c | 6 - drivers/infiniband/hw/hfi1/ud.c | 12 +- drivers/infiniband/hw/hfi1/user_sdma.c | 31 ++--- drivers/infiniband/hw/hfi1/verbs.c | 18 +-- drivers/infiniband/hw/hfi1/verbs.h | 90 +-------------- drivers/infiniband/hw/hfi1/verbs_txreq.c | 2 drivers/infiniband/hw/qib/qib.h | 1 drivers/infiniband/hw/qib/qib_driver.c | 7 - drivers/infiniband/hw/qib/qib_iba7322.c | 2 drivers/infiniband/hw/qib/qib_rc.c | 73 +++++------- drivers/infiniband/hw/qib/qib_ruc.c | 4 - drivers/infiniband/hw/qib/qib_uc.c | 6 - drivers/infiniband/hw/qib/qib_ud.c | 6 - drivers/infiniband/hw/qib/qib_verbs.c | 14 +- drivers/infiniband/hw/qib/qib_verbs.h | 94 ++------------- drivers/infiniband/sw/rdmavt/qp.c | 117 +++++++++++-------- include/rdma/ib_hdrs.h | 178 +++++++++++++++++++++++++++++ include/rdma/rdmavt_qp.h | 19 +++ 39 files changed, 643 insertions(+), 576 deletions(-) create mode 100644 include/rdma/ib_hdrs.h
--
-Denny
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
PATCH 1/13 IB/rdmavt: Add functions to get and release QP references
In reply to a message not in the archiveFrom: Mike Marciniszyn <[email protected]>
This centralizes the function and improves code readability.
--- include/rdma/rdmavt_qp.h | 19 +++++++++++++++++++ 1 files changed, 19 insertions(+), 0 deletions(-)
include/rdma/rdmavt_qp.h +19 -0
@@ -466,6 +466,25 @@ static inline struct rvt_rwqe *rvt_get_rwqe_ptr(struct rvt_rq *rq, unsigned n) }/** +* rvt_get_qp - get a QP reference +* @qp - the QP to hold +*/ +static inline void rvt_get_qp (struct rvt_qp *qp) +{ +atomic_inc (&qp->refcount); +} + +/** +* rvt_put_qp - release a QP reference +* @qp - the QP to release +*/ +static inline void rvt_put_qp (struct rvt_qp *qp) +{ +if (qp &&atomic_dec_and_test (&qp->refcount)) +wake_up (&qp->wait); +} + +/** * rvt_qp_wqe_reserve - reserve operation * @qp - the rvt qp * @wqe - the send wqe
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
PATCH 2/13 IB/rdmavt, IB/qib, IB/hfi1: Use new QP put get routines
In reply to a message not in the archiveFrom: Mike Marciniszyn <[email protected]>
This improves readability and hides the reference count mechanism from the client drivers.
--- drivers/infiniband/hw/hfi1/qp.c | 11 ++++------- drivers/infiniband/hw/hfi1/rc.c | 9 ++++----- drivers/infiniband/hw/hfi1/verbs.c | 4 ++-- drivers/infiniband/hw/hfi1/verbs_txreq.c | 2 +- drivers/infiniband/hw/qib/qib_driver.c | 3 +-- drivers/infiniband/hw/qib/qib_rc.c | 12 ++++++------ drivers/infiniband/sw/rdmavt/qp.c | 5 ++--- 7 files changed, 20 insertions(+), 26 deletions(-)
drivers/infiniband/hw/hfi1/qp.c +4 -7
@@ -202,8 +202,7 @@ static void flush_iowait(struct rvt_qp *qp)write_seqlock_irqsave (&dev->iowait_lock, flags);if (!list_empty (&priv->s_iowait.list)) {list_del_init (&priv->s_iowait.list); -if (atomic_dec_and_test (&qp->refcount)) -wake_up (&qp->wait); +rvt_put_qp (qp); }write_sequnlock_irqrestore (&dev->iowait_lock, flags); }
@@ -503,8 +502,7 @@ void hfi1_qp_wakeup(struct rvt_qp *qp, u32 flag) }spin_unlock_irqrestore (&qp->s_lock, flags);/* Notify hfi1_destroy_qp() if it is waiting. */ -if (atomic_dec_and_test (&qp->refcount)) -wake_up (&qp->wait); +rvt_put_qp (qp); } static int iowait_sleep(
@@ -544,7 +542,7 @@ static int iowait_sleep( qp->s_flags |=RVT_S_WAIT_DMA_DESC ;list_add_tail (&priv->s_iowait.list, &sde->dmawait);trace_hfi1_qpsleep (qp,RVT_S_WAIT_DMA_DESC ); -atomic_inc (&qp->refcount); +rvt_get_qp (qp); }write_sequnlock (&dev->iowait_lock); qp->s_flags &= ~RVT_S_BUSY ;
@@ -963,8 +961,7 @@ void notify_error_qp(struct rvt_qp *qp)if (!list_empty (&priv->s_iowait.list) && !(qp->s_flags &RVT_S_BUSY )) { qp->s_flags &= ~RVT_S_ANY_WAIT_IO ;list_del_init (&priv->s_iowait.list); -if (atomic_dec_and_test (&qp->refcount)) -wake_up (&qp->wait); +rvt_put_qp (qp); }write_sequnlock (&dev->iowait_lock);
drivers/infiniband/hw/hfi1/rc.c +4 -5
@@ -1389,7 +1389,7 @@ static int do_rc_ack(struct rvt_qp *qp, u32 aeth, u32 psn, int opcode,restart_rc (qp, qp->s_last_psn +1 ,0 );if (list_empty (&qp->rspwait)) { qp->r_flags |=RVT_R_RSP_SEND ; -atomic_inc (&qp->refcount); +rvt_get_qp (qp);list_add_tail (&qp->rspwait, &rcd->qp_wait_list); }
@@ -1573,7 +1573,7 @@ static void rdma_seq_err(struct rvt_qp *qp, struct hfi1_ibport *ibp, u32 psn,restart_rc (qp, qp->s_last_psn +1 ,0 );if (list_empty (&qp->rspwait)) { qp->r_flags |=RVT_R_RSP_SEND ; -atomic_inc (&qp->refcount); +rvt_get_qp (qp);list_add_tail (&qp->rspwait, &rcd->qp_wait_list); } }
@@ -1782,7 +1782,7 @@ static inline void rc_defered_ack(struct hfi1_ctxtdata *rcd, {if (list_empty (&qp->rspwait)) { qp->r_flags |=RVT_R_RSP_NAK ; -atomic_inc (&qp->refcount); +rvt_get_qp (qp);list_add_tail (&qp->rspwait, &rcd->qp_wait_list); } }
@@ -1796,8 +1796,7 @@ static inline void rc_cancel_ack(struct rvt_qp *qp)return ;list_del_init (&qp->rspwait); qp->r_flags &= ~RVT_R_RSP_NAK ; -if (atomic_dec_and_test (&qp->refcount)) -wake_up (&qp->wait); +rvt_put_qp (qp); }/**
drivers/infiniband/hw/hfi1/verbs.c +2 -2
@@ -748,7 +748,7 @@ static int wait_kmem(struct hfi1_ibdev *dev, qp->s_flags |=RVT_S_WAIT_KMEM ;list_add_tail (&priv->s_iowait.list, &dev->memwait);trace_hfi1_qpsleep (qp,RVT_S_WAIT_KMEM ); -atomic_inc (&qp->refcount); +rvt_get_qp (qp); }write_sequnlock (&dev->iowait_lock); qp->s_flags &= ~RVT_S_BUSY ;
@@ -959,7 +959,7 @@ static int pio_wait(struct rvt_qp *qp, was_empty =list_empty (&sc->piowait);list_add_tail (&priv->s_iowait.list, &sc->piowait);trace_hfi1_qpsleep (qp,RVT_S_WAIT_PIO ); -atomic_inc (&qp->refcount); +rvt_get_qp (qp);/* counting: only call wantpiobuf_intr if first user */ if (was_empty)hfi1_sc_wantpiobuf_intr (sc,1 );
drivers/infiniband/hw/hfi1/verbs_txreq.c +1 -1
@@ -109,7 +109,7 @@ struct verbs_txreq *__get_txreq(struct hfi1_ibdev *dev, qp->s_flags |=RVT_S_WAIT_TX ;list_add_tail (&priv->s_iowait.list, &dev->txwait);trace_hfi1_qpsleep (qp,RVT_S_WAIT_TX ); -atomic_inc (&qp->refcount); +rvt_get_qp (qp); } qp->s_flags &= ~RVT_S_BUSY ; }
drivers/infiniband/hw/qib/qib_driver.c +1 -2
@@ -588,8 +588,7 @@ move_along:qib_schedule_send (qp);spin_unlock_irqrestore (&qp->s_lock, flags); } -if (atomic_dec_and_test (&qp->refcount)) -wake_up (&qp->wait); +rvt_put_qp (qp); } bail:
drivers/infiniband/hw/qib/qib_rc.c +6 -6
@@ -1177,7 +1177,7 @@ static int do_rc_ack(struct rvt_qp *qp, u32 aeth, u32 psn, int opcode,qib_restart_rc (qp, qp->s_last_psn +1 ,0 );if (list_empty (&qp->rspwait)) { qp->r_flags |=RVT_R_RSP_SEND ; -atomic_inc (&qp->refcount); +rvt_get_qp (qp);list_add_tail (&qp->rspwait, &rcd->qp_wait_list); }
@@ -1361,7 +1361,7 @@ static void rdma_seq_err(struct rvt_qp *qp, struct qib_ibport *ibp, u32 psn,qib_restart_rc (qp, qp->s_last_psn +1 ,0 );if (list_empty (&qp->rspwait)) { qp->r_flags |=RVT_R_RSP_SEND ; -atomic_inc (&qp->refcount); +rvt_get_qp (qp);list_add_tail (&qp->rspwait, &rcd->qp_wait_list); } }
@@ -1640,7 +1640,7 @@ static int qib_rc_rcv_error(struct qib_other_headers *ohdr,*/ if (list_empty (&qp->rspwait)) { qp->r_flags |=RVT_R_RSP_NAK ; -atomic_inc (&qp->refcount); +rvt_get_qp (qp);list_add_tail (&qp->rspwait, &rcd->qp_wait_list); } }
@@ -2233,7 +2233,7 @@ rnr_nak:/* Queue RNR NAK for later */ if (list_empty (&qp->rspwait)) { qp->r_flags |=RVT_R_RSP_NAK ; -atomic_inc (&qp->refcount); +rvt_get_qp (qp);list_add_tail (&qp->rspwait, &rcd->qp_wait_list); }return ;
@@ -2245,7 +2245,7 @@ nack_op_err:/* Queue NAK for later */ if (list_empty (&qp->rspwait)) { qp->r_flags |=RVT_R_RSP_NAK ; -atomic_inc (&qp->refcount); +rvt_get_qp (qp);list_add_tail (&qp->rspwait, &rcd->qp_wait_list); }return ;
@@ -2259,7 +2259,7 @@ nack_inv:/* Queue NAK for later */ if (list_empty (&qp->rspwait)) { qp->r_flags |=RVT_R_RSP_NAK ; -atomic_inc (&qp->refcount); +rvt_get_qp (qp);list_add_tail (&qp->rspwait, &rcd->qp_wait_list); }return ;
drivers/infiniband/sw/rdmavt/qp.c +2 -3
@@ -488,8 +488,7 @@ static void rvt_remove_qp(struct rvt_dev_info *rdi, struct rvt_qp *qp)spin_unlock_irqrestore (&rdi->qp_dev->qpt_lock, flags);if (removed) {synchronize_rcu (); -if (atomic_dec_and_test (&qp->refcount)) -wake_up (&qp->wait); +rvt_put_qp (qp); } }
@@ -980,7 +979,7 @@ static void rvt_insert_qp(struct rvt_dev_info *rdi, struct rvt_qp *qp)struct rvt_ibport *rvp = rdi->ports[qp->port_num -1 ];unsigned long flags; -atomic_inc (&qp->refcount); +rvt_get_qp (qp);spin_lock_irqsave (&rdi->qp_dev->qpt_lock, flags);if (qp->ibqp.qp_num <=1 ) {
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
PATCH 3/13 IB/core: Add ib headers for general use
In reply to a message not in the archiveFrom: Mike Marciniszyn <[email protected]>
Add IB headers, defines, and accessors that are identical in both qib and hfi1 into the core includes.
The accessors for be maintenance of __be64 fields since alignment is potentially invalid and can differ based on the presense of the GRH.
{hfi1,qib}_ib_headers will be ib_headers. {hfi1,qib|_other_headers will be ib_other_headers.
--- include/rdma/ib_hdrs.h | 178 ++++++++++++++++++++++++++++++++++++++++++++++++ 1 files changed, 178 insertions(+), 0 deletions(-) create mode 100644 include/rdma/ib_hdrs.h
include/rdma/ib_hdrs.h new file +178 -0
@@ -0,0 +1,178 @@ +/* +* Copyright(c) 2016 Intel Corporation. +* +* This file is provided under a dual BSD/GPLv2 license. When using or +* redistributing this file, you may do so under either license. +* +* GPL LICENSE SUMMARY +* +* This program is free software; you can redistribute it and/or modify +* it under the terms of version 2 of the GNU General Public License as +* published by the Free Software Foundation. +* +* This program is distributed in the hope that it will be useful, but +* WITHOUT ANY WARRANTY; without even the implied warranty of +* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU +* General Public License for more details. +* +* BSD LICENSE +* +* Redistribution and use in source and binary forms, with or without +* modification, are permitted provided that the following conditions +* are met: +* +* - Redistributions of source code must retain the above copyright +* notice, this list of conditions and the following disclaimer. +* - Redistributions in binary form must reproduce the above copyright +* notice, this list of conditions and the following disclaimer in +* the documentation and/or other materials provided with the +* distribution. +* - Neither the name of Intel Corporation nor the names of its +* contributors may be used to endorse or promote products derived +* from this software without specific prior written permission. +* +* THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS +* "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT +* LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR +* A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT +* OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, +* SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT +* LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, +* DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY +* THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT +* (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE +* OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. +* +*/ + +#ifndef IB_HDRS_H +#define IB_HDRS_H + +#include <linux/types.h> +#include <asm/unaligned.h> +#include <rdma/ib_verbs.h> + +#define IB_SEQ_NAK (3 << 29) + +/* AETH NAK opcode values */ +#define IB_RNR_NAK 0x20 +#define IB_NAK_PSN_ERROR 0x60 +#define IB_NAK_INVALID_REQUEST 0x61 +#define IB_NAK_REMOTE_ACCESS_ERROR 0x62 +#define IB_NAK_REMOTE_OPERATIONAL_ERROR 0x63 +#define IB_NAK_INVALID_RD_REQUEST 0x64 + +#define IB_BTH_REQ_ACK BIT(31) +#define IB_BTH_SOLICITED BIT(23) +#define IB_BTH_MIG_REQ BIT(22) + +#define IB_GRH_VERSION 6 +#define IB_GRH_VERSION_MASK 0xF +#define IB_GRH_VERSION_SHIFT 28 +#define IB_GRH_TCLASS_MASK 0xFF +#define IB_GRH_TCLASS_SHIFT 20 +#define IB_GRH_FLOW_MASK 0xFFFFF +#define IB_GRH_FLOW_SHIFT 0 +#define IB_GRH_NEXT_HDR 0x1B + +struct ib_reth { +__be64 vaddr;/* potentially unaligned */ +__be32 rkey; +__be32 length; +} __packed; + +struct ib_atomic_eth { +__be64 vaddr;/* potentially unaligned */ +__be32 rkey; +__be64 swap_data;/* potentially unaligned */ +__be64 compare_data;/* potentially unaligned */ +} __packed; + +union ib_ehdrs { +struct { +__be32 deth[2 ]; +__be32 imm_data; + } ud; +struct { +struct ib_reth reth; +__be32 imm_data; + } rc; +struct { +__be32 aeth; +__be64 atomic_ack_eth;/* potentially unaligned */ + } at; +__be32 imm_data; +__be32 aeth; +__be32 ieth; +struct ib_atomic_eth atomic_eth; +} __packed; + +struct ib_other_headers { +__be32 bth[3 ]; +union ib_ehdrs u; +} __packed; + +struct ib_header { +__be16 lrh[4 ]; +union { +struct { +struct ib_grh grh; +struct ib_other_headers oth; + } l; +struct ib_other_headers oth; + } u; +} __packed; + +/* accessors for unaligned __be64 items */ + +static u64 ib_u64_get (__be64 *p) +{ +return get_unaligned_be64 (p); +} + +static void ib_u64_put (u64 val,__be64 *p) +{ +put_unaligned_be64 (val, p); +} + +static inline u64 get_ib_reth_vaddr (struct ib_reth *reth) +{ +return ib_u64_get (&reth->vaddr); +} + +static inline void put_ib_reth_vaddr (u64 val,struct ib_reth *reth) +{ +ib_u64_put (val, &reth->vaddr); +} + +static inline u64 get_ib_ateth_vaddr (struct ib_atomic_eth *ateth) +{ +return ib_u64_get (&ateth->vaddr); +} + +static inline void put_ib_ateth_vaddr (u64 val,struct ib_atomic_eth *ateth) +{ +ib_u64_put (val, &ateth->vaddr); +} + +static inline u64 get_ib_ateth_swap (struct ib_atomic_eth *ateth) +{ +return ib_u64_get (&ateth->swap_data); +} + +static inline void put_ib_ateth_swap (u64 val,struct ib_atomic_eth *ateth) +{ +ib_u64_put (val, &ateth->swap_data); +} + +static inline u64 get_ib_ateth_compare (struct ib_atomic_eth *ateth) +{ +return ib_u64_get (&ateth->compare_data); +} + +static inline void put_ib_ateth_compare (u64 val,struct ib_atomic_eth *ateth) +{ +ib_u64_put (val, &ateth->compare_data); +} + +#endif /* IB_HDRS_H */
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
PATCH 4/13 IB/qib,IB/hfi: Use core common header file
In reply to a message not in the archiveFrom: Mike Marciniszyn <[email protected]>
Use common header file structs, defines, and accessors in the drivers. The old declarations are removed.
The repositioning of the includes allows for the removal of hfi1_message_header and replaces its use with ib_header.
Also corrected are two issues with set_armed_to_active():
- The "packet" parameter is now a pointer as it should have been
- The etype is validated to insure that the header is correct
--- drivers/infiniband/hw/hfi1/chip.c | 6 +- drivers/infiniband/hw/hfi1/chip.h | 2 - drivers/infiniband/hw/hfi1/common.h | 8 -- drivers/infiniband/hw/hfi1/driver.c | 35 +++++------ drivers/infiniband/hw/hfi1/hfi.h | 7 +- drivers/infiniband/hw/hfi1/rc.c | 87 ++++++++++++--------------- drivers/infiniband/hw/hfi1/ruc.c | 4 + drivers/infiniband/hw/hfi1/trace.c | 27 ++------ drivers/infiniband/hw/hfi1/trace_ibhdrs.h | 14 ++-- drivers/infiniband/hw/hfi1/trace_rx.h | 4 + drivers/infiniband/hw/hfi1/uc.c | 6 +- drivers/infiniband/hw/hfi1/ud.c | 12 ++-- drivers/infiniband/hw/hfi1/verbs.c | 14 ++-- drivers/infiniband/hw/hfi1/verbs.h | 90 ++-------------------------- drivers/infiniband/hw/qib/qib.h | 1 drivers/infiniband/hw/qib/qib_driver.c | 4 + drivers/infiniband/hw/qib/qib_iba7322.c | 2 - drivers/infiniband/hw/qib/qib_rc.c | 61 ++++++++----------- drivers/infiniband/hw/qib/qib_ruc.c | 4 + drivers/infiniband/hw/qib/qib_uc.c | 6 +- drivers/infiniband/hw/qib/qib_ud.c | 6 +- drivers/infiniband/hw/qib/qib_verbs.c | 14 ++-- drivers/infiniband/hw/qib/qib_verbs.h | 94 +++-------------------------- 23 files changed, 162 insertions(+), 346 deletions(-)
drivers/infiniband/hw/hfi1/chip.c +3 -3
@@ -9718,12 +9718,12 @@ void hfi1_clear_tids(struct hfi1_ctxtdata *rcd)hfi1_put_tid (dd, i,PT_INVALID ,0 ,0 ); } -struct hfi1_message_header *hfi1_get_msgheader ( -struct hfi1_devdata *dd,__le32 *rhf_addr) +struct ib_header *hfi1_get_msgheader ( +struct hfi1_devdata *dd,__le32 *rhf_addr) {u32 offset =rhf_hdrq_offset (rhf_to_cpu (rhf_addr)); -return (struct hfi1_message_header *) +return (struct ib_header *) (rhf_addr - dd->rhf_offset + offset); }
drivers/infiniband/hw/hfi1/chip.h +1 -1
@@ -1336,7 +1336,7 @@ enum {u64 get_all_cpu_total (u64 __percpu *cntr);void hfi1_start_cleanup (struct hfi1_devdata *dd);void hfi1_clear_tids (struct hfi1_ctxtdata *rcd); -struct hfi1_message_header *hfi1_get_msgheader ( +struct ib_header *hfi1_get_msgheader (struct hfi1_devdata *dd,__le32 *rhf_addr);int hfi1_init_ctxt (struct send_context *sc); void hfi1_put_tid(struct hfi1_devdata *dd, u32 index,
drivers/infiniband/hw/hfi1/common.h +0 -8
@@ -320,14 +320,6 @@ struct diag_pkt {/* RHF receive type error - bypass packet errors */ #define RHF_RTE_BYPASS_NO_ERR 0x0 -/* -* This structure contains the first field common to all protocols -* that employ this chip. -*/ -struct hfi1_message_header { -__be16 lrh[4 ]; -}; -/* IB - LRH header constants */ #define HFI1_LRH_GRH 0x0003/* 1. word of IB LRH - next header: GRH */ #define HFI1_LRH_BTH 0x0002/* 1. word of IB LRH - next header: BTH */
drivers/infiniband/hw/hfi1/driver.c +17 -18
@@ -276,7 +276,7 @@ inline int hfi1_rcvbuf_validate(u32 size, u8 type, u16 *encoded)static void rcv_hdrerr (struct hfi1_ctxtdata *rcd,struct hfi1_pportdata *ppd,struct hfi1_packet *packet) { -struct hfi1_message_header *rhdr = packet->hdr; +struct ib_header *rhdr = packet->hdr;u32 rte =rhf_rcv_type_err (packet->rhf);int lnh =be16_to_cpu (rhdr->lrh[0 ]) &3 ;struct hfi1_ibport *ibp = &ppd->ibport_data;
@@ -288,10 +288,9 @@ static void rcv_hdrerr(struct hfi1_ctxtdata *rcd, struct hfi1_pportdata *ppd,if (packet->rhf &RHF_TID_ERR ) {/* For TIDERR and RC QPs preemptively schedule a NAK */ -struct hfi1_ib_header *hdr = (struct hfi1_ib_header *)rhdr; -struct hfi1_other_headers *ohdr =NULL ; +struct ib_other_headers *ohdr =NULL ;u32 tlen =rhf_pkt_len (packet->rhf);/* in bytes */ -u16 lid =be16_to_cpu (hdr->lrh[1 ]); +u16 lid =be16_to_cpu (rhdr->lrh[1 ]);u32 qp_num;u32 rcv_flags =0 ;
@@ -301,14 +300,14 @@ static void rcv_hdrerr(struct hfi1_ctxtdata *rcd, struct hfi1_pportdata *ppd,/* Check for GRH */ if (lnh ==HFI1_LRH_BTH ) { - ohdr = &hdr->u.oth; + ohdr = &rhdr->u.oth; }else if (lnh ==HFI1_LRH_GRH ) {u32 vtf; - ohdr = &hdr->u.l.oth; -if (hdr->u.l.grh.next_hdr !=IB_GRH_NEXT_HDR ) + ohdr = &rhdr->u.l.oth; +if (rhdr->u.l.grh.next_hdr !=IB_GRH_NEXT_HDR ) goto drop; - vtf =be32_to_cpu (hdr->u.l.grh.version_tclass_flow); + vtf =be32_to_cpu (rhdr->u.l.grh.version_tclass_flow);if ((vtf >>IB_GRH_VERSION_SHIFT ) !=IB_GRH_VERSION ) goto drop; rcv_flags |=HFI1_HAS_GRH ;
@@ -344,7 +343,7 @@ static void rcv_hdrerr(struct hfi1_ctxtdata *rcd, struct hfi1_pportdata *ppd,case IB_QPT_RC :hfi1_rc_hdrerr ( rcd, - hdr, + rhdr, rcv_flags, qp);break ;
@@ -452,8 +451,8 @@ void hfi1_process_ecn_slowpath(struct rvt_qp *qp, struct hfi1_packet *pkt, bool do_cnp) {struct hfi1_ibport *ibp =to_iport (qp->ibqp.device, qp->port_num); -struct hfi1_ib_header *hdr = pkt->hdr; -struct hfi1_other_headers *ohdr = pkt->ohdr; +struct ib_header *hdr = pkt->hdr; +struct ib_other_headers *ohdr = pkt->ohdr;struct ib_grh *grh =NULL ;u32 rqpn =0 , bth1;u16 rlid, dlid =be16_to_cpu (hdr->lrh[1 ]);
@@ -487,7 +486,7 @@ void hfi1_process_ecn_slowpath(struct rvt_qp *qp, struct hfi1_packet *pkt,return ; } - sc =hdr2sc ((struct hfi1_message_header *)hdr, pkt->rhf); + sc =hdr2sc (hdr, pkt->rhf); bth1 =be32_to_cpu (ohdr->bth[1 ]);if (do_cnp && (bth1 &HFI1_FECN_SMASK )) {
@@ -599,8 +598,8 @@ static void __prescan_rxq(struct hfi1_packet *packet)__le32 *rhf_addr = (__le32 *)rcd->rcvhdrq + mdata.ps_head + dd->rhf_offset;struct rvt_qp *qp; -struct hfi1_ib_header *hdr; -struct hfi1_other_headers *ohdr; +struct ib_header *hdr; +struct ib_other_headers *ohdr;struct rvt_dev_info *rdi = &dd->verbs_dev.rdi;u64 rhf =rhf_to_cpu (rhf_addr);u32 etype =rhf_rcv_type (rhf), qpn, bth1;
@@ -616,8 +615,8 @@ static void __prescan_rxq(struct hfi1_packet *packet)if (etype !=RHF_RCV_TYPE_IB ) goto next; - hdr = (struct hfi1_ib_header *) -hfi1_get_msgheader (dd, rhf_addr); + hdr =hfi1_get_msgheader (dd, rhf_addr); + lnh =be16_to_cpu (hdr->lrh[0 ]) &3 ;if (lnh ==HFI1_LRH_BTH )
@@ -892,8 +891,8 @@ static inline int set_armed_to_active(struct hfi1_ctxtdata *rcd, struct hfi1_devdata *dd) {struct work_struct *lsaw = &rcd->ppd->linkstate_active_work; -struct hfi1_message_header *hdr =hfi1_get_msgheader (packet->rcd->dd, - packet->rhf_addr); +struct ib_header *hdr =hfi1_get_msgheader (packet->rcd->dd, + packet->rhf_addr);u8 etype =rhf_rcv_type (packet->rhf);if (etype ==RHF_RCV_TYPE_IB &&hdr2sc (hdr, packet->rhf) !=0xf ) {
drivers/infiniband/hw/hfi1/hfi.h +4 -3
@@ -64,6 +64,7 @@#include <linux/kthread.h> #include <linux/i2c.h> #include <linux/i2c-algo-bit.h> +#include <rdma/ib_hdrs.h> #include <rdma/rdma_vt.h> #include "chip_registers.h"
@@ -350,7 +351,7 @@ struct hfi1_packet {struct hfi1_ctxtdata *rcd;__le32 *rhf_addr;struct rvt_qp *qp; -struct hfi1_other_headers *ohdr; +struct ib_other_headers *ohdr;u64 rhf;u32 maxcnt;u32 rhqoff;
@@ -1268,7 +1269,7 @@ static inline u32 driver_lstate(struct hfi1_pportdata *ppd)void receive_interrupt_work (struct work_struct *work);/* extract service channel from header and rhf */ -static inline int hdr2sc (struct hfi1_message_header *hdr,u64 rhf) +static inline int hdr2sc (struct ib_header *hdr,u64 rhf) {return ((be16_to_cpu (hdr->lrh[0 ]) >>12 ) &0xf ) | ((!!(rhf_dc_info (rhf))) <<4 );
@@ -1603,7 +1604,7 @@ void hfi1_process_ecn_slowpath(struct rvt_qp *qp, struct hfi1_packet *pkt,static inline bool process_ecn (struct rvt_qp *qp,struct hfi1_packet *pkt,bool do_cnp) { -struct hfi1_other_headers *ohdr = pkt->ohdr; +struct ib_other_headers *ohdr = pkt->ohdr;u32 bth1; bth1 =be32_to_cpu (ohdr->bth[1 ]);
drivers/infiniband/hw/hfi1/rc.c +40 -47
218 lines not on this page · raw message
drivers/infiniband/hw/hfi1/ruc.c +2 -2
@@ -262,7 +262,7 @@ static int gid_ok(union ib_gid *gid, __be64 gid_prefix, __be64 id)* * The s_lock will be acquired around the hfi1_migrate_qp() call. */ -int hfi1_ruc_check_hdr (struct hfi1_ibport *ibp,struct hfi1_ib_header *hdr, +int hfi1_ruc_check_hdr (struct hfi1_ibport *ibp,struct ib_header *hdr,int has_grh,struct rvt_qp *qp,u32 bth0) {__be64 guid;
@@ -765,7 +765,7 @@ static inline void build_ahg(struct rvt_qp *qp, u32 npsn) } } -void hfi1_make_ruc_header (struct rvt_qp *qp,struct hfi1_other_headers *ohdr, +void hfi1_make_ruc_header (struct rvt_qp *qp,struct ib_other_headers *ohdr,u32 bth0,u32 bth2,int middle,struct hfi1_pkt_state *ps) {
drivers/infiniband/hw/hfi1/trace.c +8 -19
@@ -47,9 +47,9 @@#define CREATE_TRACE_POINTS #include "trace.h" -u8 ibhdr_exhdr_len (struct hfi1_ib_header *hdr) +u8 ibhdr_exhdr_len (struct ib_header *hdr) { -struct hfi1_other_headers *ohdr; +struct ib_other_headers *ohdr;u8 opcode;u8 lnh = (u8)(be16_to_cpu (hdr->lrh[0 ]) &3 );
@@ -72,11 +72,6 @@ u8 ibhdr_exhdr_len(struct hfi1_ib_header *hdr)#define OP (transport, op) IB_OPCODE_## transport ## _ ## op -static u64 ib_u64_get (__be32 *p) -{ -return ((u64 )be32_to_cpu (p[0 ]) <<32 ) |be32_to_cpu (p[1 ]); -} -static const char *parse_syndrome (u8 syndrome) {switch (syndrome >>5 ) {
@@ -113,8 +108,7 @@ const char *parse_everbs_hdrs(case OP (RC ,RDMA_WRITE_ONLY_WITH_IMMEDIATE ):case OP (UC ,RDMA_WRITE_ONLY_WITH_IMMEDIATE ):trace_seq_printf (p,RETH_PRN " " IMM_PRN , - (unsigned long long )ib_u64_get ( - (__be32 *)&eh->rc.reth.vaddr), +get_ib_reth_vaddr (&eh->rc.reth),be32_to_cpu (eh->rc.reth.rkey),be32_to_cpu (eh->rc.reth.length),be32_to_cpu (eh->rc.imm_data));
@@ -126,8 +120,7 @@ const char *parse_everbs_hdrs(case OP (RC ,RDMA_WRITE_ONLY ):case OP (UC ,RDMA_WRITE_ONLY ):trace_seq_printf (p,RETH_PRN , - (unsigned long long )ib_u64_get ( - (__be32 *)&eh->rc.reth.vaddr), +get_ib_reth_vaddr (&eh->rc.reth),be32_to_cpu (eh->rc.reth.rkey),be32_to_cpu (eh->rc.reth.length));break ;
@@ -145,20 +138,16 @@ const char *parse_everbs_hdrs(be32_to_cpu (eh->at.aeth) >>24 ,parse_syndrome (be32_to_cpu (eh->at.aeth) >>24 ),be32_to_cpu (eh->at.aeth) &HFI1_MSN_MASK , - (unsigned long long ) - ib_u64_get(eh->at.atomic_ack_eth)); + ib_u64_get(&eh->at.atomic_ack_eth));break ;/* atomiceth */ case OP (RC ,COMPARE_SWAP ):case OP (RC ,FETCH_ADD ):trace_seq_printf (p,ATOMICETH_PRN , - (unsigned long long )ib_u64_get ( - eh->atomic_eth.vaddr), +get_ib_ateth_vaddr (&eh->atomic_eth), eh->atomic_eth.rkey, - (unsigned long long )ib_u64_get ( - (__be32 *)&eh->atomic_eth.swap_data), - (unsigned long long )ib_u64_get ( - (__be32 *)&eh->atomic_eth.compare_data)); +get_ib_ateth_swap (&eh->atomic_eth), +get_ib_ateth_compare (&eh->atomic_eth));break ;/* deth */ case OP (UD ,SEND_ONLY ):
drivers/infiniband/hw/hfi1/trace_ibhdrs.h +7 -7
@@ -55,7 +55,7 @@#undef TRACE_SYSTEM#define TRACE_SYSTEM hfi1_ibhdrs -u8 ibhdr_exhdr_len (struct hfi1_ib_header *hdr); +u8 ibhdr_exhdr_len (struct ib_header *hdr);const char *parse_everbs_hdrs (struct trace_seq *p,u8 opcode,void *ehdrs);#define __parse_ib_ehdrs (op, ehdrs) parse_everbs_hdrs(p, op, ehdrs)
@@ -74,7 +74,7 @@ __print_symbolic(lrh, \ DECLARE_EVENT_CLASS(hfi1_ibhdr_template, TP_PROTO(struct hfi1_devdata *dd, - struct hfi1_ib_header *hdr), + struct ib_header *hdr), TP_ARGS(dd, hdr), TP_STRUCT__entry( DD_DEV_ENTRY(dd)
@@ -102,7 +102,7 @@ DECLARE_EVENT_CLASS(hfi1_ibhdr_template,__dynamic_array (u8, ehdrs,ibhdr_exhdr_len (hdr)) ),TP_fast_assign ( - struct hfi1_other_headers *ohdr; + struct ib_other_headers *ohdr; DD_DEV_ASSIGN(dd);/* LRH */
@@ -185,19 +185,19 @@ DECLARE_EVENT_CLASS(hfi1_ibhdr_template, );DEFINE_EVENT (hfi1_ibhdr_template, input_ibhdr, - TP_PROTO(struct hfi1_devdata *dd, struct hfi1_ib_header *hdr), + TP_PROTO(struct hfi1_devdata *dd, struct ib_header *hdr),TP_ARGS (dd, hdr));DEFINE_EVENT (hfi1_ibhdr_template, pio_output_ibhdr, - TP_PROTO(struct hfi1_devdata *dd, struct hfi1_ib_header *hdr), + TP_PROTO(struct hfi1_devdata *dd, struct ib_header *hdr),TP_ARGS (dd, hdr));DEFINE_EVENT (hfi1_ibhdr_template, ack_output_ibhdr, - TP_PROTO(struct hfi1_devdata *dd, struct hfi1_ib_header *hdr), + TP_PROTO(struct hfi1_devdata *dd, struct ib_header *hdr),TP_ARGS (dd, hdr));DEFINE_EVENT (hfi1_ibhdr_template, sdma_output_ibhdr, - TP_PROTO(struct hfi1_devdata *dd, struct hfi1_ib_header *hdr), + TP_PROTO(struct hfi1_devdata *dd, struct ib_header *hdr),TP_ARGS (dd, hdr));#endif /* __HFI1_TRACE_IBHDRS_H */
drivers/infiniband/hw/hfi1/trace_rx.h +2 -2
@@ -260,7 +260,7 @@ TRACE_EVENT(hfi1_mmu_invalidate, TRACE_EVENT(snoop_capture, TP_PROTO(struct hfi1_devdata *dd, int hdr_len, - struct hfi1_ib_header *hdr, + struct ib_header *hdr, int data_len, void *data), TP_ARGS(dd, hdr_len, hdr, data_len, data),
@@ -279,7 +279,7 @@ TRACE_EVENT(snoop_capture,__dynamic_array (u8, raw_pkt, data_len) ),TP_fast_assign ( - struct hfi1_other_headers *ohdr; + struct ib_other_headers *ohdr; __entry->lnh = (u8)(be16_to_cpu(hdr->lrh[0]) & 3);if (__entry->lnh ==HFI1_LRH_BTH )
drivers/infiniband/hw/hfi1/uc.c +3 -3
@@ -70,7 +70,7 @@ const u32 uc_only_opcode =int hfi1_make_uc_req (struct rvt_qp *qp,struct hfi1_pkt_state *ps) {struct hfi1_qp_priv *priv = qp->priv; -struct hfi1_other_headers *ohdr; +struct ib_other_headers *ohdr;struct rvt_swqe *wqe;u32 hwords =5 ;u32 bth0 =0 ;
@@ -304,12 +304,12 @@ bail_no_tx:void hfi1_uc_rcv (struct hfi1_packet *packet) {struct hfi1_ibport *ibp = &packet->rcd->ppd->ibport_data; -struct hfi1_ib_header *hdr = packet->hdr; +struct ib_header *hdr = packet->hdr;u32 rcv_flags = packet->rcv_flags;void *data = packet->ebuf;u32 tlen = packet->tlen;struct rvt_qp *qp = packet->qp; -struct hfi1_other_headers *ohdr = packet->ohdr; +struct ib_other_headers *ohdr = packet->ohdr;u32 bth0, opcode;u32 hdrsize = packet->hlen;u32 psn;
drivers/infiniband/hw/hfi1/ud.c +6 -6
@@ -271,7 +271,7 @@ drop:int hfi1_make_ud_req (struct rvt_qp *qp,struct hfi1_pkt_state *ps) {struct hfi1_qp_priv *priv = qp->priv; -struct hfi1_other_headers *ohdr; +struct ib_other_headers *ohdr;struct ib_ah_attr *ah_attr;struct hfi1_pportdata *ppd;struct hfi1_ibport *ibp;
@@ -510,8 +510,8 @@ void return_cnp(struct hfi1_ibport *ibp, struct rvt_qp *qp, u32 remote_qpn,u32 bth0, plen, vl, hwords =5 ;u16 lrh0;u8 sl = ibp->sc_to_sl[sc5]; -struct hfi1_ib_header hdr; -struct hfi1_other_headers *ohdr; +struct ib_header hdr; +struct ib_other_headers *ohdr;struct pio_buf *pbuf;struct send_context *ctxt =qp_to_send_context (qp, sc5);struct hfi1_pportdata *ppd =ppd_from_ibp (ibp);
@@ -665,7 +665,7 @@ static int opa_smp_check(struct hfi1_ibport *ibp, u16 pkey, u8 sc5,*/ void hfi1_ud_rcv (struct hfi1_packet *packet) { -struct hfi1_other_headers *ohdr = packet->ohdr; +struct ib_other_headers *ohdr = packet->ohdr;int opcode;u32 hdrsize = packet->hlen;struct ib_wc wc;
@@ -675,13 +675,13 @@ void hfi1_ud_rcv(struct hfi1_packet *packet)int mgmt_pkey_idx =-1 ;struct hfi1_ibport *ibp = &packet->rcd->ppd->ibport_data;struct hfi1_pportdata *ppd =ppd_from_ibp (ibp); -struct hfi1_ib_header *hdr = packet->hdr; +struct ib_header *hdr = packet->hdr;u32 rcv_flags = packet->rcv_flags;void *data = packet->ebuf;u32 tlen = packet->tlen;struct rvt_qp *qp = packet->qp;bool has_grh = rcv_flags &HFI1_HAS_GRH ; -u8 sc5 =hdr2sc ((struct hfi1_message_header *)hdr, packet->rhf); +u8 sc5 =hdr2sc (hdr, packet->rhf);u32 bth1;u8 sl_from_sc, sl;u16 slid;
drivers/infiniband/hw/hfi1/verbs.c +7 -7
@@ -567,7 +567,7 @@ static inline opcode_handler qp_ok(int opcode, struct hfi1_packet *packet)void hfi1_ib_rcv (struct hfi1_packet *packet) {struct hfi1_ctxtdata *rcd = packet->rcd; -struct hfi1_ib_header *hdr = packet->hdr; +struct ib_header *hdr = packet->hdr;u32 tlen = packet->tlen;struct hfi1_pportdata *ppd = rcd->ppd;struct hfi1_ibport *ibp = &ppd->ibport_data;
@@ -719,7 +719,7 @@ static void verbs_sdma_complete(if (tx->wqe) {hfi1_send_complete (qp, tx->wqe,IB_WC_SUCCESS ); }else if (qp->ibqp.qp_type ==IB_QPT_RC ) { -struct hfi1_ib_header *hdr; +struct ib_header *hdr; hdr = &tx->phdr.hdr;hfi1_rc_send_complete (qp, hdr);
@@ -1200,7 +1200,7 @@ static inline send_routine get_send_routine(struct rvt_qp *qp, {struct hfi1_devdata *dd =dd_from_ibdev (qp->ibqp.device);struct hfi1_qp_priv *priv = qp->priv; -struct hfi1_ib_header *h = &tx->phdr.hdr; +struct ib_header *h = &tx->phdr.hdr;if (unlikely (!(dd->flags &HFI1_HAS_SEND_DMA )))return dd->process_pio_send;
@@ -1244,8 +1244,8 @@ int hfi1_verbs_send(struct rvt_qp *qp, struct hfi1_pkt_state *ps) {struct hfi1_devdata *dd =dd_from_ibdev (qp->ibqp.device);struct hfi1_qp_priv *priv = qp->priv; -struct hfi1_other_headers *ohdr; -struct hfi1_ib_header *hdr; +struct ib_other_headers *ohdr; +struct ib_header *hdr;send_routine sr;int ret;u8 lnh;
@@ -1754,7 +1754,7 @@ void hfi1_cnp_rcv(struct hfi1_packet *packet) {struct hfi1_ibport *ibp = &packet->rcd->ppd->ibport_data;struct hfi1_pportdata *ppd =ppd_from_ibp (ibp); -struct hfi1_ib_header *hdr = packet->hdr; +struct ib_header *hdr = packet->hdr;struct rvt_qp *qp = packet->qp;u32 lqpn, rqpn =0 ;u16 rlid =0 ;
@@ -1781,7 +1781,7 @@ void hfi1_cnp_rcv(struct hfi1_packet *packet)return ; } - sc5 =hdr2sc ((struct hfi1_message_header *)hdr, packet->rhf); + sc5 =hdr2sc (hdr, packet->rhf); sl = ibp->sc_to_sl[sc5]; lqpn = qp->ibqp.qp_num;
drivers/infiniband/hw/hfi1/verbs.h +7 -83
149 lines not on this page · raw message
drivers/infiniband/hw/qib/qib.h +1 -0
@@ -52,6 +52,7 @@#include <linux/kref.h> #include <linux/sched.h> #include <linux/kthread.h> +#include <rdma/ib_hdrs.h> #include <rdma/rdma_vt.h> #include "qib_common.h"
drivers/infiniband/hw/qib/qib_driver.c +2 -2
@@ -319,8 +319,8 @@ static u32 qib_rcv_hdrerr(struct qib_ctxtdata *rcd, struct qib_pportdata *ppd, ret =1 ; else if (eflags == QLOGIC_IB_RHF_H_TIDERR) {/* For TIDERR and RC QPs premptively schedule a NAK */ -struct qib_ib_header *hdr = (struct qib_ib_header *) rhdr; -struct qib_other_headers *ohdr =NULL ; +struct ib_header *hdr = (struct ib_header *)rhdr; +struct ib_other_headers *ohdr =NULL ;struct qib_ibport *ibp = &ppd->ibport_data;struct qib_devdata *dd = ppd->dd;struct rvt_dev_info *rdi = &dd->verbs_dev.rdi;
drivers/infiniband/hw/qib/qib_iba7322.c +1 -1
@@ -1415,7 +1415,7 @@ static void flush_fifo(struct qib_pportdata *ppd)u32 *hdr;u64 pbc;const unsigned hdrwords =7 ; -static struct qib_ib_header ibhdr = { +static struct ib_header ibhdr = { .lrh[0 ] =cpu_to_be16 (0xF000 |QIB_LRH_BTH ), .lrh[1 ] =IB_LID_PERMISSIVE , .lrh[2 ] =cpu_to_be16 (hdrwords +SIZE_OF_CRC ),
drivers/infiniband/hw/qib/qib_rc.c +26 -35
143 lines not on this page · raw message
drivers/infiniband/hw/qib/qib_ruc.c +2 -2
@@ -265,7 +265,7 @@ static int gid_ok(union ib_gid *gid, __be64 gid_prefix, __be64 id)* * The s_lock will be acquired around the qib_migrate_qp() call. */ -int qib_ruc_check_hdr (struct qib_ibport *ibp,struct qib_ib_header *hdr, +int qib_ruc_check_hdr (struct qib_ibport *ibp,struct ib_header *hdr,int has_grh,struct rvt_qp *qp,u32 bth0) {__be64 guid;
@@ -680,7 +680,7 @@ u32 qib_make_grh(struct qib_ibport *ibp, struct ib_grh *hdr,return sizeof (struct ib_grh ) /sizeof (u32); } -void qib_make_ruc_header (struct rvt_qp *qp,struct qib_other_headers *ohdr, +void qib_make_ruc_header (struct rvt_qp *qp,struct ib_other_headers *ohdr,u32 bth0,u32 bth2) {struct qib_qp_priv *priv = qp->priv;
drivers/infiniband/hw/qib/qib_uc.c +3 -3
@@ -48,7 +48,7 @@int qib_make_uc_req (struct rvt_qp *qp,unsigned long *flags) {struct qib_qp_priv *priv = qp->priv; -struct qib_other_headers *ohdr; +struct ib_other_headers *ohdr;struct rvt_swqe *wqe;u32 hwords;u32 bth0;
@@ -236,10 +236,10 @@ bail:* for the given QP. * Called at interrupt level. */ -void qib_uc_rcv (struct qib_ibport *ibp,struct qib_ib_header *hdr, +void qib_uc_rcv (struct qib_ibport *ibp,struct ib_header *hdr,int has_grh,void *data,u32 tlen,struct rvt_qp *qp) { -struct qib_other_headers *ohdr; +struct ib_other_headers *ohdr;u32 opcode;u32 hdrsize;u32 psn;
drivers/infiniband/hw/qib/qib_ud.c +3 -3
@@ -245,7 +245,7 @@ drop:int qib_make_ud_req (struct rvt_qp *qp,unsigned long *flags) {struct qib_qp_priv *priv = qp->priv; -struct qib_other_headers *ohdr; +struct ib_other_headers *ohdr;struct ib_ah_attr *ah_attr;struct qib_pportdata *ppd;struct qib_ibport *ibp;
@@ -435,10 +435,10 @@ static unsigned qib_lookup_pkey(struct qib_ibport *ibp, u16 pkey)* for the given QP. * Called at interrupt level. */ -void qib_ud_rcv (struct qib_ibport *ibp,struct qib_ib_header *hdr, +void qib_ud_rcv (struct qib_ibport *ibp,struct ib_header *hdr,int has_grh,void *data,u32 tlen,struct rvt_qp *qp) { -struct qib_other_headers *ohdr; +struct ib_other_headers *ohdr;int opcode;u32 hdrsize;u32 pad;
drivers/infiniband/hw/qib/qib_verbs.c +7 -7
@@ -313,7 +313,7 @@ static void qib_copy_from_sge(void *data, struct rvt_sge_state *ss, u32 length)* for the given QP. * Called at interrupt level. */ -static void qib_qp_rcv (struct qib_ctxtdata *rcd,struct qib_ib_header *hdr, +static void qib_qp_rcv (struct qib_ctxtdata *rcd,struct ib_header *hdr,int has_grh,void *data,u32 tlen,struct rvt_qp *qp) {struct qib_ibport *ibp = &rcd->ppd->ibport_data;
@@ -366,10 +366,10 @@ void qib_ib_rcv(struct qib_ctxtdata *rcd, void *rhdr, void *data, u32 tlen) {struct qib_pportdata *ppd = rcd->ppd;struct qib_ibport *ibp = &ppd->ibport_data; -struct qib_ib_header *hdr = rhdr; +struct ib_header *hdr = rhdr;struct qib_devdata *dd = ppd->dd;struct rvt_dev_info *rdi = &dd->verbs_dev.rdi; -struct qib_other_headers *ohdr; +struct ib_other_headers *ohdr;struct rvt_qp *qp;u32 qp_num;int lnh;
@@ -841,7 +841,7 @@ static void sdma_complete(struct qib_sdma_txreq *cookie, int status)if (tx->wqe)qib_send_complete (qp, tx->wqe,IB_WC_SUCCESS );else if (qp->ibqp.qp_type ==IB_QPT_RC ) { -struct qib_ib_header *hdr; +struct ib_header *hdr;if (tx->txreq.flags &QIB_SDMA_TXREQ_F_FREEBUF ) hdr = &tx->align_buf->hdr;
@@ -889,7 +889,7 @@ static int wait_kmem(struct qib_ibdev *dev, struct rvt_qp *qp)return ret; } -static int qib_verbs_send_dma (struct rvt_qp *qp,struct qib_ib_header *hdr, +static int qib_verbs_send_dma (struct rvt_qp *qp,struct ib_header *hdr,u32 hdrwords,struct rvt_sge_state *ss,u32 len,u32 plen,u32 dwords) {
@@ -1025,7 +1025,7 @@ static int no_bufs_available(struct rvt_qp *qp)return ret; } -static int qib_verbs_send_pio (struct rvt_qp *qp,struct qib_ib_header *ibhdr, +static int qib_verbs_send_pio (struct rvt_qp *qp,struct ib_header *ibhdr,u32 hdrwords,struct rvt_sge_state *ss,u32 len,u32 plen,u32 dwords) {
@@ -1133,7 +1133,7 @@ done:* Return zero if packet is sent or queued OK. * Return non-zero and clear qp->s_flags RVT_S_BUSY otherwise. */ -int qib_verbs_send (struct rvt_qp *qp,struct qib_ib_header *hdr, +int qib_verbs_send (struct rvt_qp *qp,struct ib_header *hdr,u32 hdrwords,struct rvt_sge_state *ss,u32 len) {struct qib_devdata *dd =dd_from_ibdev (qp->ibqp.device);
drivers/infiniband/hw/qib/qib_verbs.h +10 -84
165 lines not on this page · raw message
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
PATCH 5/13 IB/hfi1: Move serdes tune inside link start function
In reply to a message not in the archiveFrom: Dean Luick <[email protected]>
All calls to tune_serdes and start_link are paired. Move tune_serdes inside start_link.
--- drivers/infiniband/hw/hfi1/chip.c | 22 ++++++++-------------- drivers/infiniband/hw/hfi1/mad.c | 1 - 2 files changed, 8 insertions(+), 15 deletions(-)
drivers/infiniband/hw/hfi1/chip.c +8 -14
@@ -6825,7 +6825,6 @@ void handle_link_up(struct work_struct *work)set_link_down_reason (ppd,OPA_LINKDOWN_REASON_SPEED_POLICY ,0 ,OPA_LINKDOWN_REASON_SPEED_POLICY );set_link_state (ppd,HLS_DN_OFFLINE ); -tune_serdes (ppd);start_link (ppd); } }
@@ -6998,12 +6997,10 @@ void handle_link_down(struct work_struct *work)* If there is no cable attached, turn the DC off. Otherwise, * start the link bring up. */ -if (ppd->port_type ==PORT_TYPE_QSFP && !qsfp_mod_present (ppd)) { +if (ppd->port_type ==PORT_TYPE_QSFP && !qsfp_mod_present (ppd))dc_shutdown (ppd->dd); - }else { -tune_serdes (ppd); +else start_link (ppd); - } }void handle_link_bounce (struct work_struct *work)
@@ -7016,7 +7013,6 @@ void handle_link_bounce(struct work_struct *work)*/ if (ppd->host_link_state &HLS_UP ) {set_link_state (ppd,HLS_DN_OFFLINE ); -tune_serdes (ppd);start_link (ppd); }else { dd_dev_info(ppd->dd, "%s: link not up (%s), nothing to do\n",
@@ -7531,7 +7527,6 @@ done:set_link_down_reason (ppd,OPA_LINKDOWN_REASON_WIDTH_POLICY ,0 ,OPA_LINKDOWN_REASON_WIDTH_POLICY );set_link_state (ppd,HLS_DN_OFFLINE ); -tune_serdes (ppd);start_link (ppd); } }
@@ -9161,6 +9156,12 @@ set_local_link_attributes_fail:*/ int start_link (struct hfi1_pportdata *ppd) { +/* +* Tune the SerDes to a ballpark setting for optimal signal and bit +* error rate. Needs to be done before starting the link. +*/ +tune_serdes (ppd); +if (!ppd->link_enabled) { dd_dev_info(ppd->dd, "%s: stopping link start because link is disabled\n",
@@ -9401,8 +9402,6 @@ void qsfp_event(struct work_struct *work)*/ set_qsfp_int_n (ppd,1 ); -tune_serdes (ppd); -start_link (ppd); }
@@ -9544,11 +9543,6 @@ static void try_start_link(struct hfi1_pportdata *ppd) } ppd->qsfp_retry_count =0 ; -/* -* Tune the SerDes to a ballpark setting for optimal signal and bit -* error rate. Needs to be done before starting the link. -*/ -tune_serdes (ppd);start_link (ppd); }
drivers/infiniband/hw/hfi1/mad.c +0 -1
@@ -1013,7 +1013,6 @@ static int set_port_states(struct hfi1_pportdata *ppd, struct opa_smp *smp,* offline. */ set_link_state (ppd,HLS_DN_OFFLINE ); -tune_serdes (ppd);start_link (ppd); }else {set_link_state (ppd, link_state);
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
PATCH 6/13 IB/hfi1: Fix the count of user packets submitted to an SDMA engine
In reply to a message not in the archiveFrom: Harish Chegondi <[email protected]>
Each user SDMA request coming into the driver may contain multiple packets. Each user packet may use multiple SDMA descriptors to fill the send buffer. The field seqsubmitted in struct user_sdma_request counts the number of user packets submitted to an SDMA engine. Sometimes, the intermediate count may not be updated properly. However, once all the packets' descriptors are successfully submitted to the SDMA engine, the final count is updated correctly. But, if only some of the packets are submitted to the engine due to an error, the intermediate count doesn't reflect the partial number of packets submitted to the SDMA engine. This can cause a hang later in the code as the count of packets submitted to the SDMA engine doesn't match the the count of packets processed by the SDMA engine.
--- drivers/infiniband/hw/hfi1/sdma.c | 24 +++++++++++++++--------- drivers/infiniband/hw/hfi1/sdma.h | 3 ++- drivers/infiniband/hw/hfi1/user_sdma.c | 31 +++++++++++++------------------ 3 files changed, 30 insertions(+), 28 deletions(-)
drivers/infiniband/hw/hfi1/sdma.c +15 -9
@@ -2086,6 +2086,11 @@ nodesc:* @sde: sdma engine to use * @wait: wait structure to use when full (may be NULL) * @tx_list: list of sdma_txreqs to submit +* @count: pointer to a u32 which, after return will contain the total number of +* sdma_txreqs removed from the tx_list. This will include sdma_txreqs +* whose SDMA descriptors are submitted to the ring and the sdma_txreqs +* which are added to SDMA engine flush list if the SDMA engine state is +* not running. * * The call submits the list into the ring. *
@@ -2100,18 +2105,18 @@ nodesc:* side locking. * * Return: -* > 0 - Success (value is number of sdma_txreq's submitted), +* 0 - Success, * -EINVAL - sdma_txreq incomplete, -EBUSY - no space in ring (wait == NULL) * -EIOCBQUEUED - tx queued to iowait, -ECOMM bad sdma state */ int sdma_send_txlist (struct sdma_engine *sde,struct iowait *wait, -struct list_head *tx_list) +struct list_head *tx_list,u32 *count_out) {struct sdma_txreq *tx, *tx_next;int ret =0 ;unsigned long flags;u16 tail =INVALID_TAIL ; -int count =0 ; +u32 submit_count =0 , flush_count =0 , total_count;spin_lock_irqsave (&sde->tail_lock, flags); retry:
@@ -2127,33 +2132,34 @@ retry: }list_del_init (&tx->list); tail =submit_tx (sde, tx); - count++; + submit_count++;if (tail !=INVALID_TAIL && - (count &SDMA_TAIL_UPDATE_THRESH ) ==0 ) { + (submit_count &SDMA_TAIL_UPDATE_THRESH ) ==0 ) {sdma_update_tail (sde, tail); tail =INVALID_TAIL ; } } update_tail: + total_count = submit_count + flush_count;if (wait) -iowait_sdma_add (wait, count); +iowait_sdma_add (wait, total_count);if (tail !=INVALID_TAIL )sdma_update_tail (sde, tail);spin_unlock_irqrestore (&sde->tail_lock, flags); -return ret ==0 ? count : ret; + *count_out = total_count; +return ret; unlock_noconn:spin_lock (&sde->flushlist_lock);list_for_each_entry_safe (tx, tx_next, tx_list, list) { tx->wait = wait;list_del_init (&tx->list); -if (wait) -iowait_sdma_inc (wait); tx->next_descq_idx =0 ;#ifdef CONFIG_HFI1_DEBUG_SDMA_ORDER tx->sn = sde->tail_sn++;trace_hfi1_sdma_in_sn (sde, tx->sn);#endif list_add_tail (&tx->list, &sde->flushlist); + flush_count++;if (wait) { wait->tx_count++; wait->count += tx->num_desc;
drivers/infiniband/hw/hfi1/sdma.h +2 -1
@@ -847,7 +847,8 @@ int sdma_send_txreq(struct sdma_engine *sde, struct sdma_txreq *tx);int sdma_send_txlist (struct sdma_engine *sde,struct iowait *wait, -struct list_head *tx_list); +struct list_head *tx_list, +u32 *count);int sdma_ahg_alloc (struct sdma_engine *sde);void sdma_ahg_free (struct sdma_engine *sde,int ahg_index);
drivers/infiniband/hw/hfi1/user_sdma.c +13 -18
@@ -894,7 +894,7 @@ static inline u32 get_lrh_len(struct hfi1_pkt_header hdr, u32 len)static int user_sdma_send_pkts (struct user_sdma_request *req,unsigned maxpkts) { -int ret =0 ; +int ret =0 , count;unsigned npkts =0 ;struct user_sdma_txreq *tx =NULL ;struct hfi1_user_sdma_pkt_q *pq =NULL ;
@@ -1090,23 +1090,18 @@ static int user_sdma_send_pkts(struct user_sdma_request *req, unsigned maxpkts) npkts++; } dosend: - ret =sdma_send_txlist (req->sde, &pq->busy, &req->txps); -if (list_empty (&req->txps)) { - req->seqsubmitted = req->seqnum; -if (req->seqnum == req->info.npkts) { -set_bit (SDMA_REQ_SEND_DONE , &req->flags); -/* -* The txreq has already been submitted to the HW queue -* so we can free the AHG entry now. Corruption will not -* happen due to the sequential manner in which -* descriptors are processed. -*/ -if (test_bit (SDMA_REQ_HAVE_AHG , &req->flags)) -sdma_ahg_free (req->sde, req->ahg_idx); - } - }else if (ret >0 ) { - req->seqsubmitted += ret; - ret =0 ; + ret =sdma_send_txlist (req->sde, &pq->busy, &req->txps, &count); + req->seqsubmitted += count; +if (req->seqsubmitted == req->info.npkts) { +set_bit (SDMA_REQ_SEND_DONE , &req->flags); +/* +* The txreq has already been submitted to the HW queue +* so we can free the AHG entry now. Corruption will not +* happen due to the sequential manner in which +* descriptors are processed. +*/ +if (test_bit (SDMA_REQ_HAVE_AHG , &req->flags)) +sdma_ahg_free (req->sde, req->ahg_idx); }return ret;
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
PATCH 7/13 IB/hfi1: Fix user-space buffers mapping with IOMMU enabled
In reply to a message not in the archiveFrom: Tymoteusz Kielan <[email protected]>
The dma_XXX API functions return bus addresses which are physical addresses when IOMMU is disabled. Buffer mapping to user-space is done via remap_pfn_range() with PFN based on bus address instead of physical. This results in wrong pages being mapped to user-space when IOMMU is enabled.
--- drivers/infiniband/hw/hfi1/chip.c | 12 ++++---- drivers/infiniband/hw/hfi1/file_ops.c | 41 ++++++++++++++++++++-------- drivers/infiniband/hw/hfi1/hfi.h | 10 +++---- drivers/infiniband/hw/hfi1/init.c | 44 +++++++++++++++--------------- drivers/infiniband/hw/hfi1/pio.c | 20 +++++++------- drivers/infiniband/hw/hfi1/pio.h | 2 + drivers/infiniband/hw/hfi1/trace_ctxts.h | 13 ++++----- 7 files changed, 79 insertions(+), 63 deletions(-)
drivers/infiniband/hw/hfi1/chip.c +6 -6
@@ -11553,10 +11553,10 @@ void hfi1_rcvctrl(struct hfi1_devdata *dd, unsigned int op, int ctxt) !(rcvctrl & RCV_CTXT_CTRL_ENABLE_SMASK)) {/* reset the tail and hdr addresses, and sequence count */ write_kctxt_csr (dd, ctxt,RCV_HDR_ADDR , - rcd->rcvhdrq_phys); + rcd->rcvhdrq_dma);if (HFI1_CAP_KGET_MASK (rcd->flags,DMA_RTAIL ))write_kctxt_csr (dd, ctxt,RCV_HDR_TAIL_ADDR , - rcd->rcvhdrqtailaddr_phys); + rcd->rcvhdrqtailaddr_dma); rcd->seq_cnt =1 ;/* reset the cached receive header queue head value */
@@ -11621,9 +11621,9 @@ void hfi1_rcvctrl(struct hfi1_devdata *dd, unsigned int op, int ctxt)* update with a dummy tail address and then disable * receive context. */ -if (dd->rcvhdrtail_dummy_physaddr) { +if (dd->rcvhdrtail_dummy_dma) {write_kctxt_csr (dd, ctxt,RCV_HDR_TAIL_ADDR , - dd->rcvhdrtail_dummy_physaddr); + dd->rcvhdrtail_dummy_dma);/* Enabling RcvCtxtCtrl.TailUpd is intentional. */ rcvctrl |=RCV_CTXT_CTRL_TAIL_UPD_SMASK ; }
@@ -11634,7 +11634,7 @@ void hfi1_rcvctrl(struct hfi1_devdata *dd, unsigned int op, int ctxt) rcvctrl |=RCV_CTXT_CTRL_INTR_AVAIL_SMASK ;if (op &HFI1_RCVCTRL_INTRAVAIL_DIS ) rcvctrl &= ~RCV_CTXT_CTRL_INTR_AVAIL_SMASK ; -if (op &HFI1_RCVCTRL_TAILUPD_ENB && rcd->rcvhdrqtailaddr_phys) +if (op &HFI1_RCVCTRL_TAILUPD_ENB && rcd->rcvhdrqtailaddr_dma) rcvctrl |=RCV_CTXT_CTRL_TAIL_UPD_SMASK ;if (op &HFI1_RCVCTRL_TAILUPD_DIS ) {/* See comment on RcvCtxtCtrl.TailUpd above */
@@ -11706,7 +11706,7 @@ void hfi1_rcvctrl(struct hfi1_devdata *dd, unsigned int op, int ctxt)* so it doesn't contain an address that is invalid. */ write_kctxt_csr (dd, ctxt,RCV_HDR_TAIL_ADDR , - dd->rcvhdrtail_dummy_physaddr); + dd->rcvhdrtail_dummy_dma); }u32 hfi1_read_cntrs (struct hfi1_devdata *dd,char **namep,u64 **cntrp)
drivers/infiniband/hw/hfi1/file_ops.c +29 -12
98 lines not on this page · raw message
drivers/infiniband/hw/hfi1/hfi.h +5 -5
@@ -172,12 +172,12 @@ struct ctxt_eager_bufs {u32 threshold;/* head update threshold */ struct eager_buffer {void *addr; -dma_addr_t phys; +dma_addr_t dma;ssize_t len; } *buffers;struct {void *addr; -dma_addr_t phys; +dma_addr_t dma; } *rcvtids; };
@@ -208,8 +208,8 @@ struct hfi1_ctxtdata {/* size of each of the rcvhdrq entries */ u16 rcvhdrqentsize;/* mmap of hdrq, must fit in 44 bits */ -dma_addr_t rcvhdrq_phys; -dma_addr_t rcvhdrqtailaddr_phys; +dma_addr_t rcvhdrq_dma; +dma_addr_t rcvhdrqtailaddr_dma;struct ctxt_eager_bufs egrbufs;/* this receive context's assigned PIO ACK send context */ struct send_context *sc;
@@ -1165,7 +1165,7 @@ struct hfi1_devdata {/* receive context tail dummy address */ __le64 *rcvhdrtail_dummy_kvaddr; -dma_addr_t rcvhdrtail_dummy_physaddr; +dma_addr_t rcvhdrtail_dummy_dma;bool eprom_available;/* true if EPROM is available for this device */ bool aspm_supported;/* Does HW support ASPM */
drivers/infiniband/hw/hfi1/init.c +22 -22
143 lines not on this page · raw message
drivers/infiniband/hw/hfi1/pio.c +10 -10
@@ -551,11 +551,11 @@ static inline u32 group_size(u32 group) }/* -* Obtain the credit return addresses, kernel virtual and physical, for the +* Obtain the credit return addresses, kernel virtual and bus, for the * given sc. * * To understand this routine: -* o va and pa are arrays of struct credit_return. One for each physical +* o va and dma are arrays of struct credit_return. One for each physical * send context, per NUMA. * o Each send context always looks in its relative location in a struct * credit_return for its credit return.
@@ -563,14 +563,14 @@ static inline u32 group_size(u32 group)* with the same value. Use the address of the first send context in the * group. */ -static void cr_group_addresses (struct send_context *sc,dma_addr_t *pa) +static void cr_group_addresses (struct send_context *sc,dma_addr_t *dma) {u32 gc =group_context (sc->hw_context, sc->group);u32 index = sc->hw_context &0x7 ; sc->hw_free = &sc->dd->cr_base[sc->node].va[gc].cr[index]; - *pa = (unsigned long ) - &((struct credit_return *)sc->dd->cr_base[sc->node].pa)[gc]; + *dma = (unsigned long ) + &((struct credit_return *)sc->dd->cr_base[sc->node].dma)[gc]; }/*
@@ -710,7 +710,7 @@ struct send_context *sc_alloc(struct hfi1_devdata *dd, int type, {struct send_context_info *sci;struct send_context *sc =NULL ; -dma_addr_t pa; +dma_addr_t dma;unsigned long flags;u64 reg;u32 thresh;
@@ -763,7 +763,7 @@ struct send_context *sc_alloc(struct hfi1_devdata *dd, int type, sc->sw_index = sw_index; sc->hw_context = hw_context; -cr_group_addresses (sc, &pa); +cr_group_addresses (sc, &dma); sc->credits = sci->credits;/* PIO Send Memory Address details */
@@ -805,7 +805,7 @@ struct send_context *sc_alloc(struct hfi1_devdata *dd, int type, ((u64)opval << SC(CHECK_OPCODE_VALUE_SHIFT)));/* set up credit return */ - reg = pa &SC (CREDIT_RETURN_ADDR_ADDRESS_SMASK ); + reg = dma &SC (CREDIT_RETURN_ADDR_ADDRESS_SMASK );write_kctxt_csr (dd, hw_context,SC (CREDIT_RETURN_ADDR ), reg);/*
@@ -2064,7 +2064,7 @@ int init_credit_return(struct hfi1_devdata *dd) dd->cr_base[i].va =dma_zalloc_coherent ( &dd->pcidev->dev, bytes, - &dd->cr_base[i].pa, + &dd->cr_base[i].dma,GFP_KERNEL );if (!dd->cr_base[i].va) {set_dev_node (&dd->pcidev->dev, dd->node);
@@ -2097,7 +2097,7 @@ void free_credit_return(struct hfi1_devdata *dd)TXE_NUM_CONTEXTS *sizeof (struct credit_return ), dd->cr_base[i].va, - dd->cr_base[i].pa); + dd->cr_base[i].dma); } }kfree (dd->cr_base);
drivers/infiniband/hw/hfi1/pio.h +1 -1
@@ -154,7 +154,7 @@ struct credit_return {/* NUMA indexed credit return array */ struct credit_return_base {struct credit_return *va; -dma_addr_t pa; +dma_addr_t dma; };/* send context configuration sizes (one per type) */
drivers/infiniband/hw/hfi1/trace_ctxts.h +6 -7
@@ -67,9 +67,9 @@ TRACE_EVENT(hfi1_uctxtdata, __field(u64, hw_free) __field(void __iomem *, piobase) __field(u16, rcvhdrq_cnt) - __field(u64, rcvhdrq_phys) + __field(u64, rcvhdrq_dma) __field(u32, eager_cnt) - __field(u64, rcvegr_phys) + __field(u64, rcvegr_dma) ), TP_fast_assign(DD_DEV_ASSIGN(dd); __entry->ctxt = uctxt->ctxt;
@@ -77,10 +77,9 @@ TRACE_EVENT(hfi1_uctxtdata, __entry->hw_free =le64_to_cpu (*uctxt->sc->hw_free); __entry->piobase = uctxt->sc->base_addr; __entry->rcvhdrq_cnt = uctxt->rcvhdrq_cnt; - __entry->rcvhdrq_phys = uctxt->rcvhdrq_phys; + __entry->rcvhdrq_dma = uctxt->rcvhdrq_dma; __entry->eager_cnt = uctxt->egrbufs.alloced; - __entry->rcvegr_phys = - uctxt->egrbufs.rcvtids[0 ].phys; + __entry->rcvegr_dma = uctxt->egrbufs.rcvtids[0 ].dma; ), TP_printk("[%s] ctxt %u " UCTXT_FMT, __get_str(dev),
@@ -89,9 +88,9 @@ TRACE_EVENT(hfi1_uctxtdata, __entry->hw_free, __entry->piobase, __entry->rcvhdrq_cnt, - __entry->rcvhdrq_phys, + __entry->rcvhdrq_dma, __entry->eager_cnt, - __entry->rcvegr_phys + __entry->rcvegr_dma ) );
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
PATCH 8/13 IB/hfi1: Fix locking scheme for affinity settings
In reply to a message not in the archiveFrom: Tadeusz Struk <[email protected]>
Existing locking scheme in affinity.c file using the &node_affinity.lock spinlock is not very elegant. We acquire the lock to get hfi1_affinity_node entry, unlock, and then use the entry without the lock held. With more functions being added, which access and modify the entries, this can lead to race conditions. This patch makes this locking scheme more consistent. It changes the spinlock to mutex. Since all the code is executed in a user process context there is no need for a spinlock. This also allows to keep the lock not only while we look up for the node affinity entry, but over the whole section where the entry is being used.
--- drivers/infiniband/hw/hfi1/affinity.c | 91 ++++++++++++++++++--------------- drivers/infiniband/hw/hfi1/affinity.h | 3 - 2 files changed, 51 insertions(+), 43 deletions(-)
drivers/infiniband/hw/hfi1/affinity.c +50 -41
@@ -55,7 +55,7 @@struct hfi1_affinity_node_list node_affinity = { .list =LIST_HEAD_INIT (node_affinity.list), - .lock =__SPIN_LOCK_UNLOCKED (&node_affinity.lock), + .lock =__MUTEX_INITIALIZER (node_affinity.lock) };/* Name of IRQ types, indexed by enum irq_type */
@@ -159,14 +159,14 @@ void node_affinity_destroy(void)struct list_head *pos, *q;struct hfi1_affinity_node *entry; -spin_lock (&node_affinity.lock); +mutex_lock (&node_affinity.lock);list_for_each_safe (pos, q, &node_affinity.list) { entry = list_entry(pos, struct hfi1_affinity_node, list);list_del (pos);kfree (entry); } -spin_unlock (&node_affinity.lock); +mutex_unlock (&node_affinity.lock);kfree (hfi1_per_node_cntr); }
@@ -233,9 +233,8 @@ int hfi1_dev_affinity_init(struct hfi1_devdata *dd)if (cpumask_first (local_mask) >= nr_cpu_ids) local_mask =topology_core_cpumask (0 ); -spin_lock (&node_affinity.lock); +mutex_lock (&node_affinity.lock); entry =node_affinity_lookup (dd->node); -spin_unlock (&node_affinity.lock);/* * If this is the first time this NUMA node's affinity is used,
@@ -246,6 +245,7 @@ int hfi1_dev_affinity_init(struct hfi1_devdata *dd)if (!entry) {dd_dev_err (dd,"Unable to allocate global affinity node\n" ); +mutex_unlock (&node_affinity.lock);return -ENOMEM ; }init_cpu_mask_set (&entry->def_intr);
@@ -302,15 +302,18 @@ int hfi1_dev_affinity_init(struct hfi1_devdata *dd) &entry->general_intr_mask); } -spin_lock (&node_affinity.lock);node_affinity_add_tail (entry); -spin_unlock (&node_affinity.lock); } - +mutex_unlock (&node_affinity.lock);return 0 ; } -int hfi1_get_irq_affinity (struct hfi1_devdata *dd,struct hfi1_msix_entry *msix) +/* +* Function sets the irq affinity for msix. +* It *must* be called with node_affinity.lock held. +*/ +static int get_irq_affinity (struct hfi1_devdata *dd, +struct hfi1_msix_entry *msix) {int ret;cpumask_var_t diff;
@@ -328,9 +331,7 @@ int hfi1_get_irq_affinity(struct hfi1_devdata *dd, struct hfi1_msix_entry *msix)if (!ret)return -ENOMEM ; -spin_lock (&node_affinity.lock); entry =node_affinity_lookup (dd->node); -spin_unlock (&node_affinity.lock);switch (msix->type) {case IRQ_SDMA :
@@ -360,7 +361,6 @@ int hfi1_get_irq_affinity(struct hfi1_devdata *dd, struct hfi1_msix_entry *msix)* finds its CPU here. */ if (cpu ==-1 && set) { -spin_lock (&node_affinity.lock);if (cpumask_equal (&set->mask, &set->used)) {/* * We've used up all the CPUs, bump up the generation
@@ -372,7 +372,6 @@ int hfi1_get_irq_affinity(struct hfi1_devdata *dd, struct hfi1_msix_entry *msix)cpumask_andnot (diff, &set->mask, &set->used); cpu =cpumask_first (diff);cpumask_set_cpu (cpu, &set->used); -spin_unlock (&node_affinity.lock); }switch (msix->type) {
@@ -395,6 +394,16 @@ int hfi1_get_irq_affinity(struct hfi1_devdata *dd, struct hfi1_msix_entry *msix)return 0 ; } +int hfi1_get_irq_affinity (struct hfi1_devdata *dd,struct hfi1_msix_entry *msix) +{ +int ret; + +mutex_lock (&node_affinity.lock); + ret =get_irq_affinity (dd, msix); +mutex_unlock (&node_affinity.lock); +return ret; +} +void hfi1_put_irq_affinity (struct hfi1_devdata *dd,struct hfi1_msix_entry *msix) {
@@ -402,9 +411,8 @@ void hfi1_put_irq_affinity(struct hfi1_devdata *dd,struct hfi1_ctxtdata *rcd;struct hfi1_affinity_node *entry; -spin_lock (&node_affinity.lock); +mutex_lock (&node_affinity.lock); entry =node_affinity_lookup (dd->node); -spin_unlock (&node_affinity.lock);switch (msix->type) {case IRQ_SDMA :
@@ -420,21 +428,21 @@ void hfi1_put_irq_affinity(struct hfi1_devdata *dd, set = &entry->rcv_intr;break ;default : +mutex_unlock (&node_affinity.lock);return ; }if (set) { -spin_lock (&node_affinity.lock);cpumask_andnot (&set->used, &set->used, &msix->mask);if (cpumask_empty (&set->used) && set->gen) { set->gen--;cpumask_copy (&set->used, &set->mask); } -spin_unlock (&node_affinity.lock); }irq_set_affinity_hint (msix->msix.vector,NULL );cpumask_clear (&msix->mask); +mutex_unlock (&node_affinity.lock); }/* This should be called with node_affinity.lock held */
@@ -535,7 +543,7 @@ int hfi1_get_proc_affinity(int node)if (!ret) goto free_available_mask; -spin_lock (&affinity->lock); +mutex_lock (&affinity->lock);/* * If we've used all available HW threads, clear the mask and start * overloading.
@@ -643,7 +651,8 @@ int hfi1_get_proc_affinity(int node) cpu =-1 ; else cpumask_set_cpu(cpu, &set->used); -spin_unlock (&affinity->lock); + +mutex_unlock (&affinity->lock);hfi1_cdbg (PROC ,"Process assigned to CPU %d" , cpu);free_cpumask_var (intrs_mask);
@@ -664,19 +673,17 @@ void hfi1_put_proc_affinity(int cpu)if (cpu <0 )return ; -spin_lock (&affinity->lock); + +mutex_lock (&affinity->lock);cpumask_clear_cpu (cpu, &set->used);hfi1_cdbg (PROC ,"Returning CPU %d for future process assignment" , cpu);if (cpumask_empty (&set->used) && set->gen) { set->gen--;cpumask_copy (&set->used, &set->mask); } -spin_unlock (&affinity->lock); +mutex_unlock (&affinity->lock); } -/* Prevents concurrent reads and writes of the sdma_affinity attrib */ -static DEFINE_MUTEX (sdma_affinity_mutex); -int hfi1_set_sdma_affinity (struct hfi1_devdata *dd,const char *buf,size_t count) {
@@ -684,16 +691,19 @@ int hfi1_set_sdma_affinity(struct hfi1_devdata *dd, const char *buf,cpumask_var_t mask;int ret, i; -spin_lock (&node_affinity.lock); +mutex_lock (&node_affinity.lock); entry =node_affinity_lookup (dd->node); -spin_unlock (&node_affinity.lock); -if (!entry) -return -EINVAL ; +if (!entry) { + ret = -EINVAL ; + goto unlock; + } ret =zalloc_cpumask_var (&mask,GFP_KERNEL ); -if (!ret) -return -ENOMEM ; +if (!ret) { + ret = -ENOMEM ; + goto unlock; + } ret =cpulist_parse (buf, mask);if (ret)
@@ -705,13 +715,11 @@ int hfi1_set_sdma_affinity(struct hfi1_devdata *dd, const char *buf, goto out; } -mutex_lock (&sdma_affinity_mutex);/* reset the SDMA interrupt affinity details */ init_cpu_mask_set (&entry->def_intr);cpumask_copy (&entry->def_intr.mask, mask); -/* -* Reassign the affinity for each SDMA interrupt. -*/ + +/* Reassign the affinity for each SDMA interrupt. */ for (i =0 ; i < dd->num_msix_entries; i++) {struct hfi1_msix_entry *msix;
@@ -719,14 +727,15 @@ int hfi1_set_sdma_affinity(struct hfi1_devdata *dd, const char *buf,if (msix->type !=IRQ_SDMA )continue ; - ret =hfi1_get_irq_affinity (dd, msix); + ret =get_irq_affinity (dd, msix);if (ret)break ; } -mutex_unlock (&sdma_affinity_mutex); out:free_cpumask_var (mask); +unlock: +mutex_unlock (&node_affinity.lock);return ret ? ret :strnlen (buf,PAGE_SIZE ); }
@@ -734,15 +743,15 @@ int hfi1_get_sdma_affinity(struct hfi1_devdata *dd, char *buf) {struct hfi1_affinity_node *entry; -spin_lock (&node_affinity.lock); +mutex_lock (&node_affinity.lock); entry =node_affinity_lookup (dd->node); -spin_unlock (&node_affinity.lock); -if (!entry) +if (!entry) { +mutex_unlock (&node_affinity.lock);return -EINVAL ; + } -mutex_lock (&sdma_affinity_mutex);cpumap_print_to_pagebuf (true, buf, &entry->def_intr.mask); -mutex_unlock (&sdma_affinity_mutex); +mutex_unlock (&node_affinity.lock);return strnlen (buf,PAGE_SIZE ); }
drivers/infiniband/hw/hfi1/affinity.h +1 -2
@@ -121,8 +121,7 @@ struct hfi1_affinity_node_list {int num_core_siblings;int num_online_nodes;int num_online_cpus; -/* protect affinity node list */ -spinlock_t lock; +struct mutex lock;/* protects affinity nodes */ };int node_affinity_init (void );
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
PATCH 9/13 IB/rdmavt: Correct sparse annotation
In reply to a message not in the archiveFrom: Mike Marciniszyn <[email protected]>
The __must_hold() is sufficent to correct the sparse context imbalance inside a function.
Per Documentation/sparse.txt:__must_hold - The specified lock is held on function entry and exit.
--- drivers/infiniband/sw/rdmavt/qp.c | 9 +++------ 1 files changed, 3 insertions(+), 6 deletions(-)
drivers/infiniband/sw/rdmavt/qp.c +3 -6
@@ -500,12 +500,9 @@ static void rvt_remove_qp(struct rvt_dev_info *rdi, struct rvt_qp *qp)*/ static void rvt_reset_qp (struct rvt_dev_info *rdi,struct rvt_qp *qp,enum ib_qp_type type) - __releases(&qp->s_lock) - __releases(&qp->s_hlock) - __releases(&qp->r_lock) - __acquires(&qp->r_lock) - __acquires(&qp->s_hlock) - __acquires(&qp->s_lock) + __must_hold(&qp->r_lock) + __must_hold(&qp->s_hlock) + __must_hold(&qp->s_lock) {if (qp->state !=IB_QPS_RESET ) { qp->state =IB_QPS_RESET ;
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
PATCH 10/13 IB/hfi1: Move iowait_init() to priv allocate
In reply to a message not in the archiveFrom: Mike Marciniszyn <[email protected]>
The call is misplaced in the reset calldown function and causes issues with lockdep assertions that are to be added.
--- drivers/infiniband/hw/hfi1/qp.c | 14 +++++++------- 1 files changed, 7 insertions(+), 7 deletions(-)
drivers/infiniband/hw/hfi1/qp.c +7 -7
@@ -806,6 +806,13 @@ void *qp_priv_alloc(struct rvt_dev_info *rdi, struct rvt_qp *qp,kfree (priv);return ERR_PTR (-ENOMEM ); } +iowait_init ( + &priv->s_iowait, +1 , + _hfi1_do_send, + iowait_sleep, + iowait_wakeup, + iowait_sdma_drained);setup_timer (&priv->s_rnr_timer, hfi1_rc_rnr_retry, (unsigned long )qp); qp->s_timer.function = hfi1_rc_timeout;return priv;
@@ -871,13 +878,6 @@ void notify_qp_reset(struct rvt_qp *qp) {struct hfi1_qp_priv *priv = qp->priv; -iowait_init ( - &priv->s_iowait, -1 , - _hfi1_do_send, - iowait_sleep, - iowait_wakeup, - iowait_sdma_drained); priv->r_adefered =0 ;clear_ahg (qp); }
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
PATCH 11/13 IB/rdmavt: Move reset calldown to reset path
In reply to a message not in the archiveFrom: Mike Marciniszyn <[email protected]>
The reset calldown is misplaced.
It should only be called in the code that actually transitions the QP to reset.
--- drivers/infiniband/sw/rdmavt/qp.c | 11 +++++------ 1 files changed, 5 insertions(+), 6 deletions(-)
drivers/infiniband/sw/rdmavt/qp.c +5 -6
@@ -530,14 +530,13 @@ static void rvt_reset_qp(struct rvt_dev_info *rdi, struct rvt_qp *qp,spin_lock (&qp->s_lock);rvt_clear_mr_refs (qp,1 ); +/* +* Let the driver do any tear down it needs to for a qp +* that has been reset +*/ + rdi->driver_f.notify_qp_reset (qp); } -/* -* Let the driver do any tear down it needs to for a qp -* that has been reset -*/ - rdi->driver_f.notify_qp_reset (qp); - qp->remote_qpn =0 ; qp->qkey =0 ; qp->qp_access_flags =0 ;
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
PATCH 12/13 IB/rdmavt: Add qp init function
In reply to a message not in the archiveFrom: Mike Marciniszyn <[email protected]>
Add an rvt_qp_init() to initialize specific
common fields as the qp is created or reset.
The routine is shared by the rvt_reset_qp() and
the rvt_create_qp().
The intent is that lock dep assertions will only
appear in the rvt_reset_qp().
--- drivers/infiniband/sw/rdmavt/qp.c | 100 +++++++++++++++++++++---------------- 1 files changed, 58 insertions(+), 42 deletions(-)
drivers/infiniband/sw/rdmavt/qp.c +58 -42
@@ -493,50 +493,18 @@ static void rvt_remove_qp(struct rvt_dev_info *rdi, struct rvt_qp *qp) }/** -* reset_qp - initialize the QP state to the reset state -* @qp: the QP to reset +* rvt_init_qp - initialize the QP state to the reset state +* @qp: the QP to init or reinit * @type: the QP type -* r and s lock are required to be held by the caller +* +* This function is called from both rvt_create_qp() and +* rvt_reset_qp(). The difference is that the reset +* patch the necessary locks to protect against concurent +* access. */ -static void rvt_reset_qp (struct rvt_dev_info *rdi,struct rvt_qp *qp, -enum ib_qp_type type) - __must_hold(&qp->r_lock) - __must_hold(&qp->s_hlock) - __must_hold(&qp->s_lock) +static void rvt_init_qp (struct rvt_dev_info *rdi,struct rvt_qp *qp, +enum ib_qp_type type) { -if (qp->state !=IB_QPS_RESET ) { - qp->state =IB_QPS_RESET ; - -/* Let drivers flush their waitlist */ - rdi->driver_f.flush_qp_waiters (qp); - qp->s_flags &= ~(RVT_S_TIMER |RVT_S_ANY_WAIT ); -spin_unlock (&qp->s_lock); -spin_unlock (&qp->s_hlock); -spin_unlock_irq (&qp->r_lock); - -/* Stop the send queue and the retry timer */ - rdi->driver_f.stop_send_queue (qp); - -/* Wait for things to stop */ - rdi->driver_f.quiesce_qp (qp); - -/* take qp out the hash and wait for it to be unused */ -rvt_remove_qp (rdi, qp); -wait_event (qp->wait, !atomic_read (&qp->refcount)); - -/* grab the lock b/c it was locked at call time */ -spin_lock_irq (&qp->r_lock); -spin_lock (&qp->s_hlock); -spin_lock (&qp->s_lock); - -rvt_clear_mr_refs (qp,1 ); -/* -* Let the driver do any tear down it needs to for a qp -* that has been reset -*/ - rdi->driver_f.notify_qp_reset (qp); - } - qp->remote_qpn =0 ; qp->qkey =0 ; qp->qp_access_flags =0 ;
@@ -582,6 +550,54 @@ static void rvt_reset_qp(struct rvt_dev_info *rdi, struct rvt_qp *qp, }/** +* rvt_reset_qp - initialize the QP state to the reset state +* @qp: the QP to reset +* @type: the QP type +* +* r_lock, s_hlock, and s_lock are required to be held by the caller +*/ +static void rvt_reset_qp (struct rvt_dev_info *rdi,struct rvt_qp *qp, +enum ib_qp_type type) + __must_hold(&qp->s_lock) + __must_hold(&qp->s_hlock) + __must_hold(&qp->r_lock) +{ +if (qp->state !=IB_QPS_RESET ) { + qp->state =IB_QPS_RESET ; + +/* Let drivers flush their waitlist */ + rdi->driver_f.flush_qp_waiters (qp); + qp->s_flags &= ~(RVT_S_TIMER |RVT_S_ANY_WAIT ); +spin_unlock (&qp->s_lock); +spin_unlock (&qp->s_hlock); +spin_unlock_irq (&qp->r_lock); + +/* Stop the send queue and the retry timer */ + rdi->driver_f.stop_send_queue (qp); + +/* Wait for things to stop */ + rdi->driver_f.quiesce_qp (qp); + +/* take qp out the hash and wait for it to be unused */ +rvt_remove_qp (rdi, qp); +wait_event (qp->wait, !atomic_read (&qp->refcount)); + +/* grab the lock b/c it was locked at call time */ +spin_lock_irq (&qp->r_lock); +spin_lock (&qp->s_hlock); +spin_lock (&qp->s_lock); + +rvt_clear_mr_refs (qp,1 ); +/* +* Let the driver do any tear down or re-init it needs to for +* a qp that has been reset +*/ + rdi->driver_f.notify_qp_reset (qp); + } +rvt_init_qp (rdi, qp, type); +} + +/** * rvt_create_qp - create a queue pair for a device * @ibpd: the protection domain who's device we create the queue pair for * @init_attr: the attributes of the queue pair
@@ -761,7 +777,7 @@ struct ib_qp *rvt_create_qp(struct ib_pd *ibpd, } qp->ibqp.qp_num = err; qp->port_num = init_attr->port_num; -rvt_reset_qp (rdi, qp, init_attr->qp_type); +rvt_init_qp (rdi, qp, init_attr->qp_type);break ;default :
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
PATCH 13/13 IB/rdmavt, IB/hfi1: Add lockdep asserts for lock debug
In reply to a message not in the archiveFrom: Mike Marciniszyn <[email protected]>
This patch adds lockdep asserts in key code paths for insuring lock correctness.
--- drivers/infiniband/hw/hfi1/qp.c | 8 ++++++-- drivers/infiniband/hw/hfi1/rc.c | 17 +++++++++++++++++ drivers/infiniband/sw/rdmavt/qp.c | 8 ++++++++ 3 files changed, 31 insertions(+), 2 deletions(-)
drivers/infiniband/hw/hfi1/qp.c +6 -2
@@ -449,13 +449,14 @@ static void qp_pio_drain(struct rvt_qp *qp)*/ void hfi1_schedule_send (struct rvt_qp *qp) { +lockdep_assert_held (&qp->s_lock);if (hfi1_send_ok (qp))_hfi1_schedule_send (qp); }/** -* hfi1_get_credit - flush the send work queue of a QP -* @qp: the qp who's send work queue to flush +* hfi1_get_credit - handle credit in aeth +* @qp: the qp * @aeth: the Acknowledge Extended Transport Header * * The QP s_lock should be held.
@@ -464,6 +465,7 @@ void hfi1_get_credit(struct rvt_qp *qp, u32 aeth) {u32 credit = (aeth >>HFI1_AETH_CREDIT_SHIFT ) &HFI1_AETH_CREDIT_MASK ; +lockdep_assert_held (&qp->s_lock);/* * If the credit is invalid, we can send * as many packets as we like. Otherwise, we have to
@@ -822,6 +824,7 @@ void qp_priv_free(struct rvt_dev_info *rdi, struct rvt_qp *qp) {struct hfi1_qp_priv *priv = qp->priv; +lockdep_assert_held (&qp->s_lock);kfree (priv->s_ahg);kfree (priv); }
@@ -853,6 +856,7 @@ unsigned free_all_qps(struct rvt_dev_info *rdi)void flush_qp_waiters (struct rvt_qp *qp) { +lockdep_assert_held (&qp->s_lock);flush_iowait (qp);hfi1_stop_rc_timers (qp); }
drivers/infiniband/hw/hfi1/rc.c +17 -0
@@ -68,6 +68,7 @@ static inline void hfi1_add_retry_timer(struct rvt_qp *qp)struct ib_qp *ibqp = &qp->ibqp;struct rvt_dev_info *rdi =ib_to_rvt (ibqp->device); +lockdep_assert_held (&qp->s_lock); qp->s_flags |=RVT_S_TIMER ;/* 4.096 usec. * (1 << qp->timeout) */ qp->s_timer.expires = jiffies + qp->timeout_jiffies +
@@ -86,6 +87,7 @@ void hfi1_add_rnr_timer(struct rvt_qp *qp, u32 to) {struct hfi1_qp_priv *priv = qp->priv; +lockdep_assert_held (&qp->s_lock); qp->s_flags |=RVT_S_WAIT_RNR ; qp->s_timer.expires = jiffies +usecs_to_jiffies (to);add_timer (&priv->s_rnr_timer);
@@ -103,6 +105,7 @@ static inline void hfi1_mod_retry_timer(struct rvt_qp *qp)struct ib_qp *ibqp = &qp->ibqp;struct rvt_dev_info *rdi =ib_to_rvt (ibqp->device); +lockdep_assert_held (&qp->s_lock); qp->s_flags |=RVT_S_TIMER ;/* 4.096 usec. * (1 << qp->timeout) */ mod_timer(&qp->s_timer, jiffies + qp->timeout_jiffies +
@@ -120,6 +123,7 @@ static inline int hfi1_stop_retry_timer(struct rvt_qp *qp) {int rval =0 ; +lockdep_assert_held (&qp->s_lock);/* Remove QP from retry */ if (qp->s_flags &RVT_S_TIMER ) { qp->s_flags &= ~RVT_S_TIMER ;
@@ -138,6 +142,7 @@ void hfi1_stop_rc_timers(struct rvt_qp *qp) {struct hfi1_qp_priv *priv = qp->priv; +lockdep_assert_held (&qp->s_lock);/* Remove QP from all timers */ if (qp->s_flags & (RVT_S_TIMER |RVT_S_WAIT_RNR )) { qp->s_flags &= ~(RVT_S_TIMER |RVT_S_WAIT_RNR );
@@ -158,6 +163,7 @@ static inline int hfi1_stop_rnr_timer(struct rvt_qp *qp)int rval =0 ;struct hfi1_qp_priv *priv = qp->priv; +lockdep_assert_held (&qp->s_lock);/* Remove QP from rnr timer */ if (qp->s_flags &RVT_S_WAIT_RNR ) { qp->s_flags &= ~RVT_S_WAIT_RNR ;
@@ -228,6 +234,7 @@ static int make_rc_ack(struct hfi1_ibdev *dev, struct rvt_qp *qp,u32 pmtu = qp->pmtu;struct hfi1_qp_priv *priv = qp->priv; +lockdep_assert_held (&qp->s_lock);/* Don't send an ACK if we aren't supposed to. */ if (!(ib_rvt_state_ops[qp->state] &RVT_PROCESS_RECV_OK )) goto bail;
@@ -400,6 +407,7 @@ int hfi1_make_rc_req(struct rvt_qp *qp, struct hfi1_pkt_state *ps)int middle =0 ;int delta; +lockdep_assert_held (&qp->s_lock); ps->s_txreq =get_txreq (ps->dev, qp);if (IS_ERR (ps->s_txreq)) goto bail_no_tx;
@@ -958,6 +966,7 @@ static void reset_psn(struct rvt_qp *qp, u32 psn)struct rvt_swqe *wqe =rvt_get_swqe_ptr (qp, n);u32 opcode; +lockdep_assert_held (&qp->s_lock); qp->s_cur = n;/*
@@ -1043,6 +1052,8 @@ static void restart_rc(struct rvt_qp *qp, u32 psn, int wait)struct rvt_swqe *wqe =rvt_get_swqe_ptr (qp, qp->s_acked);struct hfi1_ibport *ibp; +lockdep_assert_held (&qp->r_lock); +lockdep_assert_held (&qp->s_lock);if (qp->s_retry ==0 ) {if (qp->s_mig_state ==IB_MIG_ARMED ) {hfi1_migrate_qp (qp);
@@ -1119,6 +1130,7 @@ static void reset_sending_psn(struct rvt_qp *qp, u32 psn)struct rvt_swqe *wqe;u32 n = qp->s_last; +lockdep_assert_held (&qp->s_lock);/* Find the work request corresponding to the given PSN. */ for (;;) { wqe =rvt_get_swqe_ptr (qp, n);
@@ -1148,6 +1160,7 @@ void hfi1_rc_send_complete(struct rvt_qp *qp, struct ib_header *hdr)u32 opcode;u32 psn; +lockdep_assert_held (&qp->s_lock);if (!(ib_rvt_state_ops[qp->state] &RVT_PROCESS_OR_FLUSH_SEND ))return ;
@@ -1239,6 +1252,7 @@ static struct rvt_swqe *do_rc_completion(struct rvt_qp *qp,struct ib_wc wc;unsigned i; +lockdep_assert_held (&qp->s_lock);/* * Don't decrement refcount and don't generate a * completion if the SWQE is being resent until the send
@@ -1338,6 +1352,7 @@ static int do_rc_ack(struct rvt_qp *qp, u32 aeth, u32 psn, int opcode,int diff;unsigned long to; +lockdep_assert_held (&qp->s_lock);/* * Note that NAKs implicitly ACK outstanding SEND and RDMA write * requests and implicitly NAK RDMA read and atomic requests issued
@@ -1553,6 +1568,7 @@ static void rdma_seq_err(struct rvt_qp *qp, struct hfi1_ibport *ibp, u32 psn, {struct rvt_swqe *wqe; +lockdep_assert_held (&qp->s_lock);/* Remove QP from retry timer */ hfi1_stop_rc_timers (qp);
@@ -2136,6 +2152,7 @@ void hfi1_rc_rcv(struct hfi1_packet *packet)int copy_last =0 ;u32 rkey; +lockdep_assert_held (&qp->r_lock); bth0 =be32_to_cpu (ohdr->bth[0 ]);if (hfi1_ruc_check_hdr (ibp, hdr, rcv_flags &HFI1_HAS_GRH , qp, bth0))return ;
drivers/infiniband/sw/rdmavt/qp.c +8 -0
@@ -562,6 +562,9 @@ static void rvt_reset_qp(struct rvt_dev_info *rdi, struct rvt_qp *qp, __must_hold(&qp->s_hlock) __must_hold(&qp->r_lock) { +lockdep_assert_held (&qp->r_lock); +lockdep_assert_held (&qp->s_hlock); +lockdep_assert_held (&qp->s_lock);if (qp->state !=IB_QPS_RESET ) { qp->state =IB_QPS_RESET ;
@@ -595,6 +598,9 @@ static void rvt_reset_qp(struct rvt_dev_info *rdi, struct rvt_qp *qp, rdi->driver_f.notify_qp_reset (qp); }rvt_init_qp (rdi, qp, type); +lockdep_assert_held (&qp->r_lock); +lockdep_assert_held (&qp->s_hlock); +lockdep_assert_held (&qp->s_lock); }/**
@@ -917,6 +923,8 @@ int rvt_error_qp(struct rvt_qp *qp, enum ib_wc_status err)int ret =0 ;struct rvt_dev_info *rdi =ib_to_rvt (qp->ibqp.device); +lockdep_assert_held (&qp->r_lock); +lockdep_assert_held (&qp->s_lock);if (qp->state ==IB_QPS_ERR || qp->state ==IB_QPS_RESET ) goto bail;
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
Quoted 79 lines · Dennis Dalessandro
Doug,
Here are a handful of fixes that we didn't think were fitting for the RC release so we are submitting these for 4.9. Note there is a change to the core here but it is pretty harmless. There are also some improvements to rdmavt.
This applies on the last set of fixes I sent out for RC and can be seen in my git tree:
https://github.com/ddalessa/kernel/tree/for-4.9
---
Dean Luick (1):
IB/hfi1: Move serdes tune inside link start functionHarish Chegondi (1):
IB/hfi1: Fix the count of user packets submitted to an SDMA engineMike Marciniszyn (9):
IB/rdmavt: Add functions to get and release QP references IB/rdmavt,IB/qib,IB/hfi1: Use new QP put get routines IB/core: Add ib headers for general use IB/qib,IB/hfi: Use core common header file IB/rdmavt: Correct sparse annotation
IB/hfi1: Moveiowait_init()to priv allocate IB/rdmavt: Move reset calldown to reset path IB/rdmavt: Add qp init function IB/rdmavt,IB/hfi1: Add lockdep asserts for lock debugTadeusz Struk (1):
IB/hfi1: Fix locking scheme for affinity settingsTymoteusz Kielan (1):
IB/hfi1: Fix user-space buffers mapping with IOMMU enableddrivers/infiniband/hw/hfi1/affinity.c | 91 ++++++++------- drivers/infiniband/hw/hfi1/affinity.h | 3 drivers/infiniband/hw/hfi1/chip.c | 40 +++---- drivers/infiniband/hw/hfi1/chip.h | 2 drivers/infiniband/hw/hfi1/common.h | 8 - drivers/infiniband/hw/hfi1/driver.c | 35 +++--- drivers/infiniband/hw/hfi1/file_ops.c | 41 +++++-- drivers/infiniband/hw/hfi1/hfi.h | 17 +-- drivers/infiniband/hw/hfi1/init.c | 44 ++++--- drivers/infiniband/hw/hfi1/mad.c | 1 drivers/infiniband/hw/hfi1/pio.c | 20 ++- drivers/infiniband/hw/hfi1/pio.h | 2 drivers/infiniband/hw/hfi1/qp.c | 33 +++-- drivers/infiniband/hw/hfi1/rc.c | 113 ++++++++++-------- drivers/infiniband/hw/hfi1/ruc.c | 4 - drivers/infiniband/hw/hfi1/sdma.c | 24 ++-- drivers/infiniband/hw/hfi1/sdma.h | 3 drivers/infiniband/hw/hfi1/trace.c | 27 +--- drivers/infiniband/hw/hfi1/trace_ctxts.h | 13 +- drivers/infiniband/hw/hfi1/trace_ibhdrs.h | 14 +- drivers/infiniband/hw/hfi1/trace_rx.h | 4 - drivers/infiniband/hw/hfi1/uc.c | 6 - drivers/infiniband/hw/hfi1/ud.c | 12 +- drivers/infiniband/hw/hfi1/user_sdma.c | 31 ++--- drivers/infiniband/hw/hfi1/verbs.c | 18 +-- drivers/infiniband/hw/hfi1/verbs.h | 90 +-------------- drivers/infiniband/hw/hfi1/verbs_txreq.c | 2 drivers/infiniband/hw/qib/qib.h | 1 drivers/infiniband/hw/qib/qib_driver.c | 7 - drivers/infiniband/hw/qib/qib_iba7322.c | 2 drivers/infiniband/hw/qib/qib_rc.c | 73 +++++------- drivers/infiniband/hw/qib/qib_ruc.c | 4 - drivers/infiniband/hw/qib/qib_uc.c | 6 - drivers/infiniband/hw/qib/qib_ud.c | 6 - drivers/infiniband/hw/qib/qib_verbs.c | 14 +- drivers/infiniband/hw/qib/qib_verbs.h | 94 ++------------- drivers/infiniband/sw/rdmavt/qp.c | 117 +++++++++++-------- include/rdma/ib_hdrs.h | 178 +++++++++++++++++++++++++++++ include/rdma/rdmavt_qp.h | 19 +++ 39 files changed, 643 insertions(+), 576 deletions(-) create mode 100644 include/rdma/ib_hdrs.h
Quoted 3 lines
--
-Denny
Thanks Denny, I've started an hfi1 branch and these are now in for 4.9.
Signature · 2 lines
Doug Ledford <[email protected]> GPG Key ID: 0E572FDD