Skip to contentSkip to conversation

RDMA/rxe: Various bug fixes.

PATCH Superseded Bob Pearson 5 participants 22 messages Sep 9, 2021 – Sep 14, 2021

Subsystems (2)

v3 of 4 Superseded Latest is v4 →

lore
Share & tools
lore
#0 lore raw

v3 cover RDMA/rxe: Various bug fixes.

This series of patches implements several bug fixes and minor cleanups of the rxe driver. Specifically these fix a bug exposed by blktest.

They apply cleanly to both
commit 2169b908894df2ce83e7eb4a399d3224b2635126 (origin/for-rc, for-rc) commit 6a217437f9f5482a3f6f2dc5fcd27cf0f62409ac (HEAD -> for-next,
origin/wip/jgg-for-next, origin/for-next, origin/HEAD)

These are being resubmitted to for-rc instead of for-next.

The v2 version had a typo which broke clean application to for-next. Additionally in v3 the order of the patches was changed to make it a little cleaner.

The first patch is a repeat of an earlier patch after rebasing. It adds memory barriers to kernel to kernel queues. The logic for this is the same as an earlier patch that only treated user to kernel queues. Without this patch kernel to kernel queues are expected to intermittently fail at low frequency as was seen for the other queues.

The second patch is also a repeat after rebasing. It fixes a multicast bug.

The third patch cleans up the state and type enums used by MRs.

The fourth patch separates the keys in rxe_mr and ib_mr. This allows the following sequence seen in the srp driver to work correctly.

	do {
		ib_post_send( IB_WR_LOCAL_INV )
		ib_update_fast_reg_key()
		ib_map_mr_sg()
		ib_post_send( IB_WR_REG_MR )
	} while ( !done )

The fifth patch creates duplicate mapping tables for fast MRs. This prevents rkeys referencing fast MRs from accessing data from an updated map after the call to ib_map_mr_sg() call by keeping the new and old mappings separate and atomically swapping them when a reg mr WR is executed.

The sixth patch checks the type of MRs which receive local or remote invalidate operations to prevent invalidating user MRs.

Bob Pearson (6):
RDMA/rxe: Add memory barriers to kernel queues RDMA/rxe: Fix memory allocation while locked RDMA/rxe: Cleanup MR status and type enums RDMA/rxe: Separate HW and SW l/rkeys RDMA/rxe: Create duplicate mapping tables for FMRs RDMA/rxe: Only allow invalidate for appropriate MRs

 drivers/infiniband/sw/rxe/rxe_comp.c  |  10 +-
 drivers/infiniband/sw/rxe/rxe_cq.c    |  25 +--
 drivers/infiniband/sw/rxe/rxe_loc.h   |   2 +
 drivers/infiniband/sw/rxe/rxe_mcast.c |   2 +-
 drivers/infiniband/sw/rxe/rxe_mr.c    | 267 +++++++++++++++++++-------
 drivers/infiniband/sw/rxe/rxe_mw.c    |  36 ++--
 drivers/infiniband/sw/rxe/rxe_qp.c    |  10 +-
 drivers/infiniband/sw/rxe/rxe_queue.h |  73 ++-----
 drivers/infiniband/sw/rxe/rxe_req.c   |  35 +---
 drivers/infiniband/sw/rxe/rxe_resp.c  |  38 +---
 drivers/infiniband/sw/rxe/rxe_srq.c   |   2 +-
 drivers/infiniband/sw/rxe/rxe_verbs.c |  92 +++------
 drivers/infiniband/sw/rxe/rxe_verbs.h |  48 ++---
 13 files changed, 305 insertions(+), 335 deletions(-)
Signature · 1 line
2.30.2
#1 lore raw

PATCH v3 1/6 RDMA/rxe: Add memory barriers to kernel queues

Earlier patches added memory barriers to protect user space to kernel space communications. The user space queues were previously shown to have occasional memory synchonization errors which were removed by adding smp_load_acquire, smp_store_release barriers.

This patch extends that to the case where queues are used between kernel space threads.

---
 drivers/infiniband/sw/rxe/rxe_comp.c  | 10 +---
 drivers/infiniband/sw/rxe/rxe_cq.c    | 25 ++-------
 drivers/infiniband/sw/rxe/rxe_qp.c    | 10 ++--
 drivers/infiniband/sw/rxe/rxe_queue.h | 73 ++++++++-------------------
 drivers/infiniband/sw/rxe/rxe_req.c   | 21 ++------
 drivers/infiniband/sw/rxe/rxe_resp.c  | 38 ++++----------
 drivers/infiniband/sw/rxe/rxe_srq.c   |  2 +-
 drivers/infiniband/sw/rxe/rxe_verbs.c | 53 ++++---------------
 8 files changed, 55 insertions(+), 177 deletions(-)
drivers/infiniband/sw/rxe/rxe_comp.c +2 -8
index d2d802c776fd..ed4e3f29bd65 100644
@@ -142,10 +142,7 @@ static inline enum comp_state get_wqe(struct rxe_qp *qp,
 	/* we come here whether or not we found a response packet to see if
 	 * there are any posted WQEs
 	 */
-	if (qp->is_user)
-		wqe = queue_head(qp->sq.queue, QUEUE_TYPE_FROM_USER);
-	else
-		wqe = queue_head(qp->sq.queue, QUEUE_TYPE_KERNEL);
+	wqe = queue_head(qp->sq.queue, QUEUE_TYPE_FROM_CLIENT);
 	*wqe_p = wqe;
 
 	/* no WQE or requester has not started it yet */
@@ -432,10 +429,7 @@ static void do_complete(struct rxe_qp *qp, struct rxe_send_wqe *wqe)
 	if (post)
 		make_send_cqe(qp, wqe, &cqe);
 
-	if (qp->is_user)
-		advance_consumer(qp->sq.queue, QUEUE_TYPE_FROM_USER);
-	else
-		advance_consumer(qp->sq.queue, QUEUE_TYPE_KERNEL);
+	advance_consumer(qp->sq.queue, QUEUE_TYPE_FROM_CLIENT);
 
 	if (post)
 		rxe_cq_post(qp->scq, &cqe, 0);
drivers/infiniband/sw/rxe/rxe_cq.c +5 -20
index aef288f164fd..4e26c2ea4a59 100644
@@ -25,11 +25,7 @@ int rxe_cq_chk_attr(struct rxe_dev *rxe, struct rxe_cq *cq,
 	}
 
 	if (cq) {
-		if (cq->is_user)
-			count = queue_count(cq->queue, QUEUE_TYPE_TO_USER);
-		else
-			count = queue_count(cq->queue, QUEUE_TYPE_KERNEL);
-
+		count = queue_count(cq->queue, QUEUE_TYPE_TO_CLIENT);
 		if (cqe < count) {
 			pr_warn("cqe(%d) < current # elements in queue (%d)",
 				cqe, count);
@@ -65,7 +61,7 @@ int rxe_cq_from_init(struct rxe_dev *rxe, struct rxe_cq *cq, int cqe,
 	int err;
 	enum queue_type type;
 
-	type = uresp ? QUEUE_TYPE_TO_USER : QUEUE_TYPE_KERNEL;
+	type = QUEUE_TYPE_TO_CLIENT;
 	cq->queue = rxe_queue_init(rxe, &cqe,
 			sizeof(struct rxe_cqe), type);
 	if (!cq->queue) {
@@ -117,11 +113,7 @@ int rxe_cq_post(struct rxe_cq *cq, struct rxe_cqe *cqe, int solicited)
 
 	spin_lock_irqsave(&cq->cq_lock, flags);
 
-	if (cq->is_user)
-		full = queue_full(cq->queue, QUEUE_TYPE_TO_USER);
-	else
-		full = queue_full(cq->queue, QUEUE_TYPE_KERNEL);
-
+	full = queue_full(cq->queue, QUEUE_TYPE_TO_CLIENT);
 	if (unlikely(full)) {
 		spin_unlock_irqrestore(&cq->cq_lock, flags);
 		if (cq->ibcq.event_handler) {
@@ -134,17 +126,10 @@ int rxe_cq_post(struct rxe_cq *cq, struct rxe_cqe *cqe, int solicited)
 		return -EBUSY;
 	}
 
-	if (cq->is_user)
-		addr = producer_addr(cq->queue, QUEUE_TYPE_TO_USER);
-	else
-		addr = producer_addr(cq->queue, QUEUE_TYPE_KERNEL);
-
+	addr = producer_addr(cq->queue, QUEUE_TYPE_TO_CLIENT);
 	memcpy(addr, cqe, sizeof(*cqe));
 
-	if (cq->is_user)
-		advance_producer(cq->queue, QUEUE_TYPE_TO_USER);
-	else
-		advance_producer(cq->queue, QUEUE_TYPE_KERNEL);
+	advance_producer(cq->queue, QUEUE_TYPE_TO_CLIENT);
 
 	spin_unlock_irqrestore(&cq->cq_lock, flags);
 
drivers/infiniband/sw/rxe/rxe_qp.c +3 -7
index 1ab6af7ddb25..2e923af642f8 100644
@@ -231,7 +231,7 @@ static int rxe_qp_init_req(struct rxe_dev *rxe, struct rxe_qp *qp,
 	qp->sq.max_inline = init->cap.max_inline_data = wqe_size;
 	wqe_size += sizeof(struct rxe_send_wqe);
 
-	type = uresp ? QUEUE_TYPE_FROM_USER : QUEUE_TYPE_KERNEL;
+	type = QUEUE_TYPE_FROM_CLIENT;
 	qp->sq.queue = rxe_queue_init(rxe, &qp->sq.max_wr,
 				wqe_size, type);
 	if (!qp->sq.queue)
@@ -248,12 +248,8 @@ static int rxe_qp_init_req(struct rxe_dev *rxe, struct rxe_qp *qp,
 		return err;
 	}
 
-	if (qp->is_user)
 		qp->req.wqe_index = producer_index(qp->sq.queue,
-						QUEUE_TYPE_FROM_USER);
-	else
-		qp->req.wqe_index = producer_index(qp->sq.queue,
-						QUEUE_TYPE_KERNEL);
+					QUEUE_TYPE_FROM_CLIENT);
 
 	qp->req.state		= QP_STATE_RESET;
 	qp->req.opcode		= -1;
@@ -293,7 +289,7 @@ static int rxe_qp_init_resp(struct rxe_dev *rxe, struct rxe_qp *qp,
 		pr_debug("qp#%d max_wr = %d, max_sge = %d, wqe_size = %d\n",
 			 qp_num(qp), qp->rq.max_wr, qp->rq.max_sge, wqe_size);
 
-		type = uresp ? QUEUE_TYPE_FROM_USER : QUEUE_TYPE_KERNEL;
+		type = QUEUE_TYPE_FROM_CLIENT;
 		qp->rq.queue = rxe_queue_init(rxe, &qp->rq.max_wr,
 					wqe_size, type);
 		if (!qp->rq.queue)
drivers/infiniband/sw/rxe/rxe_queue.h +20 -53

201 lines not on this page · raw message

drivers/infiniband/sw/rxe/rxe_req.c +5 -16
index 3894197a82f6..22c3edb28945 100644
@@ -49,13 +49,8 @@ static void req_retry(struct rxe_qp *qp)
 	unsigned int cons;
 	unsigned int prod;
 
-	if (qp->is_user) {
-		cons = consumer_index(q, QUEUE_TYPE_FROM_USER);
-		prod = producer_index(q, QUEUE_TYPE_FROM_USER);
-	} else {
-		cons = consumer_index(q, QUEUE_TYPE_KERNEL);
-		prod = producer_index(q, QUEUE_TYPE_KERNEL);
-	}
+	cons = consumer_index(q, QUEUE_TYPE_FROM_CLIENT);
+	prod = producer_index(q, QUEUE_TYPE_FROM_CLIENT);
 
 	qp->req.wqe_index	= cons;
 	qp->req.psn		= qp->comp.psn;
@@ -121,15 +116,9 @@ static struct rxe_send_wqe *req_next_wqe(struct rxe_qp *qp)
 	unsigned int cons;
 	unsigned int prod;
 
-	if (qp->is_user) {
-		wqe = queue_head(q, QUEUE_TYPE_FROM_USER);
-		cons = consumer_index(q, QUEUE_TYPE_FROM_USER);
-		prod = producer_index(q, QUEUE_TYPE_FROM_USER);
-	} else {
-		wqe = queue_head(q, QUEUE_TYPE_KERNEL);
-		cons = consumer_index(q, QUEUE_TYPE_KERNEL);
-		prod = producer_index(q, QUEUE_TYPE_KERNEL);
-	}
+	wqe = queue_head(q, QUEUE_TYPE_FROM_CLIENT);
+	cons = consumer_index(q, QUEUE_TYPE_FROM_CLIENT);
+	prod = producer_index(q, QUEUE_TYPE_FROM_CLIENT);
 
 	if (unlikely(qp->req.state == QP_STATE_DRAIN)) {
 		/* check to see if we are drained;
drivers/infiniband/sw/rxe/rxe_resp.c +9 -29
index 5501227ddc65..596be002d33d 100644
@@ -303,10 +303,7 @@ static enum resp_states get_srq_wqe(struct rxe_qp *qp)
 
 	spin_lock_bh(&srq->rq.consumer_lock);
 
-	if (qp->is_user)
-		wqe = queue_head(q, QUEUE_TYPE_FROM_USER);
-	else
-		wqe = queue_head(q, QUEUE_TYPE_KERNEL);
+	wqe = queue_head(q, QUEUE_TYPE_FROM_CLIENT);
 	if (!wqe) {
 		spin_unlock_bh(&srq->rq.consumer_lock);
 		return RESPST_ERR_RNR;
@@ -322,13 +319,8 @@ static enum resp_states get_srq_wqe(struct rxe_qp *qp)
 	memcpy(&qp->resp.srq_wqe, wqe, size);
 
 	qp->resp.wqe = &qp->resp.srq_wqe.wqe;
-	if (qp->is_user) {
-		advance_consumer(q, QUEUE_TYPE_FROM_USER);
-		count = queue_count(q, QUEUE_TYPE_FROM_USER);
-	} else {
-		advance_consumer(q, QUEUE_TYPE_KERNEL);
-		count = queue_count(q, QUEUE_TYPE_KERNEL);
-	}
+	advance_consumer(q, QUEUE_TYPE_FROM_CLIENT);
+	count = queue_count(q, QUEUE_TYPE_FROM_CLIENT);
 
 	if (srq->limit && srq->ibsrq.event_handler && (count < srq->limit)) {
 		srq->limit = 0;
@@ -357,12 +349,8 @@ static enum resp_states check_resource(struct rxe_qp *qp,
 			qp->resp.status = IB_WC_WR_FLUSH_ERR;
 			return RESPST_COMPLETE;
 		} else if (!srq) {
-			if (qp->is_user)
-				qp->resp.wqe = queue_head(qp->rq.queue,
-						QUEUE_TYPE_FROM_USER);
-			else
-				qp->resp.wqe = queue_head(qp->rq.queue,
-						QUEUE_TYPE_KERNEL);
+			qp->resp.wqe = queue_head(qp->rq.queue,
+					QUEUE_TYPE_FROM_CLIENT);
 			if (qp->resp.wqe) {
 				qp->resp.status = IB_WC_WR_FLUSH_ERR;
 				return RESPST_COMPLETE;
@@ -389,12 +377,8 @@ static enum resp_states check_resource(struct rxe_qp *qp,
 		if (srq)
 			return get_srq_wqe(qp);
 
-		if (qp->is_user)
-			qp->resp.wqe = queue_head(qp->rq.queue,
-					QUEUE_TYPE_FROM_USER);
-		else
-			qp->resp.wqe = queue_head(qp->rq.queue,
-					QUEUE_TYPE_KERNEL);
+		qp->resp.wqe = queue_head(qp->rq.queue,
+				QUEUE_TYPE_FROM_CLIENT);
 		return (qp->resp.wqe) ? RESPST_CHK_LENGTH : RESPST_ERR_RNR;
 	}
 
@@ -936,12 +920,8 @@ static enum resp_states do_complete(struct rxe_qp *qp,
 	}
 
 	/* have copy for srq and reference for !srq */
-	if (!qp->srq) {
-		if (qp->is_user)
-			advance_consumer(qp->rq.queue, QUEUE_TYPE_FROM_USER);
-		else
-			advance_consumer(qp->rq.queue, QUEUE_TYPE_KERNEL);
-	}
+	if (!qp->srq)
+		advance_consumer(qp->rq.queue, QUEUE_TYPE_FROM_CLIENT);
 
 	qp->resp.wqe = NULL;
 
drivers/infiniband/sw/rxe/rxe_srq.c +1 -1
index 610c98d24b5c..a9e7817e2732 100644
@@ -93,7 +93,7 @@ int rxe_srq_from_init(struct rxe_dev *rxe, struct rxe_srq *srq,
 	spin_lock_init(&srq->rq.producer_lock);
 	spin_lock_init(&srq->rq.consumer_lock);
 
-	type = uresp ? QUEUE_TYPE_FROM_USER : QUEUE_TYPE_KERNEL;
+	type = QUEUE_TYPE_FROM_CLIENT;
 	q = rxe_queue_init(rxe, &srq->rq.max_wr,
 			srq_wqe_size, type);
 	if (!q) {
drivers/infiniband/sw/rxe/rxe_verbs.c +10 -43

107 lines not on this page · raw message

Signature · 1 line
2.30.2
maintainer
#2 lore raw

Re: [PATCH for-rc v3 1/6] RDMA/rxe: Add memory barriers to kernel queues

Quoted 26 lines · Bob Pearson

Earlier patches added memory barriers to protect user space to kernel space communications. The user space queues were previously shown to have occasional memory synchonization errors which were removed by adding smp_load_acquire, smp_store_release barriers.

This patch extends that to the case where queues are used between kernel space threads.

---
 drivers/infiniband/sw/rxe/rxe_comp.c  | 10 +---
 drivers/infiniband/sw/rxe/rxe_cq.c    | 25 ++-------
 drivers/infiniband/sw/rxe/rxe_qp.c    | 10 ++--
 drivers/infiniband/sw/rxe/rxe_queue.h | 73 ++++++++-------------------
 drivers/infiniband/sw/rxe/rxe_req.c   | 21 ++------
 drivers/infiniband/sw/rxe/rxe_resp.c  | 38 ++++----------
 drivers/infiniband/sw/rxe/rxe_srq.c   |  2 +-
 drivers/infiniband/sw/rxe/rxe_verbs.c | 53 ++++---------------
 8 files changed, 55 insertions(+), 177 deletions(-)
drivers/infiniband/sw/rxe/rxe_comp.c +0 -0
index d2d802c776fd..ed4e3f29bd65 100644

drivers/infiniband/sw/rxe/rxe_comp.c static inline enum comp_state get_wqe(struct rxe_qp *qp, · line 148

Quoted 7 lines
drivers/infiniband/sw/rxe/rxe_comp.c +0 -4
@@ -142,10 +142,7 @@ static inline enum comp_state get_wqe(struct rxe_qp *qp,
        /* we come here whether or not we found a response packet to see if
         * there are any posted WQEs
         */
-       if (qp->is_user)
-               wqe = queue_head(qp->sq.queue, QUEUE_TYPE_FROM_USER);
-       else
-               wqe = queue_head(qp->sq.queue, QUEUE_TYPE_KERNEL);

This commit is very similar to the commit in
https://lore.kernel.org/linux-rdma/[email protected]/T/

Zhu Yanjun

Quoted 578 lines
drivers/infiniband/sw/rxe/rxe_comp.c +2 -4
+       wqe = queue_head(qp->sq.queue, QUEUE_TYPE_FROM_CLIENT);
        *wqe_p = wqe;
 
        /* no WQE or requester has not started it yet */
@@ -432,10 +429,7 @@ static void do_complete(struct rxe_qp *qp, struct rxe_send_wqe *wqe)
        if (post)
                make_send_cqe(qp, wqe, &cqe);
 
-       if (qp->is_user)
-               advance_consumer(qp->sq.queue, QUEUE_TYPE_FROM_USER);
-       else
-               advance_consumer(qp->sq.queue, QUEUE_TYPE_KERNEL);
+       advance_consumer(qp->sq.queue, QUEUE_TYPE_FROM_CLIENT);
 
        if (post)
                rxe_cq_post(qp->scq, &cqe, 0);
drivers/infiniband/sw/rxe/rxe_cq.c +5 -20
index aef288f164fd..4e26c2ea4a59 100644
@@ -25,11 +25,7 @@ int rxe_cq_chk_attr(struct rxe_dev *rxe, struct rxe_cq *cq,
        }
 
        if (cq) {
-               if (cq->is_user)
-                       count = queue_count(cq->queue, QUEUE_TYPE_TO_USER);
-               else
-                       count = queue_count(cq->queue, QUEUE_TYPE_KERNEL);
-
+               count = queue_count(cq->queue, QUEUE_TYPE_TO_CLIENT);
                if (cqe < count) {
                        pr_warn("cqe(%d) < current # elements in queue (%d)",
                                cqe, count);
@@ -65,7 +61,7 @@ int rxe_cq_from_init(struct rxe_dev *rxe, struct rxe_cq *cq, int cqe,
        int err;
        enum queue_type type;
 
-       type = uresp ? QUEUE_TYPE_TO_USER : QUEUE_TYPE_KERNEL;
+       type = QUEUE_TYPE_TO_CLIENT;
        cq->queue = rxe_queue_init(rxe, &cqe,
                        sizeof(struct rxe_cqe), type);
        if (!cq->queue) {
@@ -117,11 +113,7 @@ int rxe_cq_post(struct rxe_cq *cq, struct rxe_cqe *cqe, int solicited)
 
        spin_lock_irqsave(&cq->cq_lock, flags);
 
-       if (cq->is_user)
-               full = queue_full(cq->queue, QUEUE_TYPE_TO_USER);
-       else
-               full = queue_full(cq->queue, QUEUE_TYPE_KERNEL);
-
+       full = queue_full(cq->queue, QUEUE_TYPE_TO_CLIENT);
        if (unlikely(full)) {
                spin_unlock_irqrestore(&cq->cq_lock, flags);
                if (cq->ibcq.event_handler) {
@@ -134,17 +126,10 @@ int rxe_cq_post(struct rxe_cq *cq, struct rxe_cqe *cqe, int solicited)
                return -EBUSY;
        }
 
-       if (cq->is_user)
-               addr = producer_addr(cq->queue, QUEUE_TYPE_TO_USER);
-       else
-               addr = producer_addr(cq->queue, QUEUE_TYPE_KERNEL);
-
+       addr = producer_addr(cq->queue, QUEUE_TYPE_TO_CLIENT);
        memcpy(addr, cqe, sizeof(*cqe));
 
-       if (cq->is_user)
-               advance_producer(cq->queue, QUEUE_TYPE_TO_USER);
-       else
-               advance_producer(cq->queue, QUEUE_TYPE_KERNEL);
+       advance_producer(cq->queue, QUEUE_TYPE_TO_CLIENT);
 
        spin_unlock_irqrestore(&cq->cq_lock, flags);
 
drivers/infiniband/sw/rxe/rxe_qp.c +3 -7
index 1ab6af7ddb25..2e923af642f8 100644
@@ -231,7 +231,7 @@ static int rxe_qp_init_req(struct rxe_dev *rxe, struct rxe_qp *qp,
        qp->sq.max_inline = init->cap.max_inline_data = wqe_size;
        wqe_size += sizeof(struct rxe_send_wqe);
 
-       type = uresp ? QUEUE_TYPE_FROM_USER : QUEUE_TYPE_KERNEL;
+       type = QUEUE_TYPE_FROM_CLIENT;
        qp->sq.queue = rxe_queue_init(rxe, &qp->sq.max_wr,
                                wqe_size, type);
        if (!qp->sq.queue)
@@ -248,12 +248,8 @@ static int rxe_qp_init_req(struct rxe_dev *rxe, struct rxe_qp *qp,
                return err;
        }
 
-       if (qp->is_user)
                qp->req.wqe_index = producer_index(qp->sq.queue,
-                                               QUEUE_TYPE_FROM_USER);
-       else
-               qp->req.wqe_index = producer_index(qp->sq.queue,
-                                               QUEUE_TYPE_KERNEL);
+                                       QUEUE_TYPE_FROM_CLIENT);
 
        qp->req.state           = QP_STATE_RESET;
        qp->req.opcode          = -1;
@@ -293,7 +289,7 @@ static int rxe_qp_init_resp(struct rxe_dev *rxe, struct rxe_qp *qp,
                pr_debug("qp#%d max_wr = %d, max_sge = %d, wqe_size = %d\n",
                         qp_num(qp), qp->rq.max_wr, qp->rq.max_sge, wqe_size);
 
-               type = uresp ? QUEUE_TYPE_FROM_USER : QUEUE_TYPE_KERNEL;
+               type = QUEUE_TYPE_FROM_CLIENT;
                qp->rq.queue = rxe_queue_init(rxe, &qp->rq.max_wr,
                                        wqe_size, type);
                if (!qp->rq.queue)
drivers/infiniband/sw/rxe/rxe_queue.h +20 -53
index 2702b0e55fc3..d465aa9342e1 100644
@@ -35,9 +35,8 @@
 
 /* type of queue */
 enum queue_type {
-       QUEUE_TYPE_KERNEL,
-       QUEUE_TYPE_TO_USER,
-       QUEUE_TYPE_FROM_USER,
+       QUEUE_TYPE_TO_CLIENT,
+       QUEUE_TYPE_FROM_CLIENT,
 };
 
 struct rxe_queue {
@@ -87,20 +86,16 @@ static inline int queue_empty(struct rxe_queue *q, enum queue_type type)
        u32 cons;
 
        switch (type) {
-       case QUEUE_TYPE_FROM_USER:
+       case QUEUE_TYPE_FROM_CLIENT:
                /* protect user space index */
                prod = smp_load_acquire(&q->buf->producer_index);
                cons = q->index;
                break;
-       case QUEUE_TYPE_TO_USER:
+       case QUEUE_TYPE_TO_CLIENT:
                prod = q->index;
                /* protect user space index */
                cons = smp_load_acquire(&q->buf->consumer_index);
                break;
-       case QUEUE_TYPE_KERNEL:
-               prod = q->buf->producer_index;
-               cons = q->buf->consumer_index;
-               break;
        }
 
        return ((prod - cons) & q->index_mask) == 0;
@@ -112,20 +107,16 @@ static inline int queue_full(struct rxe_queue *q, enum queue_type type)
        u32 cons;
 
        switch (type) {
-       case QUEUE_TYPE_FROM_USER:
+       case QUEUE_TYPE_FROM_CLIENT:
                /* protect user space index */
                prod = smp_load_acquire(&q->buf->producer_index);
                cons = q->index;
                break;
-       case QUEUE_TYPE_TO_USER:
+       case QUEUE_TYPE_TO_CLIENT:
                prod = q->index;
                /* protect user space index */
                cons = smp_load_acquire(&q->buf->consumer_index);
                break;
-       case QUEUE_TYPE_KERNEL:
-               prod = q->buf->producer_index;
-               cons = q->buf->consumer_index;
-               break;
        }
 
        return ((prod + 1 - cons) & q->index_mask) == 0;
@@ -138,20 +129,16 @@ static inline unsigned int queue_count(const struct rxe_queue *q,
        u32 cons;
 
        switch (type) {
-       case QUEUE_TYPE_FROM_USER:
+       case QUEUE_TYPE_FROM_CLIENT:
                /* protect user space index */
                prod = smp_load_acquire(&q->buf->producer_index);
                cons = q->index;
                break;
-       case QUEUE_TYPE_TO_USER:
+       case QUEUE_TYPE_TO_CLIENT:
                prod = q->index;
                /* protect user space index */
                cons = smp_load_acquire(&q->buf->consumer_index);
                break;
-       case QUEUE_TYPE_KERNEL:
-               prod = q->buf->producer_index;
-               cons = q->buf->consumer_index;
-               break;
        }
 
        return (prod - cons) & q->index_mask;
@@ -162,7 +149,7 @@ static inline void advance_producer(struct rxe_queue *q, enum queue_type type)
        u32 prod;
 
        switch (type) {
-       case QUEUE_TYPE_FROM_USER:
+       case QUEUE_TYPE_FROM_CLIENT:
                pr_warn_once("Normally kernel should not write user space index\n");
                /* protect user space index */
                prod = smp_load_acquire(&q->buf->producer_index);
@@ -170,15 +157,11 @@ static inline void advance_producer(struct rxe_queue *q, enum queue_type type)
                /* same */
                smp_store_release(&q->buf->producer_index, prod);
                break;
-       case QUEUE_TYPE_TO_USER:
+       case QUEUE_TYPE_TO_CLIENT:
                prod = q->index;
                q->index = (prod + 1) & q->index_mask;
                q->buf->producer_index = q->index;
                break;
-       case QUEUE_TYPE_KERNEL:
-               prod = q->buf->producer_index;
-               q->buf->producer_index = (prod + 1) & q->index_mask;
-               break;
        }
 }
 
@@ -187,12 +170,12 @@ static inline void advance_consumer(struct rxe_queue *q, enum queue_type type)
        u32 cons;
 
        switch (type) {
-       case QUEUE_TYPE_FROM_USER:
+       case QUEUE_TYPE_FROM_CLIENT:
                cons = q->index;
                q->index = (cons + 1) & q->index_mask;
                q->buf->consumer_index = q->index;
                break;
-       case QUEUE_TYPE_TO_USER:
+       case QUEUE_TYPE_TO_CLIENT:
                pr_warn_once("Normally kernel should not write user space index\n");
                /* protect user space index */
                cons = smp_load_acquire(&q->buf->consumer_index);
@@ -200,10 +183,6 @@ static inline void advance_consumer(struct rxe_queue *q, enum queue_type type)
                /* same */
                smp_store_release(&q->buf->consumer_index, cons);
                break;
-       case QUEUE_TYPE_KERNEL:
-               cons = q->buf->consumer_index;
-               q->buf->consumer_index = (cons + 1) & q->index_mask;
-               break;
        }
 }
 
@@ -212,17 +191,14 @@ static inline void *producer_addr(struct rxe_queue *q, enum queue_type type)
        u32 prod;
 
        switch (type) {
-       case QUEUE_TYPE_FROM_USER:
+       case QUEUE_TYPE_FROM_CLIENT:
                /* protect user space index */
                prod = smp_load_acquire(&q->buf->producer_index);
                prod &= q->index_mask;
                break;
-       case QUEUE_TYPE_TO_USER:
+       case QUEUE_TYPE_TO_CLIENT:
                prod = q->index;
                break;
-       case QUEUE_TYPE_KERNEL:
-               prod = q->buf->producer_index;
-               break;
        }
 
        return q->buf->data + (prod << q->log2_elem_size);
@@ -233,17 +209,14 @@ static inline void *consumer_addr(struct rxe_queue *q, enum queue_type type)
        u32 cons;
 
        switch (type) {
-       case QUEUE_TYPE_FROM_USER:
+       case QUEUE_TYPE_FROM_CLIENT:
                cons = q->index;
                break;
-       case QUEUE_TYPE_TO_USER:
+       case QUEUE_TYPE_TO_CLIENT:
                /* protect user space index */
                cons = smp_load_acquire(&q->buf->consumer_index);
                cons &= q->index_mask;
                break;
-       case QUEUE_TYPE_KERNEL:
-               cons = q->buf->consumer_index;
-               break;
        }
 
        return q->buf->data + (cons << q->log2_elem_size);
@@ -255,17 +228,14 @@ static inline unsigned int producer_index(struct rxe_queue *q,
        u32 prod;
 
        switch (type) {
-       case QUEUE_TYPE_FROM_USER:
+       case QUEUE_TYPE_FROM_CLIENT:
                /* protect user space index */
                prod = smp_load_acquire(&q->buf->producer_index);
                prod &= q->index_mask;
                break;
-       case QUEUE_TYPE_TO_USER:
+       case QUEUE_TYPE_TO_CLIENT:
                prod = q->index;
                break;
-       case QUEUE_TYPE_KERNEL:
-               prod = q->buf->producer_index;
-               break;
        }
 
        return prod;
@@ -277,17 +247,14 @@ static inline unsigned int consumer_index(struct rxe_queue *q,
        u32 cons;
 
        switch (type) {
-       case QUEUE_TYPE_FROM_USER:
+       case QUEUE_TYPE_FROM_CLIENT:
                cons = q->index;
                break;
-       case QUEUE_TYPE_TO_USER:
+       case QUEUE_TYPE_TO_CLIENT:
                /* protect user space index */
                cons = smp_load_acquire(&q->buf->consumer_index);
                cons &= q->index_mask;
                break;
-       case QUEUE_TYPE_KERNEL:
-               cons = q->buf->consumer_index;
-               break;
        }
 
        return cons;
drivers/infiniband/sw/rxe/rxe_req.c +5 -16
index 3894197a82f6..22c3edb28945 100644
@@ -49,13 +49,8 @@ static void req_retry(struct rxe_qp *qp)
        unsigned int cons;
        unsigned int prod;
 
-       if (qp->is_user) {
-               cons = consumer_index(q, QUEUE_TYPE_FROM_USER);
-               prod = producer_index(q, QUEUE_TYPE_FROM_USER);
-       } else {
-               cons = consumer_index(q, QUEUE_TYPE_KERNEL);
-               prod = producer_index(q, QUEUE_TYPE_KERNEL);
-       }
+       cons = consumer_index(q, QUEUE_TYPE_FROM_CLIENT);
+       prod = producer_index(q, QUEUE_TYPE_FROM_CLIENT);
 
        qp->req.wqe_index       = cons;
        qp->req.psn             = qp->comp.psn;
@@ -121,15 +116,9 @@ static struct rxe_send_wqe *req_next_wqe(struct rxe_qp *qp)
        unsigned int cons;
        unsigned int prod;
 
-       if (qp->is_user) {
-               wqe = queue_head(q, QUEUE_TYPE_FROM_USER);
-               cons = consumer_index(q, QUEUE_TYPE_FROM_USER);
-               prod = producer_index(q, QUEUE_TYPE_FROM_USER);
-       } else {
-               wqe = queue_head(q, QUEUE_TYPE_KERNEL);
-               cons = consumer_index(q, QUEUE_TYPE_KERNEL);
-               prod = producer_index(q, QUEUE_TYPE_KERNEL);
-       }
+       wqe = queue_head(q, QUEUE_TYPE_FROM_CLIENT);
+       cons = consumer_index(q, QUEUE_TYPE_FROM_CLIENT);
+       prod = producer_index(q, QUEUE_TYPE_FROM_CLIENT);
 
        if (unlikely(qp->req.state == QP_STATE_DRAIN)) {
                /* check to see if we are drained;
drivers/infiniband/sw/rxe/rxe_resp.c +9 -29
index 5501227ddc65..596be002d33d 100644
@@ -303,10 +303,7 @@ static enum resp_states get_srq_wqe(struct rxe_qp *qp)
 
        spin_lock_bh(&srq->rq.consumer_lock);
 
-       if (qp->is_user)
-               wqe = queue_head(q, QUEUE_TYPE_FROM_USER);
-       else
-               wqe = queue_head(q, QUEUE_TYPE_KERNEL);
+       wqe = queue_head(q, QUEUE_TYPE_FROM_CLIENT);
        if (!wqe) {
                spin_unlock_bh(&srq->rq.consumer_lock);
                return RESPST_ERR_RNR;
@@ -322,13 +319,8 @@ static enum resp_states get_srq_wqe(struct rxe_qp *qp)
        memcpy(&qp->resp.srq_wqe, wqe, size);
 
        qp->resp.wqe = &qp->resp.srq_wqe.wqe;
-       if (qp->is_user) {
-               advance_consumer(q, QUEUE_TYPE_FROM_USER);
-               count = queue_count(q, QUEUE_TYPE_FROM_USER);
-       } else {
-               advance_consumer(q, QUEUE_TYPE_KERNEL);
-               count = queue_count(q, QUEUE_TYPE_KERNEL);
-       }
+       advance_consumer(q, QUEUE_TYPE_FROM_CLIENT);
+       count = queue_count(q, QUEUE_TYPE_FROM_CLIENT);
 
        if (srq->limit && srq->ibsrq.event_handler && (count < srq->limit)) {
                srq->limit = 0;
@@ -357,12 +349,8 @@ static enum resp_states check_resource(struct rxe_qp *qp,
                        qp->resp.status = IB_WC_WR_FLUSH_ERR;
                        return RESPST_COMPLETE;
                } else if (!srq) {
-                       if (qp->is_user)
-                               qp->resp.wqe = queue_head(qp->rq.queue,
-                                               QUEUE_TYPE_FROM_USER);
-                       else
-                               qp->resp.wqe = queue_head(qp->rq.queue,
-                                               QUEUE_TYPE_KERNEL);
+                       qp->resp.wqe = queue_head(qp->rq.queue,
+                                       QUEUE_TYPE_FROM_CLIENT);
                        if (qp->resp.wqe) {
                                qp->resp.status = IB_WC_WR_FLUSH_ERR;
                                return RESPST_COMPLETE;
@@ -389,12 +377,8 @@ static enum resp_states check_resource(struct rxe_qp *qp,
                if (srq)
                        return get_srq_wqe(qp);
 
-               if (qp->is_user)
-                       qp->resp.wqe = queue_head(qp->rq.queue,
-                                       QUEUE_TYPE_FROM_USER);
-               else
-                       qp->resp.wqe = queue_head(qp->rq.queue,
-                                       QUEUE_TYPE_KERNEL);
+               qp->resp.wqe = queue_head(qp->rq.queue,
+                               QUEUE_TYPE_FROM_CLIENT);
                return (qp->resp.wqe) ? RESPST_CHK_LENGTH : RESPST_ERR_RNR;
        }
 
@@ -936,12 +920,8 @@ static enum resp_states do_complete(struct rxe_qp *qp,
        }
 
        /* have copy for srq and reference for !srq */
-       if (!qp->srq) {
-               if (qp->is_user)
-                       advance_consumer(qp->rq.queue, QUEUE_TYPE_FROM_USER);
-               else
-                       advance_consumer(qp->rq.queue, QUEUE_TYPE_KERNEL);
-       }
+       if (!qp->srq)
+               advance_consumer(qp->rq.queue, QUEUE_TYPE_FROM_CLIENT);
 
        qp->resp.wqe = NULL;
 
drivers/infiniband/sw/rxe/rxe_srq.c +1 -1
index 610c98d24b5c..a9e7817e2732 100644
@@ -93,7 +93,7 @@ int rxe_srq_from_init(struct rxe_dev *rxe, struct rxe_srq *srq,
        spin_lock_init(&srq->rq.producer_lock);
        spin_lock_init(&srq->rq.consumer_lock);
 
-       type = uresp ? QUEUE_TYPE_FROM_USER : QUEUE_TYPE_KERNEL;
+       type = QUEUE_TYPE_FROM_CLIENT;
        q = rxe_queue_init(rxe, &srq->rq.max_wr,
                        srq_wqe_size, type);
        if (!q) {
drivers/infiniband/sw/rxe/rxe_verbs.c +10 -44
index 267b5a9c345d..dc70e3edeba6 100644
@@ -218,11 +218,7 @@ static int post_one_recv(struct rxe_rq *rq, const struct ib_recv_wr *ibwr)
        int num_sge = ibwr->num_sge;
        int full;
 
-       if (rq->is_user)
-               full = queue_full(rq->queue, QUEUE_TYPE_FROM_USER);
-       else
-               full = queue_full(rq->queue, QUEUE_TYPE_KERNEL);
-
+       full = queue_full(rq->queue, QUEUE_TYPE_FROM_CLIENT);
        if (unlikely(full)) {
                err = -ENOMEM;
                goto err1;
@@ -237,11 +233,7 @@ static int post_one_recv(struct rxe_rq *rq, const struct ib_recv_wr *ibwr)
        for (i = 0; i < num_sge; i++)
                length += ibwr->sg_list[i].length;
 
-       if (rq->is_user)
-               recv_wqe = producer_addr(rq->queue, QUEUE_TYPE_FROM_USER);
-       else
-               recv_wqe = producer_addr(rq->queue, QUEUE_TYPE_KERNEL);
-
+       recv_wqe = producer_addr(rq->queue, QUEUE_TYPE_FROM_CLIENT);
        recv_wqe->wr_id = ibwr->wr_id;
        recv_wqe->num_sge = num_sge;
 
@@ -254,10 +246,7 @@ static int post_one_recv(struct rxe_rq *rq, const struct ib_recv_wr *ibwr)
        recv_wqe->dma.cur_sge           = 0;
        recv_wqe->dma.sge_offset        = 0;
 
-       if (rq->is_user)
-               advance_producer(rq->queue, QUEUE_TYPE_FROM_USER);
-       else
-               advance_producer(rq->queue, QUEUE_TYPE_KERNEL);
+       advance_producer(rq->queue, QUEUE_TYPE_FROM_CLIENT);
 
        return 0;
 
@@ -633,27 +622,17 @@ static int post_one_send(struct rxe_qp *qp, const struct ib_send_wr *ibwr,
 
        spin_lock_irqsave(&qp->sq.sq_lock, flags);
 
-       if (qp->is_user)
-               full = queue_full(sq->queue, QUEUE_TYPE_FROM_USER);
-       else
-               full = queue_full(sq->queue, QUEUE_TYPE_KERNEL);
+       full = queue_full(sq->queue, QUEUE_TYPE_FROM_CLIENT);
 
        if (unlikely(full)) {
                spin_unlock_irqrestore(&qp->sq.sq_lock, flags);
                return -ENOMEM;
        }
 
-       if (qp->is_user)
-               send_wqe = producer_addr(sq->queue, QUEUE_TYPE_FROM_USER);
-       else
-               send_wqe = producer_addr(sq->queue, QUEUE_TYPE_KERNEL);
-
+       send_wqe = producer_addr(sq->queue, QUEUE_TYPE_FROM_CLIENT);
        init_send_wqe(qp, ibwr, mask, length, send_wqe);
 
-       if (qp->is_user)
-               advance_producer(sq->queue, QUEUE_TYPE_FROM_USER);
-       else
-               advance_producer(sq->queue, QUEUE_TYPE_KERNEL);
+       advance_producer(sq->queue, QUEUE_TYPE_FROM_CLIENT);
 
        spin_unlock_irqrestore(&qp->sq.sq_lock, flags);
 
@@ -845,18 +824,12 @@ static int rxe_poll_cq(struct ib_cq *ibcq, int num_entries, struct ib_wc *wc)
 
        spin_lock_irqsave(&cq->cq_lock, flags);
        for (i = 0; i < num_entries; i++) {
-               if (cq->is_user)
-                       cqe = queue_head(cq->queue, QUEUE_TYPE_TO_USER);
-               else
-                       cqe = queue_head(cq->queue, QUEUE_TYPE_KERNEL);
+               cqe = queue_head(cq->queue, QUEUE_TYPE_TO_CLIENT);
                if (!cqe)
                        break;
 
                memcpy(wc++, &cqe->ibwc, sizeof(*wc));
-               if (cq->is_user)
-                       advance_consumer(cq->queue, QUEUE_TYPE_TO_USER);
-               else
-                       advance_consumer(cq->queue, QUEUE_TYPE_KERNEL);
+               advance_consumer(cq->queue, QUEUE_TYPE_TO_CLIENT);
        }
        spin_unlock_irqrestore(&cq->cq_lock, flags);
 
@@ -868,10 +841,7 @@ static int rxe_peek_cq(struct ib_cq *ibcq, int wc_cnt)
        struct rxe_cq *cq = to_rcq(ibcq);
        int count;
 
-       if (cq->is_user)
-               count = queue_count(cq->queue, QUEUE_TYPE_TO_USER);
-       else
-               count = queue_count(cq->queue, QUEUE_TYPE_KERNEL);
+       count = queue_count(cq->queue, QUEUE_TYPE_TO_CLIENT);
 
        return (count > wc_cnt) ? wc_cnt : count;
 }
@@ -887,10 +857,7 @@ static int rxe_req_notify_cq(struct ib_cq *ibcq, enum ib_cq_notify_flags flags)
        if (cq->notify != IB_CQ_NEXT_COMP)
                cq->notify = flags & IB_CQ_SOLICITED_MASK;
 
-       if (cq->is_user)
-               empty = queue_empty(cq->queue, QUEUE_TYPE_TO_USER);
-       else
-               empty = queue_empty(cq->queue, QUEUE_TYPE_KERNEL);
+       empty = queue_empty(cq->queue, QUEUE_TYPE_TO_CLIENT);
 
        if ((flags & IB_CQ_REPORT_MISSED_EVENTS) && !empty)
                ret = 1;
--

2.30.2

#3 lore raw

Re: [PATCH for-rc v3 1/6] RDMA/rxe: Add memory barriers to kernel queues

Quoted 613 lines · Zhu Yanjun
Quoted 26 lines · Bob Pearson

Earlier patches added memory barriers to protect user space to kernel space communications. The user space queues were previously shown to have occasional memory synchonization errors which were removed by adding smp_load_acquire, smp_store_release barriers.

This patch extends that to the case where queues are used between kernel space threads.

---
 drivers/infiniband/sw/rxe/rxe_comp.c  | 10 +---
 drivers/infiniband/sw/rxe/rxe_cq.c    | 25 ++-------
 drivers/infiniband/sw/rxe/rxe_qp.c    | 10 ++--
 drivers/infiniband/sw/rxe/rxe_queue.h | 73 ++++++++-------------------
 drivers/infiniband/sw/rxe/rxe_req.c   | 21 ++------
 drivers/infiniband/sw/rxe/rxe_resp.c  | 38 ++++----------
 drivers/infiniband/sw/rxe/rxe_srq.c   |  2 +-
 drivers/infiniband/sw/rxe/rxe_verbs.c | 53 ++++---------------
 8 files changed, 55 insertions(+), 177 deletions(-)
drivers/infiniband/sw/rxe/rxe_comp.c +0 -0
index d2d802c776fd..ed4e3f29bd65 100644

drivers/infiniband/sw/rxe/rxe_comp.c static inline enum comp_state get_wqe(struct rxe_qp *qp, · line 148

Quoted 7 lines
drivers/infiniband/sw/rxe/rxe_comp.c +0 -4
@@ -142,10 +142,7 @@ static inline enum comp_state get_wqe(struct rxe_qp *qp,
        /* we come here whether or not we found a response packet to see if
         * there are any posted WQEs
         */
-       if (qp->is_user)
-               wqe = queue_head(qp->sq.queue, QUEUE_TYPE_FROM_USER);
-       else
-               wqe = queue_head(qp->sq.queue, QUEUE_TYPE_KERNEL);

This commit is very similar to the commit in
https://lore.kernel.org/linux-rdma/[email protected]/T/

Zhu Yanjun

Quoted 573 lines
drivers/infiniband/sw/rxe/rxe_comp.c +2 -4
+       wqe = queue_head(qp->sq.queue, QUEUE_TYPE_FROM_CLIENT);
        *wqe_p = wqe;
 
        /* no WQE or requester has not started it yet */
@@ -432,10 +429,7 @@ static void do_complete(struct rxe_qp *qp, struct rxe_send_wqe *wqe)
        if (post)
                make_send_cqe(qp, wqe, &cqe);
 
-       if (qp->is_user)
-               advance_consumer(qp->sq.queue, QUEUE_TYPE_FROM_USER);
-       else
-               advance_consumer(qp->sq.queue, QUEUE_TYPE_KERNEL);
+       advance_consumer(qp->sq.queue, QUEUE_TYPE_FROM_CLIENT);
 
        if (post)
                rxe_cq_post(qp->scq, &cqe, 0);
drivers/infiniband/sw/rxe/rxe_cq.c +5 -20
index aef288f164fd..4e26c2ea4a59 100644
@@ -25,11 +25,7 @@ int rxe_cq_chk_attr(struct rxe_dev *rxe, struct rxe_cq *cq,
        }
 
        if (cq) {
-               if (cq->is_user)
-                       count = queue_count(cq->queue, QUEUE_TYPE_TO_USER);
-               else
-                       count = queue_count(cq->queue, QUEUE_TYPE_KERNEL);
-
+               count = queue_count(cq->queue, QUEUE_TYPE_TO_CLIENT);
                if (cqe < count) {
                        pr_warn("cqe(%d) < current # elements in queue (%d)",
                                cqe, count);
@@ -65,7 +61,7 @@ int rxe_cq_from_init(struct rxe_dev *rxe, struct rxe_cq *cq, int cqe,
        int err;
        enum queue_type type;
 
-       type = uresp ? QUEUE_TYPE_TO_USER : QUEUE_TYPE_KERNEL;
+       type = QUEUE_TYPE_TO_CLIENT;
        cq->queue = rxe_queue_init(rxe, &cqe,
                        sizeof(struct rxe_cqe), type);
        if (!cq->queue) {
@@ -117,11 +113,7 @@ int rxe_cq_post(struct rxe_cq *cq, struct rxe_cqe *cqe, int solicited)
 
        spin_lock_irqsave(&cq->cq_lock, flags);
 
-       if (cq->is_user)
-               full = queue_full(cq->queue, QUEUE_TYPE_TO_USER);
-       else
-               full = queue_full(cq->queue, QUEUE_TYPE_KERNEL);
-
+       full = queue_full(cq->queue, QUEUE_TYPE_TO_CLIENT);
        if (unlikely(full)) {
                spin_unlock_irqrestore(&cq->cq_lock, flags);
                if (cq->ibcq.event_handler) {
@@ -134,17 +126,10 @@ int rxe_cq_post(struct rxe_cq *cq, struct rxe_cqe *cqe, int solicited)
                return -EBUSY;
        }
 
-       if (cq->is_user)
-               addr = producer_addr(cq->queue, QUEUE_TYPE_TO_USER);
-       else
-               addr = producer_addr(cq->queue, QUEUE_TYPE_KERNEL);
-
+       addr = producer_addr(cq->queue, QUEUE_TYPE_TO_CLIENT);
        memcpy(addr, cqe, sizeof(*cqe));
 
-       if (cq->is_user)
-               advance_producer(cq->queue, QUEUE_TYPE_TO_USER);
-       else
-               advance_producer(cq->queue, QUEUE_TYPE_KERNEL);
+       advance_producer(cq->queue, QUEUE_TYPE_TO_CLIENT);
 
        spin_unlock_irqrestore(&cq->cq_lock, flags);
 
drivers/infiniband/sw/rxe/rxe_qp.c +3 -7
index 1ab6af7ddb25..2e923af642f8 100644
@@ -231,7 +231,7 @@ static int rxe_qp_init_req(struct rxe_dev *rxe, struct rxe_qp *qp,
        qp->sq.max_inline = init->cap.max_inline_data = wqe_size;
        wqe_size += sizeof(struct rxe_send_wqe);
 
-       type = uresp ? QUEUE_TYPE_FROM_USER : QUEUE_TYPE_KERNEL;
+       type = QUEUE_TYPE_FROM_CLIENT;
        qp->sq.queue = rxe_queue_init(rxe, &qp->sq.max_wr,
                                wqe_size, type);
        if (!qp->sq.queue)
@@ -248,12 +248,8 @@ static int rxe_qp_init_req(struct rxe_dev *rxe, struct rxe_qp *qp,
                return err;
        }
 
-       if (qp->is_user)
                qp->req.wqe_index = producer_index(qp->sq.queue,
-                                               QUEUE_TYPE_FROM_USER);
-       else
-               qp->req.wqe_index = producer_index(qp->sq.queue,
-                                               QUEUE_TYPE_KERNEL);
+                                       QUEUE_TYPE_FROM_CLIENT);
 
        qp->req.state           = QP_STATE_RESET;
        qp->req.opcode          = -1;
@@ -293,7 +289,7 @@ static int rxe_qp_init_resp(struct rxe_dev *rxe, struct rxe_qp *qp,
                pr_debug("qp#%d max_wr = %d, max_sge = %d, wqe_size = %d\n",
                         qp_num(qp), qp->rq.max_wr, qp->rq.max_sge, wqe_size);
 
-               type = uresp ? QUEUE_TYPE_FROM_USER : QUEUE_TYPE_KERNEL;
+               type = QUEUE_TYPE_FROM_CLIENT;
                qp->rq.queue = rxe_queue_init(rxe, &qp->rq.max_wr,
                                        wqe_size, type);
                if (!qp->rq.queue)
drivers/infiniband/sw/rxe/rxe_queue.h +20 -53
index 2702b0e55fc3..d465aa9342e1 100644
@@ -35,9 +35,8 @@
 
 /* type of queue */
 enum queue_type {
-       QUEUE_TYPE_KERNEL,
-       QUEUE_TYPE_TO_USER,
-       QUEUE_TYPE_FROM_USER,
+       QUEUE_TYPE_TO_CLIENT,
+       QUEUE_TYPE_FROM_CLIENT,
 };
 
 struct rxe_queue {
@@ -87,20 +86,16 @@ static inline int queue_empty(struct rxe_queue *q, enum queue_type type)
        u32 cons;
 
        switch (type) {
-       case QUEUE_TYPE_FROM_USER:
+       case QUEUE_TYPE_FROM_CLIENT:
                /* protect user space index */
                prod = smp_load_acquire(&q->buf->producer_index);
                cons = q->index;
                break;
-       case QUEUE_TYPE_TO_USER:
+       case QUEUE_TYPE_TO_CLIENT:
                prod = q->index;
                /* protect user space index */
                cons = smp_load_acquire(&q->buf->consumer_index);
                break;
-       case QUEUE_TYPE_KERNEL:
-               prod = q->buf->producer_index;
-               cons = q->buf->consumer_index;
-               break;
        }
 
        return ((prod - cons) & q->index_mask) == 0;
@@ -112,20 +107,16 @@ static inline int queue_full(struct rxe_queue *q, enum queue_type type)
        u32 cons;
 
        switch (type) {
-       case QUEUE_TYPE_FROM_USER:
+       case QUEUE_TYPE_FROM_CLIENT:
                /* protect user space index */
                prod = smp_load_acquire(&q->buf->producer_index);
                cons = q->index;
                break;
-       case QUEUE_TYPE_TO_USER:
+       case QUEUE_TYPE_TO_CLIENT:
                prod = q->index;
                /* protect user space index */
                cons = smp_load_acquire(&q->buf->consumer_index);
                break;
-       case QUEUE_TYPE_KERNEL:
-               prod = q->buf->producer_index;
-               cons = q->buf->consumer_index;
-               break;
        }
 
        return ((prod + 1 - cons) & q->index_mask) == 0;
@@ -138,20 +129,16 @@ static inline unsigned int queue_count(const struct rxe_queue *q,
        u32 cons;
 
        switch (type) {
-       case QUEUE_TYPE_FROM_USER:
+       case QUEUE_TYPE_FROM_CLIENT:
                /* protect user space index */
                prod = smp_load_acquire(&q->buf->producer_index);
                cons = q->index;
                break;
-       case QUEUE_TYPE_TO_USER:
+       case QUEUE_TYPE_TO_CLIENT:
                prod = q->index;
                /* protect user space index */
                cons = smp_load_acquire(&q->buf->consumer_index);
                break;
-       case QUEUE_TYPE_KERNEL:
-               prod = q->buf->producer_index;
-               cons = q->buf->consumer_index;
-               break;
        }
 
        return (prod - cons) & q->index_mask;
@@ -162,7 +149,7 @@ static inline void advance_producer(struct rxe_queue *q, enum queue_type type)
        u32 prod;
 
        switch (type) {
-       case QUEUE_TYPE_FROM_USER:
+       case QUEUE_TYPE_FROM_CLIENT:
                pr_warn_once("Normally kernel should not write user space index\n");
                /* protect user space index */
                prod = smp_load_acquire(&q->buf->producer_index);
@@ -170,15 +157,11 @@ static inline void advance_producer(struct rxe_queue *q, enum queue_type type)
                /* same */
                smp_store_release(&q->buf->producer_index, prod);
                break;
-       case QUEUE_TYPE_TO_USER:
+       case QUEUE_TYPE_TO_CLIENT:
                prod = q->index;
                q->index = (prod + 1) & q->index_mask;
                q->buf->producer_index = q->index;
                break;
-       case QUEUE_TYPE_KERNEL:
-               prod = q->buf->producer_index;
-               q->buf->producer_index = (prod + 1) & q->index_mask;
-               break;
        }
 }
 
@@ -187,12 +170,12 @@ static inline void advance_consumer(struct rxe_queue *q, enum queue_type type)
        u32 cons;
 
        switch (type) {
-       case QUEUE_TYPE_FROM_USER:
+       case QUEUE_TYPE_FROM_CLIENT:
                cons = q->index;
                q->index = (cons + 1) & q->index_mask;
                q->buf->consumer_index = q->index;
                break;
-       case QUEUE_TYPE_TO_USER:
+       case QUEUE_TYPE_TO_CLIENT:
                pr_warn_once("Normally kernel should not write user space index\n");
                /* protect user space index */
                cons = smp_load_acquire(&q->buf->consumer_index);
@@ -200,10 +183,6 @@ static inline void advance_consumer(struct rxe_queue *q, enum queue_type type)
                /* same */
                smp_store_release(&q->buf->consumer_index, cons);
                break;
-       case QUEUE_TYPE_KERNEL:
-               cons = q->buf->consumer_index;
-               q->buf->consumer_index = (cons + 1) & q->index_mask;
-               break;
        }
 }
 
@@ -212,17 +191,14 @@ static inline void *producer_addr(struct rxe_queue *q, enum queue_type type)
        u32 prod;
 
        switch (type) {
-       case QUEUE_TYPE_FROM_USER:
+       case QUEUE_TYPE_FROM_CLIENT:
                /* protect user space index */
                prod = smp_load_acquire(&q->buf->producer_index);
                prod &= q->index_mask;
                break;
-       case QUEUE_TYPE_TO_USER:
+       case QUEUE_TYPE_TO_CLIENT:
                prod = q->index;
                break;
-       case QUEUE_TYPE_KERNEL:
-               prod = q->buf->producer_index;
-               break;
        }
 
        return q->buf->data + (prod << q->log2_elem_size);
@@ -233,17 +209,14 @@ static inline void *consumer_addr(struct rxe_queue *q, enum queue_type type)
        u32 cons;
 
        switch (type) {
-       case QUEUE_TYPE_FROM_USER:
+       case QUEUE_TYPE_FROM_CLIENT:
                cons = q->index;
                break;
-       case QUEUE_TYPE_TO_USER:
+       case QUEUE_TYPE_TO_CLIENT:
                /* protect user space index */
                cons = smp_load_acquire(&q->buf->consumer_index);
                cons &= q->index_mask;
                break;
-       case QUEUE_TYPE_KERNEL:
-               cons = q->buf->consumer_index;
-               break;
        }
 
        return q->buf->data + (cons << q->log2_elem_size);
@@ -255,17 +228,14 @@ static inline unsigned int producer_index(struct rxe_queue *q,
        u32 prod;
 
        switch (type) {
-       case QUEUE_TYPE_FROM_USER:
+       case QUEUE_TYPE_FROM_CLIENT:
                /* protect user space index */
                prod = smp_load_acquire(&q->buf->producer_index);
                prod &= q->index_mask;
                break;
-       case QUEUE_TYPE_TO_USER:
+       case QUEUE_TYPE_TO_CLIENT:
                prod = q->index;
                break;
-       case QUEUE_TYPE_KERNEL:
-               prod = q->buf->producer_index;
-               break;
        }
 
        return prod;
@@ -277,17 +247,14 @@ static inline unsigned int consumer_index(struct rxe_queue *q,
        u32 cons;
 
        switch (type) {
-       case QUEUE_TYPE_FROM_USER:
+       case QUEUE_TYPE_FROM_CLIENT:
                cons = q->index;
                break;
-       case QUEUE_TYPE_TO_USER:
+       case QUEUE_TYPE_TO_CLIENT:
                /* protect user space index */
                cons = smp_load_acquire(&q->buf->consumer_index);
                cons &= q->index_mask;
                break;
-       case QUEUE_TYPE_KERNEL:
-               cons = q->buf->consumer_index;
-               break;
        }
 
        return cons;
drivers/infiniband/sw/rxe/rxe_req.c +5 -16
index 3894197a82f6..22c3edb28945 100644
@@ -49,13 +49,8 @@ static void req_retry(struct rxe_qp *qp)
        unsigned int cons;
        unsigned int prod;
 
-       if (qp->is_user) {
-               cons = consumer_index(q, QUEUE_TYPE_FROM_USER);
-               prod = producer_index(q, QUEUE_TYPE_FROM_USER);
-       } else {
-               cons = consumer_index(q, QUEUE_TYPE_KERNEL);
-               prod = producer_index(q, QUEUE_TYPE_KERNEL);
-       }
+       cons = consumer_index(q, QUEUE_TYPE_FROM_CLIENT);
+       prod = producer_index(q, QUEUE_TYPE_FROM_CLIENT);
 
        qp->req.wqe_index       = cons;
        qp->req.psn             = qp->comp.psn;
@@ -121,15 +116,9 @@ static struct rxe_send_wqe *req_next_wqe(struct rxe_qp *qp)
        unsigned int cons;
        unsigned int prod;
 
-       if (qp->is_user) {
-               wqe = queue_head(q, QUEUE_TYPE_FROM_USER);
-               cons = consumer_index(q, QUEUE_TYPE_FROM_USER);
-               prod = producer_index(q, QUEUE_TYPE_FROM_USER);
-       } else {
-               wqe = queue_head(q, QUEUE_TYPE_KERNEL);
-               cons = consumer_index(q, QUEUE_TYPE_KERNEL);
-               prod = producer_index(q, QUEUE_TYPE_KERNEL);
-       }
+       wqe = queue_head(q, QUEUE_TYPE_FROM_CLIENT);
+       cons = consumer_index(q, QUEUE_TYPE_FROM_CLIENT);
+       prod = producer_index(q, QUEUE_TYPE_FROM_CLIENT);
 
        if (unlikely(qp->req.state == QP_STATE_DRAIN)) {
                /* check to see if we are drained;
drivers/infiniband/sw/rxe/rxe_resp.c +9 -29
index 5501227ddc65..596be002d33d 100644
@@ -303,10 +303,7 @@ static enum resp_states get_srq_wqe(struct rxe_qp *qp)
 
        spin_lock_bh(&srq->rq.consumer_lock);
 
-       if (qp->is_user)
-               wqe = queue_head(q, QUEUE_TYPE_FROM_USER);
-       else
-               wqe = queue_head(q, QUEUE_TYPE_KERNEL);
+       wqe = queue_head(q, QUEUE_TYPE_FROM_CLIENT);
        if (!wqe) {
                spin_unlock_bh(&srq->rq.consumer_lock);
                return RESPST_ERR_RNR;
@@ -322,13 +319,8 @@ static enum resp_states get_srq_wqe(struct rxe_qp *qp)
        memcpy(&qp->resp.srq_wqe, wqe, size);
 
        qp->resp.wqe = &qp->resp.srq_wqe.wqe;
-       if (qp->is_user) {
-               advance_consumer(q, QUEUE_TYPE_FROM_USER);
-               count = queue_count(q, QUEUE_TYPE_FROM_USER);
-       } else {
-               advance_consumer(q, QUEUE_TYPE_KERNEL);
-               count = queue_count(q, QUEUE_TYPE_KERNEL);
-       }
+       advance_consumer(q, QUEUE_TYPE_FROM_CLIENT);
+       count = queue_count(q, QUEUE_TYPE_FROM_CLIENT);
 
        if (srq->limit && srq->ibsrq.event_handler && (count < srq->limit)) {
                srq->limit = 0;
@@ -357,12 +349,8 @@ static enum resp_states check_resource(struct rxe_qp *qp,
                        qp->resp.status = IB_WC_WR_FLUSH_ERR;
                        return RESPST_COMPLETE;
                } else if (!srq) {
-                       if (qp->is_user)
-                               qp->resp.wqe = queue_head(qp->rq.queue,
-                                               QUEUE_TYPE_FROM_USER);
-                       else
-                               qp->resp.wqe = queue_head(qp->rq.queue,
-                                               QUEUE_TYPE_KERNEL);
+                       qp->resp.wqe = queue_head(qp->rq.queue,
+                                       QUEUE_TYPE_FROM_CLIENT);
                        if (qp->resp.wqe) {
                                qp->resp.status = IB_WC_WR_FLUSH_ERR;
                                return RESPST_COMPLETE;
@@ -389,12 +377,8 @@ static enum resp_states check_resource(struct rxe_qp *qp,
                if (srq)
                        return get_srq_wqe(qp);
 
-               if (qp->is_user)
-                       qp->resp.wqe = queue_head(qp->rq.queue,
-                                       QUEUE_TYPE_FROM_USER);
-               else
-                       qp->resp.wqe = queue_head(qp->rq.queue,
-                                       QUEUE_TYPE_KERNEL);
+               qp->resp.wqe = queue_head(qp->rq.queue,
+                               QUEUE_TYPE_FROM_CLIENT);
                return (qp->resp.wqe) ? RESPST_CHK_LENGTH : RESPST_ERR_RNR;
        }
 
@@ -936,12 +920,8 @@ static enum resp_states do_complete(struct rxe_qp *qp,
        }
 
        /* have copy for srq and reference for !srq */
-       if (!qp->srq) {
-               if (qp->is_user)
-                       advance_consumer(qp->rq.queue, QUEUE_TYPE_FROM_USER);
-               else
-                       advance_consumer(qp->rq.queue, QUEUE_TYPE_KERNEL);
-       }
+       if (!qp->srq)
+               advance_consumer(qp->rq.queue, QUEUE_TYPE_FROM_CLIENT);
 
        qp->resp.wqe = NULL;
 
drivers/infiniband/sw/rxe/rxe_srq.c +1 -1
index 610c98d24b5c..a9e7817e2732 100644
@@ -93,7 +93,7 @@ int rxe_srq_from_init(struct rxe_dev *rxe, struct rxe_srq *srq,
        spin_lock_init(&srq->rq.producer_lock);
        spin_lock_init(&srq->rq.consumer_lock);
 
-       type = uresp ? QUEUE_TYPE_FROM_USER : QUEUE_TYPE_KERNEL;
+       type = QUEUE_TYPE_FROM_CLIENT;
        q = rxe_queue_init(rxe, &srq->rq.max_wr,
                        srq_wqe_size, type);
        if (!q) {
drivers/infiniband/sw/rxe/rxe_verbs.c +10 -43
index 267b5a9c345d..dc70e3edeba6 100644
@@ -218,11 +218,7 @@ static int post_one_recv(struct rxe_rq *rq, const struct ib_recv_wr *ibwr)
        int num_sge = ibwr->num_sge;
        int full;
 
-       if (rq->is_user)
-               full = queue_full(rq->queue, QUEUE_TYPE_FROM_USER);
-       else
-               full = queue_full(rq->queue, QUEUE_TYPE_KERNEL);
-
+       full = queue_full(rq->queue, QUEUE_TYPE_FROM_CLIENT);
        if (unlikely(full)) {
                err = -ENOMEM;
                goto err1;
@@ -237,11 +233,7 @@ static int post_one_recv(struct rxe_rq *rq, const struct ib_recv_wr *ibwr)
        for (i = 0; i < num_sge; i++)
                length += ibwr->sg_list[i].length;
 
-       if (rq->is_user)
-               recv_wqe = producer_addr(rq->queue, QUEUE_TYPE_FROM_USER);
-       else
-               recv_wqe = producer_addr(rq->queue, QUEUE_TYPE_KERNEL);
-
+       recv_wqe = producer_addr(rq->queue, QUEUE_TYPE_FROM_CLIENT);
        recv_wqe->wr_id = ibwr->wr_id;
        recv_wqe->num_sge = num_sge;
 
@@ -254,10 +246,7 @@ static int post_one_recv(struct rxe_rq *rq, const struct ib_recv_wr *ibwr)
        recv_wqe->dma.cur_sge           = 0;
        recv_wqe->dma.sge_offset        = 0;
 
-       if (rq->is_user)
-               advance_producer(rq->queue, QUEUE_TYPE_FROM_USER);
-       else
-               advance_producer(rq->queue, QUEUE_TYPE_KERNEL);
+       advance_producer(rq->queue, QUEUE_TYPE_FROM_CLIENT);
 
        return 0;
 
@@ -633,27 +622,17 @@ static int post_one_send(struct rxe_qp *qp, const struct ib_send_wr *ibwr,
 
        spin_lock_irqsave(&qp->sq.sq_lock, flags);
 
-       if (qp->is_user)
-               full = queue_full(sq->queue, QUEUE_TYPE_FROM_USER);
-       else
-               full = queue_full(sq->queue, QUEUE_TYPE_KERNEL);
+       full = queue_full(sq->queue, QUEUE_TYPE_FROM_CLIENT);
 
        if (unlikely(full)) {
                spin_unlock_irqrestore(&qp->sq.sq_lock, flags);
                return -ENOMEM;
        }
 
-       if (qp->is_user)
-               send_wqe = producer_addr(sq->queue, QUEUE_TYPE_FROM_USER);
-       else
-               send_wqe = producer_addr(sq->queue, QUEUE_TYPE_KERNEL);
-
+       send_wqe = producer_addr(sq->queue, QUEUE_TYPE_FROM_CLIENT);
        init_send_wqe(qp, ibwr, mask, length, send_wqe);
 
-       if (qp->is_user)
-               advance_producer(sq->queue, QUEUE_TYPE_FROM_USER);
-       else
-               advance_producer(sq->queue, QUEUE_TYPE_KERNEL);
+       advance_producer(sq->queue, QUEUE_TYPE_FROM_CLIENT);
 
        spin_unlock_irqrestore(&qp->sq.sq_lock, flags);
 
@@ -845,18 +824,12 @@ static int rxe_poll_cq(struct ib_cq *ibcq, int num_entries, struct ib_wc *wc)
 
        spin_lock_irqsave(&cq->cq_lock, flags);
        for (i = 0; i < num_entries; i++) {
-               if (cq->is_user)
-                       cqe = queue_head(cq->queue, QUEUE_TYPE_TO_USER);
-               else
-                       cqe = queue_head(cq->queue, QUEUE_TYPE_KERNEL);
+               cqe = queue_head(cq->queue, QUEUE_TYPE_TO_CLIENT);
                if (!cqe)
                        break;
 
                memcpy(wc++, &cqe->ibwc, sizeof(*wc));
-               if (cq->is_user)
-                       advance_consumer(cq->queue, QUEUE_TYPE_TO_USER);
-               else
-                       advance_consumer(cq->queue, QUEUE_TYPE_KERNEL);
+               advance_consumer(cq->queue, QUEUE_TYPE_TO_CLIENT);
        }
        spin_unlock_irqrestore(&cq->cq_lock, flags);
 
@@ -868,10 +841,7 @@ static int rxe_peek_cq(struct ib_cq *ibcq, int wc_cnt)
        struct rxe_cq *cq = to_rcq(ibcq);
        int count;
 
-       if (cq->is_user)
-               count = queue_count(cq->queue, QUEUE_TYPE_TO_USER);
-       else
-               count = queue_count(cq->queue, QUEUE_TYPE_KERNEL);
+       count = queue_count(cq->queue, QUEUE_TYPE_TO_CLIENT);
 
        return (count > wc_cnt) ? wc_cnt : count;
 }
@@ -887,10 +857,7 @@ static int rxe_req_notify_cq(struct ib_cq *ibcq, enum ib_cq_notify_flags flags)
        if (cq->notify != IB_CQ_NEXT_COMP)
                cq->notify = flags & IB_CQ_SOLICITED_MASK;
 
-       if (cq->is_user)
-               empty = queue_empty(cq->queue, QUEUE_TYPE_TO_USER);
-       else
-               empty = queue_empty(cq->queue, QUEUE_TYPE_KERNEL);
+       empty = queue_empty(cq->queue, QUEUE_TYPE_TO_CLIENT);
 
Quoted 5 lines
Quoted 5 lines
drivers/infiniband/sw/rxe/rxe_verbs.c +0 -1
        if ((flags & IB_CQ_REPORT_MISSED_EVENTS) && !empty)
                ret = 1;
--

2.30.2

It's the same one. In the cover letter that is called out. It still needs to get done.

Bob

#4 lore raw

回复: [PATCH for-rc v3 1/6] RDMA/rxe: Add memory barriers to kernel queues

In reply to Bob Pearson

Hi Bob,

Why do you want to use FROM_CLIENT and TO_CLIENT suffix?
It seems readable to use FROM_USER and TO_USER suffix (i.e. between user space and kernel space).

Best Regards,
Xiao Yang

-----邮件原件-----
发件人: Bob Pearson <[email protected]>
发送时间: 2021年9月10日 4:45
收件人: [email protected]; [email protected]; [email protected]; [email protected]; [email protected] 抄送: Bob Pearson <[email protected]>
主题: [PATCH for-rc v3 1/6] RDMA/rxe: Add memory barriers to kernel queues

Earlier patches added memory barriers to protect user space to kernel space communications. The user space queues were previously shown to have occasional memory synchonization errors which were removed by adding smp_load_acquire, smp_store_release barriers.

This patch extends that to the case where queues are used between kernel space threads.

---
 drivers/infiniband/sw/rxe/rxe_comp.c  | 10 +---
 drivers/infiniband/sw/rxe/rxe_cq.c    | 25 ++-------
 drivers/infiniband/sw/rxe/rxe_qp.c    | 10 ++--
 drivers/infiniband/sw/rxe/rxe_queue.h | 73 ++++++++-------------------
 drivers/infiniband/sw/rxe/rxe_req.c   | 21 ++------
 drivers/infiniband/sw/rxe/rxe_resp.c  | 38 ++++----------
 drivers/infiniband/sw/rxe/rxe_srq.c   |  2 +-
 drivers/infiniband/sw/rxe/rxe_verbs.c | 53 ++++---------------
 8 files changed, 55 insertions(+), 177 deletions(-)
drivers/infiniband/sw/rxe/rxe_comp.c +1 -4
index d2d802c776fd..ed4e3f29bd65 100644
@@ -142,10 +142,7 @@ static inline enum comp_state get_wqe(struct rxe_qp *qp,
 	/* we come here whether or not we found a response packet to see if
 	 * there are any posted WQEs
 	 */
-	if (qp->is_user)
-		wqe = queue_head(qp->sq.queue, QUEUE_TYPE_FROM_USER);
-	else
-		wqe = queue_head(qp->sq.queue, QUEUE_TYPE_KERNEL);
+	wqe = queue_head(qp->sq.queue, QUEUE_TYPE_FROM_CLIENT);
 	*wqe_p = wqe;
 
 	/* no WQE or requester has not started it yet */ @@ -432,10 +429,7 @@ static void do_complete(struct rxe_qp *qp, struct rxe_send_wqe *wqe)

if (post)
make_send_cqe(qp, wqe, &cqe);

- if (qp->is_user)
- advance_consumer(qp->sq.queue, QUEUE_TYPE_FROM_USER);
- else
- advance_consumer(qp->sq.queue, QUEUE_TYPE_KERNEL);
+ advance_consumer(qp->sq.queue, QUEUE_TYPE_FROM_CLIENT);

if (post)
rxe_cq_post(qp->scq, &cqe, 0);

drivers/infiniband/sw/rxe/rxe_cq.c +5 -20
index aef288f164fd..4e26c2ea4a59 100644
@@ -25,11 +25,7 @@ int rxe_cq_chk_attr(struct rxe_dev *rxe, struct rxe_cq *cq,
 	}
 
 	if (cq) {
-		if (cq->is_user)
-			count = queue_count(cq->queue, QUEUE_TYPE_TO_USER);
-		else
-			count = queue_count(cq->queue, QUEUE_TYPE_KERNEL);
-
+		count = queue_count(cq->queue, QUEUE_TYPE_TO_CLIENT);
 		if (cqe < count) {
 			pr_warn("cqe(%d) < current # elements in queue (%d)",
 				cqe, count);
@@ -65,7 +61,7 @@ int rxe_cq_from_init(struct rxe_dev *rxe, struct rxe_cq *cq, int cqe,
 	int err;
 	enum queue_type type;
 
-	type = uresp ? QUEUE_TYPE_TO_USER : QUEUE_TYPE_KERNEL;
+	type = QUEUE_TYPE_TO_CLIENT;
 	cq->queue = rxe_queue_init(rxe, &cqe,
 			sizeof(struct rxe_cqe), type);
 	if (!cq->queue) {
@@ -117,11 +113,7 @@ int rxe_cq_post(struct rxe_cq *cq, struct rxe_cqe *cqe, int solicited)
 
 	spin_lock_irqsave(&cq->cq_lock, flags);
 
-	if (cq->is_user)
-		full = queue_full(cq->queue, QUEUE_TYPE_TO_USER);
-	else
-		full = queue_full(cq->queue, QUEUE_TYPE_KERNEL);
-
+	full = queue_full(cq->queue, QUEUE_TYPE_TO_CLIENT);
 	if (unlikely(full)) {
 		spin_unlock_irqrestore(&cq->cq_lock, flags);
 		if (cq->ibcq.event_handler) {
@@ -134,17 +126,10 @@ int rxe_cq_post(struct rxe_cq *cq, struct rxe_cqe *cqe, int solicited)
 		return -EBUSY;
 	}
 
-	if (cq->is_user)
-		addr = producer_addr(cq->queue, QUEUE_TYPE_TO_USER);
-	else
-		addr = producer_addr(cq->queue, QUEUE_TYPE_KERNEL);
-
+	addr = producer_addr(cq->queue, QUEUE_TYPE_TO_CLIENT);
 	memcpy(addr, cqe, sizeof(*cqe));
 
-	if (cq->is_user)
-		advance_producer(cq->queue, QUEUE_TYPE_TO_USER);
-	else
-		advance_producer(cq->queue, QUEUE_TYPE_KERNEL);
+	advance_producer(cq->queue, QUEUE_TYPE_TO_CLIENT);
 
 	spin_unlock_irqrestore(&cq->cq_lock, flags);
 
drivers/infiniband/sw/rxe/rxe_qp.c +3 -7
index 1ab6af7ddb25..2e923af642f8 100644
@@ -231,7 +231,7 @@ static int rxe_qp_init_req(struct rxe_dev *rxe, struct rxe_qp *qp,
 	qp->sq.max_inline = init->cap.max_inline_data = wqe_size;
 	wqe_size += sizeof(struct rxe_send_wqe);
 
-	type = uresp ? QUEUE_TYPE_FROM_USER : QUEUE_TYPE_KERNEL;
+	type = QUEUE_TYPE_FROM_CLIENT;
 	qp->sq.queue = rxe_queue_init(rxe, &qp->sq.max_wr,
 				wqe_size, type);
 	if (!qp->sq.queue)
@@ -248,12 +248,8 @@ static int rxe_qp_init_req(struct rxe_dev *rxe, struct rxe_qp *qp,
 		return err;
 	}
 
-	if (qp->is_user)
 		qp->req.wqe_index = producer_index(qp->sq.queue,
-						QUEUE_TYPE_FROM_USER);
-	else
-		qp->req.wqe_index = producer_index(qp->sq.queue,
-						QUEUE_TYPE_KERNEL);
+					QUEUE_TYPE_FROM_CLIENT);
 
 	qp->req.state		= QP_STATE_RESET;
 	qp->req.opcode		= -1;
@@ -293,7 +289,7 @@ static int rxe_qp_init_resp(struct rxe_dev *rxe, struct rxe_qp *qp,
 		pr_debug("qp#%d max_wr = %d, max_sge = %d, wqe_size = %d\n",
 			 qp_num(qp), qp->rq.max_wr, qp->rq.max_sge, wqe_size);
 
-		type = uresp ? QUEUE_TYPE_FROM_USER : QUEUE_TYPE_KERNEL;
+		type = QUEUE_TYPE_FROM_CLIENT;
 		qp->rq.queue = rxe_queue_init(rxe, &qp->rq.max_wr,
 					wqe_size, type);
 		if (!qp->rq.queue)
drivers/infiniband/sw/rxe/rxe_queue.h +4 -9
index 2702b0e55fc3..d465aa9342e1 100644
@@ -35,9 +35,8 @@
 
 /* type of queue */
 enum queue_type {
-	QUEUE_TYPE_KERNEL,
-	QUEUE_TYPE_TO_USER,
-	QUEUE_TYPE_FROM_USER,
+	QUEUE_TYPE_TO_CLIENT,
+	QUEUE_TYPE_FROM_CLIENT,
 };
 
 struct rxe_queue {
@@ -87,20 +86,16 @@ static inline int queue_empty(struct rxe_queue *q, enum queue_type type)
 	u32 cons;
 
 	switch (type) {
-	case QUEUE_TYPE_FROM_USER:
+	case QUEUE_TYPE_FROM_CLIENT:
 		/* protect user space index */
 		prod = smp_load_acquire(&q->buf->producer_index);
 		cons = q->index;
 		break;
-	case QUEUE_TYPE_TO_USER:
+	case QUEUE_TYPE_TO_CLIENT:
 		prod = q->index;
 		/* protect user space index */
 		cons = smp_load_acquire(&q->buf->consumer_index);
 		break;
-	case QUEUE_TYPE_KERNEL:
-		prod = q->buf->producer_index;
-		cons = q->buf->consumer_index;
-		break;
 	}
 
 	return ((prod - cons) & q->index_mask) == 0; @@ -112,20 +107,16 @@ static inline int queue_full(struct rxe_queue *q, enum queue_type type)

u32 cons;

switch (type) {
- case QUEUE_TYPE_FROM_USER:
+ case QUEUE_TYPE_FROM_CLIENT:
/* protect user space index */
prod = smp_load_acquire(&q->buf->producer_index); cons = q->index;
break;
- case QUEUE_TYPE_TO_USER:
+ case QUEUE_TYPE_TO_CLIENT:
prod = q->index;
/* protect user space index */
cons = smp_load_acquire(&q->buf->consumer_index); break;
- case QUEUE_TYPE_KERNEL:
- prod = q->buf->producer_index;
- cons = q->buf->consumer_index;
- break;
}

return ((prod + 1 - cons) & q->index_mask) == 0; @@ -138,20 +129,16 @@ static inline unsigned int queue_count(const struct rxe_queue *q, u32 cons;

switch (type) {
- case QUEUE_TYPE_FROM_USER:
+ case QUEUE_TYPE_FROM_CLIENT:
/* protect user space index */
prod = smp_load_acquire(&q->buf->producer_index); cons = q->index;
break;
- case QUEUE_TYPE_TO_USER:
+ case QUEUE_TYPE_TO_CLIENT:
prod = q->index;
/* protect user space index */
cons = smp_load_acquire(&q->buf->consumer_index); break;
- case QUEUE_TYPE_KERNEL:
- prod = q->buf->producer_index;
- cons = q->buf->consumer_index;
- break;
}

return (prod - cons) & q->index_mask;
@@ -162,7 +149,7 @@ static inline void advance_producer(struct rxe_queue *q, enum queue_type type)
u32 prod;

switch (type) {
- case QUEUE_TYPE_FROM_USER:
+ case QUEUE_TYPE_FROM_CLIENT:
pr_warn_once("Normally kernel should not write user space index\n"); /* protect user space index */
prod = smp_load_acquire(&q->buf->producer_index);
@@ -170,15 +157,11 @@ static inline void advance_producer(struct rxe_queue *q, enum queue_type type)
/* same */
smp_store_release(&q->buf->producer_index, prod);
break;
- case QUEUE_TYPE_TO_USER:
+ case QUEUE_TYPE_TO_CLIENT:
prod = q->index;
q->index = (prod + 1) & q->index_mask;
q->buf->producer_index = q->index;
break;
- case QUEUE_TYPE_KERNEL:
- prod = q->buf->producer_index;
- q->buf->producer_index = (prod + 1) & q->index_mask;
- break;
}
}

@@ -187,12 +170,12 @@ static inline void advance_consumer(struct rxe_queue *q, enum queue_type type)
u32 cons;

switch (type) {
- case QUEUE_TYPE_FROM_USER:
+ case QUEUE_TYPE_FROM_CLIENT:
cons = q->index;
q->index = (cons + 1) & q->index_mask;
q->buf->consumer_index = q->index;
break;
- case QUEUE_TYPE_TO_USER:
+ case QUEUE_TYPE_TO_CLIENT:
pr_warn_once("Normally kernel should not write user space index\n"); /* protect user space index */
cons = smp_load_acquire(&q->buf->consumer_index);
@@ -200,10 +183,6 @@ static inline void advance_consumer(struct rxe_queue *q, enum queue_type type)
/* same */
smp_store_release(&q->buf->consumer_index, cons);
break;
- case QUEUE_TYPE_KERNEL:
- cons = q->buf->consumer_index;
- q->buf->consumer_index = (cons + 1) & q->index_mask;
- break;
}
}

@@ -212,17 +191,14 @@ static inline void *producer_addr(struct rxe_queue *q, enum queue_type type)
u32 prod;

switch (type) {
- case QUEUE_TYPE_FROM_USER:
+ case QUEUE_TYPE_FROM_CLIENT:
/* protect user space index */
prod = smp_load_acquire(&q->buf->producer_index); prod &= q->index_mask;
break;
- case QUEUE_TYPE_TO_USER:
+ case QUEUE_TYPE_TO_CLIENT:
prod = q->index;
break;
- case QUEUE_TYPE_KERNEL:
- prod = q->buf->producer_index;
- break;
}

return q->buf->data + (prod << q->log2_elem_size); @@ -233,17 +209,14 @@ static inline void *consumer_addr(struct rxe_queue *q, enum queue_type type) u32 cons;

switch (type) {
- case QUEUE_TYPE_FROM_USER:
+ case QUEUE_TYPE_FROM_CLIENT:
cons = q->index;
break;
- case QUEUE_TYPE_TO_USER:
+ case QUEUE_TYPE_TO_CLIENT:
/* protect user space index */
cons = smp_load_acquire(&q->buf->consumer_index); cons &= q->index_mask;
break;
- case QUEUE_TYPE_KERNEL:
- cons = q->buf->consumer_index;
- break;
}

return q->buf->data + (cons << q->log2_elem_size); @@ -255,17 +228,14 @@ static inline unsigned int producer_index(struct rxe_queue *q, u32 prod;

switch (type) {
- case QUEUE_TYPE_FROM_USER:
+ case QUEUE_TYPE_FROM_CLIENT:
/* protect user space index */
prod = smp_load_acquire(&q->buf->producer_index); prod &= q->index_mask;
break;
- case QUEUE_TYPE_TO_USER:
+ case QUEUE_TYPE_TO_CLIENT:
prod = q->index;
break;
- case QUEUE_TYPE_KERNEL:
- prod = q->buf->producer_index;
- break;
}

return prod;
@@ -277,17 +247,14 @@ static inline unsigned int consumer_index(struct rxe_queue *q,
u32 cons;

switch (type) {
- case QUEUE_TYPE_FROM_USER:
+ case QUEUE_TYPE_FROM_CLIENT:
cons = q->index;
break;
- case QUEUE_TYPE_TO_USER:
+ case QUEUE_TYPE_TO_CLIENT:
/* protect user space index */
cons = smp_load_acquire(&q->buf->consumer_index); cons &= q->index_mask;
break;
- case QUEUE_TYPE_KERNEL:
- cons = q->buf->consumer_index;
- break;
}

return cons;

drivers/infiniband/sw/rxe/rxe_req.c +5 -16
index 3894197a82f6..22c3edb28945 100644
@@ -49,13 +49,8 @@ static void req_retry(struct rxe_qp *qp)
 	unsigned int cons;
 	unsigned int prod;
 
-	if (qp->is_user) {
-		cons = consumer_index(q, QUEUE_TYPE_FROM_USER);
-		prod = producer_index(q, QUEUE_TYPE_FROM_USER);
-	} else {
-		cons = consumer_index(q, QUEUE_TYPE_KERNEL);
-		prod = producer_index(q, QUEUE_TYPE_KERNEL);
-	}
+	cons = consumer_index(q, QUEUE_TYPE_FROM_CLIENT);
+	prod = producer_index(q, QUEUE_TYPE_FROM_CLIENT);
 
 	qp->req.wqe_index	= cons;
 	qp->req.psn		= qp->comp.psn;
@@ -121,15 +116,9 @@ static struct rxe_send_wqe *req_next_wqe(struct rxe_qp *qp)
 	unsigned int cons;
 	unsigned int prod;
 
-	if (qp->is_user) {
-		wqe = queue_head(q, QUEUE_TYPE_FROM_USER);
-		cons = consumer_index(q, QUEUE_TYPE_FROM_USER);
-		prod = producer_index(q, QUEUE_TYPE_FROM_USER);
-	} else {
-		wqe = queue_head(q, QUEUE_TYPE_KERNEL);
-		cons = consumer_index(q, QUEUE_TYPE_KERNEL);
-		prod = producer_index(q, QUEUE_TYPE_KERNEL);
-	}
+	wqe = queue_head(q, QUEUE_TYPE_FROM_CLIENT);
+	cons = consumer_index(q, QUEUE_TYPE_FROM_CLIENT);
+	prod = producer_index(q, QUEUE_TYPE_FROM_CLIENT);
 
 	if (unlikely(qp->req.state == QP_STATE_DRAIN)) {
 		/* check to see if we are drained;
drivers/infiniband/sw/rxe/rxe_resp.c +9 -29
index 5501227ddc65..596be002d33d 100644
@@ -303,10 +303,7 @@ static enum resp_states get_srq_wqe(struct rxe_qp *qp)
 
 	spin_lock_bh(&srq->rq.consumer_lock);
 
-	if (qp->is_user)
-		wqe = queue_head(q, QUEUE_TYPE_FROM_USER);
-	else
-		wqe = queue_head(q, QUEUE_TYPE_KERNEL);
+	wqe = queue_head(q, QUEUE_TYPE_FROM_CLIENT);
 	if (!wqe) {
 		spin_unlock_bh(&srq->rq.consumer_lock);
 		return RESPST_ERR_RNR;
@@ -322,13 +319,8 @@ static enum resp_states get_srq_wqe(struct rxe_qp *qp)
 	memcpy(&qp->resp.srq_wqe, wqe, size);
 
 	qp->resp.wqe = &qp->resp.srq_wqe.wqe;
-	if (qp->is_user) {
-		advance_consumer(q, QUEUE_TYPE_FROM_USER);
-		count = queue_count(q, QUEUE_TYPE_FROM_USER);
-	} else {
-		advance_consumer(q, QUEUE_TYPE_KERNEL);
-		count = queue_count(q, QUEUE_TYPE_KERNEL);
-	}
+	advance_consumer(q, QUEUE_TYPE_FROM_CLIENT);
+	count = queue_count(q, QUEUE_TYPE_FROM_CLIENT);
 
 	if (srq->limit && srq->ibsrq.event_handler && (count < srq->limit)) {
 		srq->limit = 0;
@@ -357,12 +349,8 @@ static enum resp_states check_resource(struct rxe_qp *qp,
 			qp->resp.status = IB_WC_WR_FLUSH_ERR;
 			return RESPST_COMPLETE;
 		} else if (!srq) {
-			if (qp->is_user)
-				qp->resp.wqe = queue_head(qp->rq.queue,
-						QUEUE_TYPE_FROM_USER);
-			else
-				qp->resp.wqe = queue_head(qp->rq.queue,
-						QUEUE_TYPE_KERNEL);
+			qp->resp.wqe = queue_head(qp->rq.queue,
+					QUEUE_TYPE_FROM_CLIENT);
 			if (qp->resp.wqe) {
 				qp->resp.status = IB_WC_WR_FLUSH_ERR;
 				return RESPST_COMPLETE;
@@ -389,12 +377,8 @@ static enum resp_states check_resource(struct rxe_qp *qp,
 		if (srq)
 			return get_srq_wqe(qp);
 
-		if (qp->is_user)
-			qp->resp.wqe = queue_head(qp->rq.queue,
-					QUEUE_TYPE_FROM_USER);
-		else
-			qp->resp.wqe = queue_head(qp->rq.queue,
-					QUEUE_TYPE_KERNEL);
+		qp->resp.wqe = queue_head(qp->rq.queue,
+				QUEUE_TYPE_FROM_CLIENT);
 		return (qp->resp.wqe) ? RESPST_CHK_LENGTH : RESPST_ERR_RNR;
 	}
 
@@ -936,12 +920,8 @@ static enum resp_states do_complete(struct rxe_qp *qp,
 	}
 
 	/* have copy for srq and reference for !srq */
-	if (!qp->srq) {
-		if (qp->is_user)
-			advance_consumer(qp->rq.queue, QUEUE_TYPE_FROM_USER);
-		else
-			advance_consumer(qp->rq.queue, QUEUE_TYPE_KERNEL);
-	}
+	if (!qp->srq)
+		advance_consumer(qp->rq.queue, QUEUE_TYPE_FROM_CLIENT);
 
 	qp->resp.wqe = NULL;
 
drivers/infiniband/sw/rxe/rxe_srq.c +1 -1
index 610c98d24b5c..a9e7817e2732 100644
@@ -93,7 +93,7 @@ int rxe_srq_from_init(struct rxe_dev *rxe, struct rxe_srq *srq,
 	spin_lock_init(&srq->rq.producer_lock);
 	spin_lock_init(&srq->rq.consumer_lock);
 
-	type = uresp ? QUEUE_TYPE_FROM_USER : QUEUE_TYPE_KERNEL;
+	type = QUEUE_TYPE_FROM_CLIENT;
 	q = rxe_queue_init(rxe, &srq->rq.max_wr,
 			srq_wqe_size, type);
 	if (!q) {
drivers/infiniband/sw/rxe/rxe_verbs.c +9 -39

96 lines not on this page · raw message

cq->notify = flags & IB_CQ_SOLICITED_MASK;

- if (cq->is_user)
- empty = queue_empty(cq->queue, QUEUE_TYPE_TO_USER);
- else
- empty = queue_empty(cq->queue, QUEUE_TYPE_KERNEL);
+ empty = queue_empty(cq->queue, QUEUE_TYPE_TO_CLIENT);

if ((flags & IB_CQ_REPORT_MISSED_EVENTS) && !empty)
ret = 1;
--
2.30.2

#5 lore raw

Re: 回复: [PATCH for-rc v3 1/6] RDMA/rxe: Add memory barriers to kernel queues

Quoted 8 lines · On 9/14/21 1:04 AM, [email protected] wrote:

Hi Bob,

Why do you want to use FROM_CLIENT and TO_CLIENT suffix?
It seems readable to use FROM_USER and TO_USER suffix (i.e. between user space and kernel space).

Best Regards,
Xiao Yang

The whole purpose of this patch is to extend the memory barriers to support user <-> kernel *and* kernel <-> kernel. Changing the name made it clearer that it's not just user space. It also made it easier to make sure I had gotten all of them changed/looked at. Now that is done, if everyone wants it back to USER I could be talked into it.

Bob

#6 lore raw

PATCH v3 2/6 RDMA/rxe: Fix memory allocation while locked

In reply to Bob Pearson

rxe_mcast_add_grp_elem() in rxe_mcast.c calls rxe_alloc() while holding spinlocks which in turn calls kzalloc(size, GFP_KERNEL) which is incorrect. This patch replaces rxe_alloc() by rxe_alloc_locked() which uses GFP_ATOMIC. This bug was caused by the below mentioned commit and failing to handle the need for the atomic allocate.

Fixes 4276fd0dddc9 ("Remove RXE_POOL_ATOMIC")
---
 drivers/infiniband/sw/rxe/rxe_mcast.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
drivers/infiniband/sw/rxe/rxe_mcast.c +1 -1
index 0ea9a5aa4ec0..1c1d1b53312d 100644
@@ -85,7 +85,7 @@ int rxe_mcast_add_grp_elem(struct rxe_dev *rxe, struct rxe_qp *qp,
 		goto out;
 	}
 
-	elem = rxe_alloc(&rxe->mc_elem_pool);
+	elem = rxe_alloc_locked(&rxe->mc_elem_pool);
 	if (!elem) {
 		err = -ENOMEM;
 		goto out;
Signature · 1 line
2.30.2
#7 lore raw

PATCH v3 3/6 RDMA/rxe: Cleanup MR status and type enums

In reply to Bob Pearson

Eliminate RXE_MR_STATE_ZOMBIE which is not compatible with IBA. RXE_MR_STATE_INVALID is better.

Replace RXE_MR_TYPE_XXX by IB_MR_TYPE_XXX which covers all the needed types.

---
 drivers/infiniband/sw/rxe/rxe_mr.c    | 20 ++++++++++++--------
 drivers/infiniband/sw/rxe/rxe_verbs.h |  9 +--------
 2 files changed, 13 insertions(+), 16 deletions(-)
drivers/infiniband/sw/rxe/rxe_mr.c +12 -8
index 5890a8246216..0cc24154762c 100644
@@ -24,17 +24,22 @@ u8 rxe_get_next_key(u32 last_key)
 
 int mr_check_range(struct rxe_mr *mr, u64 iova, size_t length)
 {
+
+
 	switch (mr->type) {
-	case RXE_MR_TYPE_DMA:
+	case IB_MR_TYPE_DMA:
 		return 0;
 
-	case RXE_MR_TYPE_MR:
+	case IB_MR_TYPE_USER:
+	case IB_MR_TYPE_MEM_REG:
 		if (iova < mr->iova || length > mr->length ||
 		    iova > mr->iova + mr->length - length)
 			return -EFAULT;
 		return 0;
 
 	default:
+		pr_warn("%s: mr type (%d) not supported\n",
+			__func__, mr->type);
 		return -EFAULT;
 	}
 }
@@ -51,7 +56,6 @@ static void rxe_mr_init(int access, struct rxe_mr *mr)
 	mr->ibmr.lkey = lkey;
 	mr->ibmr.rkey = rkey;
 	mr->state = RXE_MR_STATE_INVALID;
-	mr->type = RXE_MR_TYPE_NONE;
 	mr->map_shift = ilog2(RXE_BUF_PER_MAP);
 }
 
@@ -100,7 +104,7 @@ void rxe_mr_init_dma(struct rxe_pd *pd, int access, struct rxe_mr *mr)
 	mr->ibmr.pd = &pd->ibpd;
 	mr->access = access;
 	mr->state = RXE_MR_STATE_VALID;
-	mr->type = RXE_MR_TYPE_DMA;
+	mr->type = IB_MR_TYPE_DMA;
 }
 
 int rxe_mr_init_user(struct rxe_pd *pd, u64 start, u64 length, u64 iova,
@@ -173,7 +177,7 @@ int rxe_mr_init_user(struct rxe_pd *pd, u64 start, u64 length, u64 iova,
 	mr->va = start;
 	mr->offset = ib_umem_offset(umem);
 	mr->state = RXE_MR_STATE_VALID;
-	mr->type = RXE_MR_TYPE_MR;
+	mr->type = IB_MR_TYPE_USER;
 
 	return 0;
 
@@ -203,7 +207,7 @@ int rxe_mr_init_fast(struct rxe_pd *pd, int max_pages, struct rxe_mr *mr)
 	mr->ibmr.pd = &pd->ibpd;
 	mr->max_buf = max_pages;
 	mr->state = RXE_MR_STATE_FREE;
-	mr->type = RXE_MR_TYPE_MR;
+	mr->type = IB_MR_TYPE_MEM_REG;
 
 	return 0;
 
@@ -302,7 +306,7 @@ int rxe_mr_copy(struct rxe_mr *mr, u64 iova, void *addr, int length,
 	if (length == 0)
 		return 0;
 
-	if (mr->type == RXE_MR_TYPE_DMA) {
+	if (mr->type == IB_MR_TYPE_DMA) {
 		u8 *src, *dest;
 
 		src = (dir == RXE_TO_MR_OBJ) ? addr : ((void *)(uintptr_t)iova);
@@ -564,7 +568,7 @@ int rxe_dereg_mr(struct ib_mr *ibmr, struct ib_udata *udata)
 		return -EINVAL;
 	}
 
-	mr->state = RXE_MR_STATE_ZOMBIE;
+	mr->state = RXE_MR_STATE_INVALID;
 	rxe_drop_ref(mr_pd(mr));
 	rxe_drop_index(mr);
 	rxe_drop_ref(mr);
drivers/infiniband/sw/rxe/rxe_verbs.h +1 -8
index ac2a2148027f..c6aca2293294 100644
@@ -267,18 +267,11 @@ struct rxe_qp {
 };
 
 enum rxe_mr_state {
-	RXE_MR_STATE_ZOMBIE,
 	RXE_MR_STATE_INVALID,
 	RXE_MR_STATE_FREE,
 	RXE_MR_STATE_VALID,
 };
 
-enum rxe_mr_type {
-	RXE_MR_TYPE_NONE,
-	RXE_MR_TYPE_DMA,
-	RXE_MR_TYPE_MR,
-};
-
 enum rxe_mr_copy_dir {
 	RXE_TO_MR_OBJ,
 	RXE_FROM_MR_OBJ,
@@ -314,7 +307,7 @@ struct rxe_mr {
 	struct ib_umem		*umem;
 
 	enum rxe_mr_state	state;
-	enum rxe_mr_type	type;
+	enum ib_mr_type		type;
 	u64			va;
 	u64			iova;
 	size_t			length;
Signature · 1 line
2.30.2
#8 lore raw

PATCH v3 4/6 RDMA/rxe: Separate HW and SW l/rkeys

In reply to Bob Pearson

Separate software and simulated hardware lkeys and rkeys for MRs and MWs. This makes struct ib_mr and struct ib_mw isolated from hardware changes triggered by executing work requests.

This change fixes a bug seen in blktest.

---
 drivers/infiniband/sw/rxe/rxe_loc.h   |  1 +
 drivers/infiniband/sw/rxe/rxe_mr.c    | 69 ++++++++++++++++++++++-----
 drivers/infiniband/sw/rxe/rxe_mw.c    | 30 ++++++------
 drivers/infiniband/sw/rxe/rxe_req.c   | 14 ++----
 drivers/infiniband/sw/rxe/rxe_verbs.h | 18 ++-----
 5 files changed, 81 insertions(+), 51 deletions(-)
drivers/infiniband/sw/rxe/rxe_loc.h +1 -0
index f0c954575bde..4fd73b51fabf 100644
@@ -86,6 +86,7 @@ struct rxe_mr *lookup_mr(struct rxe_pd *pd, int access, u32 key,
 int mr_check_range(struct rxe_mr *mr, u64 iova, size_t length);
 int advance_dma_data(struct rxe_dma_info *dma, unsigned int length);
 int rxe_invalidate_mr(struct rxe_qp *qp, u32 rkey);
+int rxe_reg_fast_mr(struct rxe_qp *qp, struct rxe_send_wqe *wqe);
 int rxe_dereg_mr(struct ib_mr *ibmr, struct ib_udata *udata);
 void rxe_mr_cleanup(struct rxe_pool_entry *arg);
 
drivers/infiniband/sw/rxe/rxe_mr.c +58 -11

99 lines not on this page · raw message

drivers/infiniband/sw/rxe/rxe_mw.c +14 -16
index 5ba77df7598e..a5e2ea7d80f0 100644
@@ -21,7 +21,7 @@ int rxe_alloc_mw(struct ib_mw *ibmw, struct ib_udata *udata)
 	}
 
 	rxe_add_index(mw);
-	ibmw->rkey = (mw->pelem.index << 8) | rxe_get_next_key(-1);
+	mw->rkey = ibmw->rkey = (mw->pelem.index << 8) | rxe_get_next_key(-1);
 	mw->state = (mw->ibmw.type == IB_MW_TYPE_2) ?
 			RXE_MW_STATE_FREE : RXE_MW_STATE_VALID;
 	spin_lock_init(&mw->lock);
@@ -71,6 +71,8 @@ int rxe_dealloc_mw(struct ib_mw *ibmw)
 static int rxe_check_bind_mw(struct rxe_qp *qp, struct rxe_send_wqe *wqe,
 			 struct rxe_mw *mw, struct rxe_mr *mr)
 {
+	u32 key = wqe->wr.wr.mw.rkey & 0xff;
+
 	if (mw->ibmw.type == IB_MW_TYPE_1) {
 		if (unlikely(mw->state != RXE_MW_STATE_VALID)) {
 			pr_err_once(
@@ -108,7 +110,7 @@ static int rxe_check_bind_mw(struct rxe_qp *qp, struct rxe_send_wqe *wqe,
 		}
 	}
 
-	if (unlikely((wqe->wr.wr.mw.rkey & 0xff) == (mw->ibmw.rkey & 0xff))) {
+	if (unlikely(key == (mw->rkey & 0xff))) {
 		pr_err_once("attempt to bind MW with same key\n");
 		return -EINVAL;
 	}
@@ -161,13 +163,9 @@ static int rxe_check_bind_mw(struct rxe_qp *qp, struct rxe_send_wqe *wqe,
 static void rxe_do_bind_mw(struct rxe_qp *qp, struct rxe_send_wqe *wqe,
 		      struct rxe_mw *mw, struct rxe_mr *mr)
 {
-	u32 rkey;
-	u32 new_rkey;
-
-	rkey = mw->ibmw.rkey;
-	new_rkey = (rkey & 0xffffff00) | (wqe->wr.wr.mw.rkey & 0x000000ff);
+	u32 key = wqe->wr.wr.mw.rkey & 0xff;
 
-	mw->ibmw.rkey = new_rkey;
+	mw->rkey = (mw->rkey & ~0xff) | key;
 	mw->access = wqe->wr.wr.mw.access;
 	mw->state = RXE_MW_STATE_VALID;
 	mw->addr = wqe->wr.wr.mw.addr;
@@ -197,29 +195,29 @@ int rxe_bind_mw(struct rxe_qp *qp, struct rxe_send_wqe *wqe)
 	struct rxe_mw *mw;
 	struct rxe_mr *mr;
 	struct rxe_dev *rxe = to_rdev(qp->ibqp.device);
+	u32 mw_rkey = wqe->wr.wr.mw.mw_rkey;
+	u32 mr_lkey = wqe->wr.wr.mw.mr_lkey;
 	unsigned long flags;
 
-	mw = rxe_pool_get_index(&rxe->mw_pool,
-				wqe->wr.wr.mw.mw_rkey >> 8);
+	mw = rxe_pool_get_index(&rxe->mw_pool, mw_rkey >> 8);
 	if (unlikely(!mw)) {
 		ret = -EINVAL;
 		goto err;
 	}
 
-	if (unlikely(mw->ibmw.rkey != wqe->wr.wr.mw.mw_rkey)) {
+	if (unlikely(mw->rkey != mw_rkey)) {
 		ret = -EINVAL;
 		goto err_drop_mw;
 	}
 
 	if (likely(wqe->wr.wr.mw.length)) {
-		mr = rxe_pool_get_index(&rxe->mr_pool,
-					wqe->wr.wr.mw.mr_lkey >> 8);
+		mr = rxe_pool_get_index(&rxe->mr_pool, mr_lkey >> 8);
 		if (unlikely(!mr)) {
 			ret = -EINVAL;
 			goto err_drop_mw;
 		}
 
-		if (unlikely(mr->ibmr.lkey != wqe->wr.wr.mw.mr_lkey)) {
+		if (unlikely(mr->lkey != mr_lkey)) {
 			ret = -EINVAL;
 			goto err_drop_mr;
 		}
@@ -292,7 +290,7 @@ int rxe_invalidate_mw(struct rxe_qp *qp, u32 rkey)
 		goto err;
 	}
 
-	if (rkey != mw->ibmw.rkey) {
+	if (rkey != mw->rkey) {
 		ret = -EINVAL;
 		goto err_drop_ref;
 	}
@@ -323,7 +321,7 @@ struct rxe_mw *rxe_lookup_mw(struct rxe_qp *qp, int access, u32 rkey)
 	if (!mw)
 		return NULL;
 
-	if (unlikely((rxe_mw_rkey(mw) != rkey) || rxe_mw_pd(mw) != pd ||
+	if (unlikely((mw->rkey != rkey) || rxe_mw_pd(mw) != pd ||
 		     (mw->ibmw.type == IB_MW_TYPE_2 && mw->qp != qp) ||
 		     (mw->length == 0) ||
 		     (access && !(access & mw->access)) ||
drivers/infiniband/sw/rxe/rxe_req.c +5 -9
index 22c3edb28945..ac18dcd6905b 100644
@@ -561,7 +561,6 @@ static void update_state(struct rxe_qp *qp, struct rxe_send_wqe *wqe,
 static int rxe_do_local_ops(struct rxe_qp *qp, struct rxe_send_wqe *wqe)
 {
 	u8 opcode = wqe->wr.opcode;
-	struct rxe_mr *mr;
 	u32 rkey;
 	int ret;
 
@@ -579,14 +578,11 @@ static int rxe_do_local_ops(struct rxe_qp *qp, struct rxe_send_wqe *wqe)
 		}
 		break;
 	case IB_WR_REG_MR:
-		mr = to_rmr(wqe->wr.wr.reg.mr);
-		rxe_add_ref(mr);
-		mr->state = RXE_MR_STATE_VALID;
-		mr->access = wqe->wr.wr.reg.access;
-		mr->ibmr.lkey = wqe->wr.wr.reg.key;
-		mr->ibmr.rkey = wqe->wr.wr.reg.key;
-		mr->iova = wqe->wr.wr.reg.mr->iova;
-		rxe_drop_ref(mr);
+		ret = rxe_reg_fast_mr(qp, wqe);
+		if (unlikely(ret)) {
+			wqe->status = IB_WC_LOC_QP_OP_ERR;
+			return ret;
+		}
 		break;
 	case IB_WR_BIND_MW:
 		ret = rxe_bind_mw(qp, wqe);
drivers/infiniband/sw/rxe/rxe_verbs.h +3 -15
index c6aca2293294..31c38b2f7d0a 100644
@@ -306,6 +306,8 @@ struct rxe_mr {
 
 	struct ib_umem		*umem;
 
+	u32			lkey;
+	u32			rkey;
 	enum rxe_mr_state	state;
 	enum ib_mr_type		type;
 	u64			va;
@@ -343,6 +345,7 @@ struct rxe_mw {
 	enum rxe_mw_state	state;
 	struct rxe_qp		*qp; /* Type 2 only */
 	struct rxe_mr		*mr;
+	u32			rkey;
 	int			access;
 	u64			addr;
 	u64			length;
@@ -467,26 +470,11 @@ static inline struct rxe_pd *mr_pd(struct rxe_mr *mr)
 	return to_rpd(mr->ibmr.pd);
 }
 
-static inline u32 mr_lkey(struct rxe_mr *mr)
-{
-	return mr->ibmr.lkey;
-}
-
-static inline u32 mr_rkey(struct rxe_mr *mr)
-{
-	return mr->ibmr.rkey;
-}
-
 static inline struct rxe_pd *rxe_mw_pd(struct rxe_mw *mw)
 {
 	return to_rpd(mw->ibmw.pd);
 }
 
-static inline u32 rxe_mw_rkey(struct rxe_mw *mw)
-{
-	return mw->ibmw.rkey;
-}
-
 int rxe_register_device(struct rxe_dev *rxe, const char *ibdev_name);
 
 void rxe_mc_cleanup(struct rxe_pool_entry *arg);
Signature · 1 line
2.30.2
#9 lore raw

PATCH v3 5/6 RDMA/rxe: Create duplicate mapping tables for FMRs

In reply to Bob Pearson

For fast memory regions create duplicate mapping tables so ib_map_mr_sg() can build a new mapping table which is then swapped into place synchronously with the execution of an IB_WR_REG_MR work request.

Currently the rxe driver uses the same table for receiving RDMA operations and for building new tables in preparation for reusing the MR. This exposes users to potentially incorrect results.

---
 drivers/infiniband/sw/rxe/rxe_loc.h   |   1 +
 drivers/infiniband/sw/rxe/rxe_mr.c    | 196 +++++++++++++++++---------
 drivers/infiniband/sw/rxe/rxe_mw.c    |   6 +-
 drivers/infiniband/sw/rxe/rxe_verbs.c |  39 ++---
 drivers/infiniband/sw/rxe/rxe_verbs.h |  21 +--
 5 files changed, 161 insertions(+), 102 deletions(-)
drivers/infiniband/sw/rxe/rxe_loc.h +1 -0
index 4fd73b51fabf..1ca43b859d80 100644
@@ -87,6 +87,7 @@ int mr_check_range(struct rxe_mr *mr, u64 iova, size_t length);
 int advance_dma_data(struct rxe_dma_info *dma, unsigned int length);
 int rxe_invalidate_mr(struct rxe_qp *qp, u32 rkey);
 int rxe_reg_fast_mr(struct rxe_qp *qp, struct rxe_send_wqe *wqe);
+int rxe_mr_set_page(struct ib_mr *ibmr, u64 addr);
 int rxe_dereg_mr(struct ib_mr *ibmr, struct ib_udata *udata);
 void rxe_mr_cleanup(struct rxe_pool_entry *arg);
 
drivers/infiniband/sw/rxe/rxe_mr.c +132 -64

355 lines not on this page · raw message

drivers/infiniband/sw/rxe/rxe_mw.c +3 -3
index a5e2ea7d80f0..9534a7fe1a98 100644
@@ -142,15 +142,15 @@ static int rxe_check_bind_mw(struct rxe_qp *qp, struct rxe_send_wqe *wqe,
 
 	/* C10-75 */
 	if (mw->access & IB_ZERO_BASED) {
-		if (unlikely(wqe->wr.wr.mw.length > mr->length)) {
+		if (unlikely(wqe->wr.wr.mw.length > mr->cur_map_set->length)) {
 			pr_err_once(
 				"attempt to bind a ZB MW outside of the MR\n");
 			return -EINVAL;
 		}
 	} else {
-		if (unlikely((wqe->wr.wr.mw.addr < mr->iova) ||
+		if (unlikely((wqe->wr.wr.mw.addr < mr->cur_map_set->iova) ||
 			     ((wqe->wr.wr.mw.addr + wqe->wr.wr.mw.length) >
-			      (mr->iova + mr->length)))) {
+			      (mr->cur_map_set->iova + mr->cur_map_set->length)))) {
 			pr_err_once(
 				"attempt to bind a VA MW outside of the MR\n");
 			return -EINVAL;
drivers/infiniband/sw/rxe/rxe_verbs.c +12 -27
index dc70e3edeba6..e7f482184359 100644
@@ -954,41 +954,26 @@ static struct ib_mr *rxe_alloc_mr(struct ib_pd *ibpd, enum ib_mr_type mr_type,
 	return ERR_PTR(err);
 }
 
-static int rxe_set_page(struct ib_mr *ibmr, u64 addr)
-{
-	struct rxe_mr *mr = to_rmr(ibmr);
-	struct rxe_map *map;
-	struct rxe_phys_buf *buf;
-
-	if (unlikely(mr->nbuf == mr->num_buf))
-		return -ENOMEM;
-
-	map = mr->map[mr->nbuf / RXE_BUF_PER_MAP];
-	buf = &map->buf[mr->nbuf % RXE_BUF_PER_MAP];
-
-	buf->addr = addr;
-	buf->size = ibmr->page_size;
-	mr->nbuf++;
-
-	return 0;
-}
-
+/* build next_map_set from scatterlist
+ * The IB_WR_REG_MR WR will swap map_sets
+ */
 static int rxe_map_mr_sg(struct ib_mr *ibmr, struct scatterlist *sg,
 			 int sg_nents, unsigned int *sg_offset)
 {
 	struct rxe_mr *mr = to_rmr(ibmr);
+	struct rxe_map_set *set = mr->next_map_set;
 	int n;
 
-	mr->nbuf = 0;
+	set->nbuf = 0;
 
-	n = ib_sg_to_pages(ibmr, sg, sg_nents, sg_offset, rxe_set_page);
+	n = ib_sg_to_pages(ibmr, sg, sg_nents, sg_offset, rxe_mr_set_page);
 
-	mr->va = ibmr->iova;
-	mr->iova = ibmr->iova;
-	mr->length = ibmr->length;
-	mr->page_shift = ilog2(ibmr->page_size);
-	mr->page_mask = ibmr->page_size - 1;
-	mr->offset = mr->iova & mr->page_mask;
+	set->va = ibmr->iova;
+	set->iova = ibmr->iova;
+	set->length = ibmr->length;
+	set->page_shift = ilog2(ibmr->page_size);
+	set->page_mask = ibmr->page_size - 1;
+	set->offset = set->iova & set->page_mask;
 
 	return n;
 }
drivers/infiniband/sw/rxe/rxe_verbs.h +13 -8
index 31c38b2f7d0a..9eabc8f30359 100644
@@ -293,6 +293,17 @@ struct rxe_map {
 	struct rxe_phys_buf	buf[RXE_BUF_PER_MAP];
 };
 
+struct rxe_map_set {
+	struct rxe_map		**map;
+	u64			va;
+	u64			iova;
+	size_t			length;
+	u32			offset;
+	u32			nbuf;
+	int			page_shift;
+	int			page_mask;
+};
+
 static inline int rkey_is_mw(u32 rkey)
 {
 	u32 index = rkey >> 8;
@@ -310,26 +321,20 @@ struct rxe_mr {
 	u32			rkey;
 	enum rxe_mr_state	state;
 	enum ib_mr_type		type;
-	u64			va;
-	u64			iova;
-	size_t			length;
-	u32			offset;
 	int			access;
 
-	int			page_shift;
-	int			page_mask;
 	int			map_shift;
 	int			map_mask;
 
 	u32			num_buf;
-	u32			nbuf;
 
 	u32			max_buf;
 	u32			num_map;
 
 	atomic_t		num_mw;
 
-	struct rxe_map		**map;
+	struct rxe_map_set	*cur_map_set;
+	struct rxe_map_set	*next_map_set;
 };
 
 enum rxe_mw_state {
Signature · 1 line
2.30.2
#10 lore raw

PATCH v3 6/6 RDMA/rxe: Only allow invalidate for appropriate MRs

In reply to Bob Pearson

Local and remote invalidate operations are not allowed by IBA for MRs created by (re)register memory verbs. This patch checks the MR type in rxe_invalidate_mr().

---
 drivers/infiniband/sw/rxe/rxe_mr.c | 6 ++++++
 1 file changed, 6 insertions(+)
drivers/infiniband/sw/rxe/rxe_mr.c +6 -0
index 8d658d42abed..53271df10e47 100644
@@ -605,6 +605,12 @@ int rxe_invalidate_mr(struct rxe_qp *qp, u32 rkey)
 		goto err_drop_ref;
 	}
 
+	if (unlikely(mr->type != IB_MR_TYPE_MEM_REG)) {
+		pr_warn("%s: mr->type (%d) is wrong type\n", __func__, mr->type);
+		ret = -EINVAL;
+		goto err_drop_ref;
+	}
+
 	mr->state = RXE_MR_STATE_FREE;
 	ret = 0;
 
Signature · 1 line
2.30.2
#11 lore raw
In reply to Bob Pearson
Quoted 10 lines · Bob Pearson

This series of patches implements several bug fixes and minor cleanups of the rxe driver. Specifically these fix a bug exposed by blktest.

They apply cleanly to both
commit 2169b908894df2ce83e7eb4a399d3224b2635126 (origin/for-rc, for-rc) commit 6a217437f9f5482a3f6f2dc5fcd27cf0f62409ac (HEAD -> for-next,
origin/wip/jgg-for-next, origin/for-next, origin/HEAD)

These are being resubmitted to for-rc instead of for-next.

Hi Bob,

Thanks for having rebased and reposted this patch series. I have applied this series on top of commit 2169b908894d ("IB/hfi1: make hist static"). A kernel bug was triggered while running test srp/001. I have attached the kernel configuration used in my test to this email.

Thanks,

Bart.

ib_srpt Received SRP_LOGIN_REQ with i_port_id fe80:0000:0000:0000:5054:00ff:fe86:7464, t_port_id 5054:00ff:fe86:7464:5054:00ff:fe86:7464 and it_iu_len 8260 on port 1 (guid=fe80:0000:0000:0000:5054:00ff:fe86:7464); pkey 0xffff

BUG: unable to handle page fault for address: ffffc900e357d614
#PF: supervisor read access in kernel mode
#PF: error_code(0x0000) - not-present page
PGD 100000067 P4D 100000067 PUD 0
Oops: 0000 [#1] PREEMPT SMP KASAN
CPU: 26 PID: 148 Comm: ksoftirqd/26 Tainted: G            E     5.14.0-rc6-dbg+ #2
Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS 1.14.0-2 04/01/2014
RIP: 0010:rxe_completer+0x96d/0x1050 [rdma_rxe]
Code: e0 49 8b 44 24 08 44 89 e9 41 d3 e6 4e 8d a4 30 80 01 00 00 4d 85 e4 0f 84 f9 00 00 00 49 8d bc 24 94 00 00 00 e8 73 a8 b1 e0 <41> 8b 84 24 94 00 00 00 85 c0 0f 84 df 00 00 00 83 f8 03 0f 84 bf
RSP: 0018:ffff8881014075f8 EFLAGS: 00010246
RAX: 0000000000000000 RBX: ffff88813c67c000 RCX: dffffc0000000000
RDX: 0000000000000007 RSI: ffffffff826920c0 RDI: ffffc900e357d614
RBP: ffff8881014076e8 R08: ffffffffa09b228d R09: ffff88813c67c57b
R10: ffffed10278cf8af R11: 0000000000000000 R12: ffffc900e357d580
R13: 000000000000000a R14: 00000000d9c99400 R15: ffff8881515ddd08
FS:  0000000000000000(0000) GS:ffff88842d100000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: ffffc900e357d614 CR3: 0000000002e29005 CR4: 0000000000770ee0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
PKRU: 55555554
Call Trace:
 rxe_do_task+0xdd/0x160 [rdma_rxe]
 rxe_run_task+0x67/0x80 [rdma_rxe]
 rxe_comp_queue_pkt+0x75/0x80 [rdma_rxe]
 rxe_rcv+0x345/0x480 [rdma_rxe]
 rxe_xmit_packet+0x1af/0x300 [rdma_rxe]
 send_ack.isra.0+0x88/0xd0 [rdma_rxe]
 rxe_responder+0xf4c/0x15e0 [rdma_rxe]
 rxe_do_task+0xdd/0x160 [rdma_rxe]
 rxe_run_task+0x67/0x80 [rdma_rxe]
 rxe_resp_queue_pkt+0x5a/0x60 [rdma_rxe]
 rxe_rcv+0x370/0x480 [rdma_rxe]
 rxe_xmit_packet+0x1af/0x300 [rdma_rxe]
 rxe_requester+0x4f4/0xe80 [rdma_rxe]
 rxe_do_task+0xdd/0x160 [rdma_rxe]
 tasklet_action_common.constprop.0+0x168/0x1b0
 tasklet_action+0x44/0x60
 __do_softirq+0x1db/0x6ed
 run_ksoftirqd+0x37/0x60
 smpboot_thread_fn+0x302/0x410
 kthread+0x1f6/0x220
 ret_from_fork+0x1f/0x30
Modules linked in: ib_srp(E) scsi_transport_srp(E) target_core_user(E) uio(E) target_core_pscsi(E) target_core_file(E) ib_srpt(E) target_core_iblock(E) target_core_mod(E) ib_umad(E) rdma_ucm(E) ib_iser(E) libiscsi(E) scsi_transport_iscsi(E) rdma_cm(E) iw_cm(E) scsi_debug(E) ib_cm(E) rdma_rxe(E) ip6_udp_tunnel(E) udp_tunnel(E) ib_uverbs(E) null_blk(E) ib_core(E) brd(E) af_packet(E) nft_fib_inet(E) nft_fib_ipv4(E) nft_fib_ipv6(E) nft_fib(E) nft_reject_inet(E) nf_reject_ipv4(E) nf_reject_ipv6(E) nft_reject(E) nft_ct(E) nft_chain_nat(E) nf_tables(E) ebtable_nat(E) iTCO_wdt(E) watchdog(E) ebtable_broute(E) intel_rapl_msr(E) intel_pmc_bxt(E) ip6table_nat(E) ip6table_mangle(E) ip6table_raw(E) iptable_nat(E) nf_nat(E) nf_conntrack(E) nf_defrag_ipv6(E) nf_defrag_ipv4(E) libcrc32c(E) iptable_mangle(E) iptable_raw(E) ip_set(E) nfnetlink(E) ebtable_filter(E) ebtables(E) ip6table_filter(E) ip6_tables(E) rfkill(E) iptable_filter(E) ip_tables(E) x_tables(E) bpfilter(E) intel_rapl_common(E)
 iosf_mbi(E) isst_if_common(E) i2c_i801(E) pcspkr(E) i2c_smbus(E) virtio_net(E) lpc_ich(E) virtio_balloon(E) net_failover(E) failover(E) tiny_power_button(E) button(E) fuse(E) crct10dif_pclmul(E) crc32_pclmul(E) crc32c_intel(E) ghash_clmulni_intel(E) aesni_intel(E) 
crypto_simd(E) cryptd(E) sr_mod(E) serio_raw(E) cdrom(E) virtio_gpu(E) virtio_dma_buf(E) drm_kms_helper(E) syscopyarea(E) sysfillrect(E) sysimgblt(E) fb_sys_fops(E) cec(E) drm(E) qemu_fw_cfg(E) sg(E) nbd(E) dm_multipath(E) dm_mod(E) scsi_dh_rdac(E) scsi_dh_emc(E) scsi_dh_alua(E) virtio_rng(E)
CR2: ffffc900e357d614
---[ end trace 0667a278da47193a ]---
RIP: 0010:rxe_completer+0x96d/0x1050 [rdma_rxe]
Code: e0 49 8b 44 24 08 44 89 e9 41 d3 e6 4e 8d a4 30 80 01 00 00 4d 85 e4 0f 84 f9 00 00 00 49 8d bc 24 94 00 00 00 e8 73 a8 b1 e0 <41> 8b 84 24 94 00 00 00 85 c0 0f 84 df 00 00 00 83 f8 03 0f 84 bf
RSP: 0018:ffff8881014075f8 EFLAGS: 00010246
RAX: 0000000000000000 RBX: ffff88813c67c000 RCX: dffffc0000000000
RDX: 0000000000000007 RSI: ffffffff826920c0 RDI: ffffc900e357d614
RBP: ffff8881014076e8 R08: ffffffffa09b228d R09: ffff88813c67c57b
R10: ffffed10278cf8af R11: 0000000000000000 R12: ffffc900e357d580
R13: 000000000000000a R14: 00000000d9c99400 R15: ffff8881515ddd08
FS:  0000000000000000(0000) GS:ffff88842d100000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: ffffc900e357d614 CR3: 0000000002e29005 CR4: 0000000000770ee0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
PKRU: 55555554
Kernel panic - not syncing: Fatal exception in interrupt

Kernel Offset: disabled
Rebooting in 90 seconds..

#12 lore raw

Bart,

I was able to run this test case but it is not failing. On my system it passes in ~1sec. I have several questions about your system setup.

1. Which rdma-core are you running? Out of box or the github tree?
2. Can you run ib_send_bw? Python test suite in rdma-core?
3. Where did you get the kernel bits? Which git tree? Which branch?

Thanks,

Bob Pearson

-----Original Message-----
From: Bart Van Assche <[email protected]>
Sent: Thursday, September 9, 2021 4:52 PM
To: Bob Pearson <[email protected]>; [email protected]; [email protected]; [email protected]; [email protected]
Subject: Re: [PATCH for-rc v3 0/6] RDMA/rxe: Various bug fixes.

Quoted 9 lines · Bob Pearson

This series of patches implements several bug fixes and minor cleanups of the rxe driver. Specifically these fix a bug exposed by blktest.

They apply cleanly to both
commit 2169b908894df2ce83e7eb4a399d3224b2635126 (origin/for-rc, for-rc) commit 6a217437f9f5482a3f6f2dc5fcd27cf0f62409ac (HEAD -> for-next,
origin/wip/jgg-for-next, origin/for-next, origin/HEAD)

These are being resubmitted to for-rc instead of for-next.

Hi Bob,

Thanks for having rebased and reposted this patch series. I have applied this series on top of commit 2169b908894d ("IB/hfi1: make hist static"). A kernel bug was triggered while running test srp/001. I have attached the kernel configuration used in my test to this email.

Thanks,

Bart.

ib_srpt Received SRP_LOGIN_REQ with i_port_id fe80:0000:0000:0000:5054:00ff:fe86:7464, t_port_id 5054:00ff:fe86:7464:5054:00ff:fe86:7464 and it_iu_len 8260 on port 1 (guid=fe80:0000:0000:0000:5054:00ff:fe86:7464); pkey 0xffff

BUG: unable to handle page fault for address: ffffc900e357d614
#PF: supervisor read access in kernel mode
#PF: error_code(0x0000) - not-present page PGD 100000067 P4D 100000067 PUD 0
Oops: 0000 [#1] PREEMPT SMP KASAN
CPU: 26 PID: 148 Comm: ksoftirqd/26 Tainted: G            E     5.14.0-rc6-dbg+ #2
Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS 1.14.0-2 04/01/2014
RIP: 0010:rxe_completer+0x96d/0x1050 [rdma_rxe]
Code: e0 49 8b 44 24 08 44 89 e9 41 d3 e6 4e 8d a4 30 80 01 00 00 4d 85 e4 0f 84 f9 00 00 00 49 8d bc 24 94 00 00 00 e8 73 a8 b1 e0 <41> 8b 84 24 94 00 00 00 85 c0 0f 84 df 00 00 00 83 f8 03 0f 84 bf
RSP: 0018:ffff8881014075f8 EFLAGS: 00010246
RAX: 0000000000000000 RBX: ffff88813c67c000 RCX: dffffc0000000000
RDX: 0000000000000007 RSI: ffffffff826920c0 RDI: ffffc900e357d614
RBP: ffff8881014076e8 R08: ffffffffa09b228d R09: ffff88813c67c57b
R10: ffffed10278cf8af R11: 0000000000000000 R12: ffffc900e357d580
R13: 000000000000000a R14: 00000000d9c99400 R15: ffff8881515ddd08
FS:  0000000000000000(0000) GS:ffff88842d100000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: ffffc900e357d614 CR3: 0000000002e29005 CR4: 0000000000770ee0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
PKRU: 55555554
Call Trace:
  rxe_do_task+0xdd/0x160 [rdma_rxe]
  rxe_run_task+0x67/0x80 [rdma_rxe]
  rxe_comp_queue_pkt+0x75/0x80 [rdma_rxe]
  rxe_rcv+0x345/0x480 [rdma_rxe]
  rxe_xmit_packet+0x1af/0x300 [rdma_rxe]
  send_ack.isra.0+0x88/0xd0 [rdma_rxe]
  rxe_responder+0xf4c/0x15e0 [rdma_rxe]
  rxe_do_task+0xdd/0x160 [rdma_rxe]
  rxe_run_task+0x67/0x80 [rdma_rxe]
  rxe_resp_queue_pkt+0x5a/0x60 [rdma_rxe]
  rxe_rcv+0x370/0x480 [rdma_rxe]
  rxe_xmit_packet+0x1af/0x300 [rdma_rxe]
  rxe_requester+0x4f4/0xe80 [rdma_rxe]
  rxe_do_task+0xdd/0x160 [rdma_rxe]
  tasklet_action_common.constprop.0+0x168/0x1b0
  tasklet_action+0x44/0x60
  __do_softirq+0x1db/0x6ed
  run_ksoftirqd+0x37/0x60
  smpboot_thread_fn+0x302/0x410
  kthread+0x1f6/0x220
  ret_from_fork+0x1f/0x30
Modules linked in: ib_srp(E) scsi_transport_srp(E) target_core_user(E) uio(E) target_core_pscsi(E) target_core_file(E) ib_srpt(E) target_core_iblock(E) target_core_mod(E) ib_umad(E) rdma_ucm(E) ib_iser(E) libiscsi(E) scsi_transport_iscsi(E) rdma_cm(E) iw_cm(E)

scsi_debug(E) ib_cm(E) rdma_rxe(E) ip6_udp_tunnel(E) udp_tunnel(E) ib_uverbs(E) null_blk(E) ib_core(E) brd(E) af_packet(E) nft_fib_inet(E) nft_fib_ipv4(E) nft_fib_ipv6(E) nft_fib(E) nft_reject_inet(E) nf_reject_ipv4(E) nf_reject_ipv6(E) nft_reject(E) nft_ct(E) nft_chain_nat(E) nf_tables(E) ebtable_nat(E) iTCO_wdt(E) watchdog(E) ebtable_broute(E) intel_rapl_msr(E) intel_pmc_bxt(E) ip6table_nat(E) ip6table_mangle(E) ip6table_raw(E) iptable_nat(E) nf_nat(E) nf_conntrack(E) nf_defrag_ipv6(E) nf_defrag_ipv4(E) libcrc32c(E) iptable_mangle(E) iptable_raw(E) ip_set(E) nfnetlink(E) ebtable_filter(E) ebtables(E) ip6table_filter(E) ip6_tables(E) rfkill(E) iptable_filter(E) ip_tables(E) x_tables(E) bpfilter(E) intel_rapl_common(E)
iosf_mbi(E) isst_if_common(E) i2c_i801(E) pcspkr(E) i2c_smbus(E) virtio_net(E) lpc_ich(E) virtio_balloon(E) net_failover(E) failover(E) tiny_power_button(E) button(E) fuse(E) crct10dif_pclmul(E) crc32_pclmul(E) crc32c_intel(E) ghash_clmulni_intel(E) aesni_intel(E)
crypto_simd(E) cryptd(E) sr_mod(E) serio_raw(E) cdrom(E) virtio_gpu(E) virtio_dma_buf(E) drm_kms_helper(E) syscopyarea(E) sysfillrect(E) sysimgblt(E) fb_sys_fops(E) cec(E) drm(E) qemu_fw_cfg(E) sg(E) nbd(E) dm_multipath(E) dm_mod(E) scsi_dh_rdac(E) scsi_dh_emc(E) scsi_dh_alua(E) virtio_rng(E)
CR2: ffffc900e357d614

---[ end trace 0667a278da47193a ]---
RIP: 0010:rxe_completer+0x96d/0x1050 [rdma_rxe]
Code: e0 49 8b 44 24 08 44 89 e9 41 d3 e6 4e 8d a4 30 80 01 00 00 4d 85 e4 0f 84 f9 00 00 00 49 8d bc 24 94 00 00 00 e8 73 a8 b1 e0 <41> 8b 84 24 94 00 00 00 85 c0 0f 84 df 00 00 00 83 f8 03 0f 84 bf
RSP: 0018:ffff8881014075f8 EFLAGS: 00010246
RAX: 0000000000000000 RBX: ffff88813c67c000 RCX: dffffc0000000000
RDX: 0000000000000007 RSI: ffffffff826920c0 RDI: ffffc900e357d614
RBP: ffff8881014076e8 R08: ffffffffa09b228d R09: ffff88813c67c57b
R10: ffffed10278cf8af R11: 0000000000000000 R12: ffffc900e357d580
R13: 000000000000000a R14: 00000000d9c99400 R15: ffff8881515ddd08
FS:  0000000000000000(0000) GS:ffff88842d100000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: ffffc900e357d614 CR3: 0000000002e29005 CR4: 0000000000770ee0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
PKRU: 55555554
Kernel panic - not syncing: Fatal exception in interrupt Kernel Offset: disabled Rebooting in 90 seconds..
#13 lore raw
Quoted 1 line · Robert B

1. Which rdma-core are you running? Out of box or the github tree?

I'm using the rdma-core package included in openSUSE Tumbleweed. blktests pass with that rdma-core package against older kernel versions so I think the rdma-core package is fine. The version number of the rdma-core package I'm using is as follows:

$ rpm -q rdma-core
rdma-core-36.0-1.1.x86_64

The rdma tool comes from the iproute2 package:

$ rpm -qf /sbin/rdma
iproute2-5.13-1.1.x86_64
Quoted 1 line

3. Where did you get the kernel bits? Which git tree? Which branch?

Hmm ... wasn't that mentioned in my previous email? I mentioned a commit SHA and these SHA numbers are unique and unambiguous. Anyway: commit 2169b908894d comes from the for-rc branch of the following git repository:
git://git.kernel.org/pub/scm/linux/kernel/git/rdma/rdma.git.

Bart.

#14 lore raw
Quoted 14 lines · Bart Van Assche
Quoted 1 line · Robert B

1. Which rdma-core are you running? Out of box or the github tree?

I'm using the rdma-core package included in openSUSE Tumbleweed. blktests pass with that rdma-core package against older kernel versions so I think the rdma-core package is fine. The version number of the rdma-core package I'm using is as follows:

$ rpm -q rdma-core
rdma-core-36.0-1.1.x86_64

The rdma tool comes from the iproute2 package:

$ rpm -qf /sbin/rdma
iproute2-5.13-1.1.x86_64
Quoted 10 lines
Quoted 1 line

3. Where did you get the kernel bits? Which git tree? Which branch?

Hmm ... wasn't that mentioned in my previous email? I mentioned a commit SHA and these SHA numbers are unique and unambiguous. Anyway: commit 2169b908894d comes from the for-rc branch of the following git repository:
git://git.kernel.org/pub/scm/linux/kernel/git/rdma/rdma.git.

Bart.

You'd be surprised how much I don't know. I do know the numbers are unique but I haven't the faintest idea how to decode them into useful strings.

In theory you are correct and rdma-core and kernels are supposed to be forwards and backwards compatible but that is a goal and sometimes regressions do occur. I can try to run with that version just to make sure.

There is a problem I have seen where some newer distros do not create the default IPV6 address from the MAC address. They randomize it (Ubuntu does this) and rxe is broken as a result. I end up having to add a line like

sudo ip addr add dev enp6s0 fe80::b62e:99ff:fef9:fa2e/64
(where the MAC address is b4:2e:99:f9:fa:2e) just before the line
sudo rdma link add rxe_1 type rxe netdev enp6s0

But, when this is an issue rxe is really broken and almost nothing works so that may not be an issue for you.

I will try to recreate your setup and retest.

Thanks,

Bob

#15 lore raw
In reply to Bart Van Assche
Quoted 14 lines · Bart Van Assche
Quoted 1 line · Robert B

1. Which rdma-core are you running? Out of box or the github tree?

I'm using the rdma-core package included in openSUSE Tumbleweed. blktests pass with that rdma-core package against older kernel versions so I think the rdma-core package is fine. The version number of the rdma-core package I'm using is as follows:

$ rpm -q rdma-core
rdma-core-36.0-1.1.x86_64

The rdma tool comes from the iproute2 package:

$ rpm -qf /sbin/rdma
iproute2-5.13-1.1.x86_64
Quoted 10 lines
Quoted 1 line

3. Where did you get the kernel bits? Which git tree? Which branch?

Hmm ... wasn't that mentioned in my previous email? I mentioned a commit SHA and these SHA numbers are unique and unambiguous. Anyway: commit 2169b908894d comes from the for-rc branch of the following git repository:
git://git.kernel.org/pub/scm/linux/kernel/git/rdma/rdma.git.

Bart.

OK I checked out the kernel with the SHA number above and applied the patch series and rebuilt and reinstalled the kernel. I checked out v36.0 of rdma-core and rebuilt that. rdma is version 5.9.0 but I doubt that will have any effect. My startup script is

export LD_LIBRARY_PATH=/home/bob/src/rdma-core/build/lib/:/usr/local/lib:/usr/lib

sudo ip link set dev enp0s3 mtu 8500

sudo ip addr add dev enp0s3 fe80::0a00:27ff:fe94:8a69/64

sudo rdma link add rxe0 type rxe netdev enp0s3

I am running on a Virtualbox VM instance of Ubuntu 21.04 with 20 cores and 8GB of RAM.

The test looks like

    sudo ./check -q srp/001

    srp/001 (Create and remove LUNs)                             [passed]

        runtime  1.174s  ...  1.236s

There were no issues.

Any guesses what else to look at?

Thanks,

Bob

#16 lore raw
Quoted 52 lines · Bob Pearson
Quoted 14 lines · Bart Van Assche
Quoted 1 line · Robert B

1. Which rdma-core are you running? Out of box or the github tree?

I'm using the rdma-core package included in openSUSE Tumbleweed. blktests pass with that rdma-core package against older kernel versions so I think the rdma-core package is fine. The version number of the rdma-core package I'm using is as follows:

$ rpm -q rdma-core
rdma-core-36.0-1.1.x86_64

The rdma tool comes from the iproute2 package:

$ rpm -qf /sbin/rdma
iproute2-5.13-1.1.x86_64
Quoted 10 lines
Quoted 1 line

3. Where did you get the kernel bits? Which git tree? Which branch?

Hmm ... wasn't that mentioned in my previous email? I mentioned a commit SHA and these SHA numbers are unique and unambiguous. Anyway: commit 2169b908894d comes from the for-rc branch of the following git repository:
git://git.kernel.org/pub/scm/linux/kernel/git/rdma/rdma.git.

Bart.

OK I checked out the kernel with the SHA number above and applied the patch series and rebuilt and reinstalled the kernel. I checked out v36.0 of rdma-core and rebuilt that. rdma is version 5.9.0 but I doubt that will have any effect. My startup script is

export LD_LIBRARY_PATH=/home/bob/src/rdma-core/build/lib/:/usr/local/lib:/usr/lib

sudo ip link set dev enp0s3 mtu 8500

sudo ip addr add dev enp0s3 fe80::0a00:27ff:fe94:8a69/64

sudo rdma link add rxe0 type rxe netdev enp0s3

I am running on a Virtualbox VM instance of Ubuntu 21.04 with 20 cores and 8GB of RAM.

The test looks like

    sudo ./check -q srp/001

    srp/001 (Create and remove LUNs)                             [passed]

        runtime  1.174s  ...  1.236s
Quoted 8 lines

There were no issues.

Any guesses what else to look at?

Thanks,

Bob

The 8500 is not required. It runs fine with 4K MTU just as well.

#17 lore raw
In reply to Bob Pearson
Quoted 26 lines · Bob Pearson

OK I checked out the kernel with the SHA number above and applied the patch series and rebuilt and reinstalled the kernel. I checked out v36.0 of rdma-core and rebuilt that. rdma is version 5.9.0 but I doubt that will have any effect. My startup script is

export LD_LIBRARY_PATH=/home/bob/src/rdma-core/build/lib/:/usr/local/lib:/usr/lib

sudo ip link set dev enp0s3 mtu 8500

sudo ip addr add dev enp0s3 fe80::0a00:27ff:fe94:8a69/64

sudo rdma link add rxe0 type rxe netdev enp0s3

I am running on a Virtualbox VM instance of Ubuntu 21.04 with 20 cores and 8GB of RAM.

The test looks like

     sudo ./check -q srp/001

     srp/001 (Create and remove LUNs)                             [passed]

         runtime  1.174s  ...  1.236s
Quoted 3 lines

There were no issues.

Any guesses what else to look at?

The test I ran is different. I did not run any of the ip link / ip addr / rdma link commands since the blktests scripts already run the rdma link command. The bug I reported in my previous email is reproducible and triggers a VM halt.

Are we using the same kernel config? I attached my kernel config to my previous email. The source code location of the crash address is as follows:

(gdb) list *(rxe_completer+0x96d)
0x228d is in rxe_completer (drivers/infiniband/sw/rxe/rxe_comp.c:149).
144 */
145 wqe = queue_head(qp->sq.queue, QUEUE_TYPE_FROM_CLIENT);
146 *wqe_p = wqe;
147
148 /* no WQE or requester has not started it yet */
149 if (!wqe || wqe->state == wqe_state_posted)
150 return pkt ? COMPST_DONE : COMPST_EXIT;
151
152 /* WQE does not require an ack */
153 if (wqe->state == wqe_state_done)

The disassembly output is as follows:

drivers/infiniband/sw/rxe/rxe_comp.c:
149 if (!wqe || wqe->state == wqe_state_posted)

   0x0000000000002277 <+2391>:  test   %r12,%r12
   0x000000000000227a <+2394>:  je     0x2379 <rxe_completer+2649>
   0x0000000000002280 <+2400>:  lea    0x94(%r12),%rdi
   0x0000000000002288 <+2408>:  call   0x228d <rxe_completer+2413>
   0x000000000000228d <+2413>:  mov    0x94(%r12),%eax
   0x0000000000002295 <+2421>:  test   %eax,%eax
   0x0000000000002297 <+2423>:  je     0x237c <rxe_completer+2652>

So the instruction that triggers the crash is "mov 0x94(%r12),%eax". Does consumer_addr() perhaps return an invalid address under certain circumstances?

Thanks,

Bart.

#18 lore raw
Quoted 65 lines · Bart Van Assche
Quoted 22 lines · Bob Pearson

OK I checked out the kernel with the SHA number above and applied the patch series and rebuilt and reinstalled the kernel. I checked out v36.0 of rdma-core and rebuilt that. rdma is version 5.9.0 but I doubt that will have any effect. My startup script is

     export LD_LIBRARY_PATH=/home/bob/src/rdma-core/build/lib/:/usr/local/lib:/usr/lib

     sudo ip link set dev enp0s3 mtu 8500

     sudo ip addr add dev enp0s3 fe80::0a00:27ff:fe94:8a69/64

     sudo rdma link add rxe0 type rxe netdev enp0s3

I am running on a Virtualbox VM instance of Ubuntu 21.04 with 20 cores and 8GB of RAM.

The test looks like

     sudo ./check -q srp/001

Quoted 7 lines

     srp/001 (Create and remove LUNs)                             [passed]

         runtime  1.174s  ...  1.236s

There were no issues.

Any guesses what else to look at?

The test I ran is different. I did not run any of the ip link / ip addr / rdma link commands since the blktests scripts already run the rdma link command. The bug I reported in my previous email is reproducible and triggers a VM halt.

Are we using the same kernel config? I attached my kernel config to my previous email. The source code location of the crash address is as follows:

(gdb) list *(rxe_completer+0x96d)
0x228d is in rxe_completer (drivers/infiniband/sw/rxe/rxe_comp.c:149). 144              */
145             wqe = queue_head(qp->sq.queue, QUEUE_TYPE_FROM_CLIENT); 146             *wqe_p = wqe;
147
148             /* no WQE or requester has not started it yet */ 149             if (!wqe || wqe->state == wqe_state_posted) 150                     return pkt ? COMPST_DONE : COMPST_EXIT; 151
152             /* WQE does not require an ack */
153             if (wqe->state == wqe_state_done)

The disassembly output is as follows:

drivers/infiniband/sw/rxe/rxe_comp.c:
149             if (!wqe || wqe->state == wqe_state_posted) 0x0000000000002277 <+2391>:  test   %r12,%r12 0x000000000000227a <+2394>:  je     0x2379 <rxe_completer+2649> 0x0000000000002280 <+2400>:  lea    0x94(%r12),%rdi 0x0000000000002288 <+2408>:  call   0x228d <rxe_completer+2413> 0x000000000000228d <+2413>:  mov    0x94(%r12),%eax 0x0000000000002295 <+2421>:  test   %eax,%eax 0x0000000000002297 <+2423>:  je     0x237c <rxe_completer+2652>

Quoted 7 lines

So the instruction that triggers the crash is "mov 0x94(%r12),%eax". Does consumer_addr() perhaps return an invalid address under certain circumstances?

Thanks,

Bart.

The most likely cause of this was fixed by a patch submitted 8/20/2021 by Xiao Yang. It is copied here

From: Xiao Yang <[email protected]>
To: <[email protected]>
Cc: <[email protected]>, <[email protected]>, <[email protected]>,

	<[email protected]>, <[email protected]>,
	Xiao Yang <[email protected]>

Subject: [PATCH] RDMA/rxe: Zero out index member of struct rxe_queue
Date: Fri, 20 Aug 2021 19:15:09 +0800 [thread overview]
Message-ID: <[email protected]> (raw)

1) New index member of struct rxe_queue is introduced but not zeroed so the initial value of index may be random.
2) Current index is not masked off to index_mask.
In such case, producer_addr() and consumer_addr() will get an invalid address by the random index and then accessing the invalid address triggers the following panic:
"BUG: unable to handle page fault for address: ffff9ae2c07a1414"

Fix the issue by using kzalloc() to zero out index member.

Fixes 5bcf5a59c41e ("RDMA/rxe: Protext kernel index from user space")
---
 drivers/infiniband/sw/rxe/rxe_queue.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
drivers/infiniband/sw/rxe/rxe_queue.c +1 -1
index 85b812586ed4..72d95398e604 100644
@@ -63,7 +63,7 @@ struct rxe_queue *rxe_queue_init(struct rxe_dev *rxe, int *num_elem,
 	if (*num_elem < 0)
 		goto err1;
 
-	q = kmalloc(sizeof(*q), GFP_KERNEL);
+	q = kzalloc(sizeof(*q), GFP_KERNEL);
 	if (!q)
 		goto err1;
 
Signature · 8 lines
2.25.1

If kmalloc returns a dirty block of memory you could get random values in the q index which could
easily give a page fault. Once the rxe driver writes a new value it will be masked before storing
and should always be in the allocated buffer. I am not seeing this error perhaps because I am
running in a VM. I just don't know. It should be added to the other fixes.

Bob
Tested-by
#19 lore raw
Quoted 11 lines · Bob Pearson
Fixes 5bcf5a59c41e ("RDMA/rxe: Protext kernel index from user space")
---
  drivers/infiniband/sw/rxe/rxe_queue.c | 2 +-
  1 file changed, 1 insertion(+), 1 deletion(-)
drivers/infiniband/sw/rxe/rxe_queue.c +0 -0
index 85b812586ed4..72d95398e604 100644

drivers/infiniband/sw/rxe/rxe_queue.c struct rxe_queue *rxe_queue_init(struct rxe_dev *rxe, int *num_elem, · line 68

Quoted 7 lines
drivers/infiniband/sw/rxe/rxe_queue.c +1 -1
@@ -63,7 +63,7 @@ struct rxe_queue *rxe_queue_init(struct rxe_dev *rxe, int *num_elem,
  	if (*num_elem < 0)
  		goto err1;
 
-	q = kmalloc(sizeof(*q), GFP_KERNEL);
+	q = kzalloc(sizeof(*q), GFP_KERNEL);
  	if (!q)
  		goto err1;

Hi Bob,

If I rebase this patch series on top of kernel v5.15-rc1 then the srp tests from the blktests suite pass. Kernel v5.15-rc1 includes the above patch. Feel free to add the following to this patch series:

Thanks,

Bart.

#20 lore raw
Quoted 21 lines · Bart Van Assche
Quoted 11 lines · Bob Pearson
Fixes 5bcf5a59c41e ("RDMA/rxe: Protext kernel index from user space")

---
  drivers/infiniband/sw/rxe/rxe_queue.c | 2 +-

  1 file changed, 1 insertion(+), 1 deletion(-)
drivers/infiniband/sw/rxe/rxe_queue.c +0 -0
index 85b812586ed4..72d95398e604 100644

drivers/infiniband/sw/rxe/rxe_queue.c struct rxe_queue *rxe_queue_init(struct rxe_dev *rxe, int *num_elem, · line 68

Quoted 6 lines
drivers/infiniband/sw/rxe/rxe_queue.c +1 -0
@@ -63,7 +63,7 @@ struct rxe_queue *rxe_queue_init(struct rxe_dev *rxe, int *num_elem,
       if (*num_elem < 0)
           goto err1;
   -    q = kmalloc(sizeof(*q), GFP_KERNEL);
+    q = kzalloc(sizeof(*q), GFP_KERNEL);
       if (!q)
           goto err1;

Hi Bob,

Quoted 7 lines

If I rebase this patch series on top of kernel v5.15-rc1 then the srp tests from the blktests suite pass. Kernel v5.15-rc1 includes the above patch. Feel free to add the following to this patch series:

Thanks,

Bart.

Sadly, I have been trying to resolve the note from Shaib Rao who was trying to make rping work. His solution was not correct but it led to a can of worms. The kernel verbs consumer APIs were all using the same APIs from rxe_queue.h to manipulate the client ends of the queues but that was totally incorrect. These are written from the POV of the driver and use the private index which is not supposed to be visible to users of the queues. A whole day later I think I have that one about fixed. So I will be resubmitting the series again in the morning. Its all just memory barriers so it may not affect you.

Bob

#21 lore raw
In reply to Bart Van Assche
Quoted 65 lines · Bart Van Assche
Quoted 22 lines · Bob Pearson

OK I checked out the kernel with the SHA number above and applied the patch series and rebuilt and reinstalled the kernel. I checked out v36.0 of rdma-core and rebuilt that. rdma is version 5.9.0 but I doubt that will have any effect. My startup script is

     export LD_LIBRARY_PATH=/home/bob/src/rdma-core/build/lib/:/usr/local/lib:/usr/lib

     sudo ip link set dev enp0s3 mtu 8500

     sudo ip addr add dev enp0s3 fe80::0a00:27ff:fe94:8a69/64

     sudo rdma link add rxe0 type rxe netdev enp0s3

I am running on a Virtualbox VM instance of Ubuntu 21.04 with 20 cores and 8GB of RAM.

The test looks like

     sudo ./check -q srp/001

Quoted 7 lines

     srp/001 (Create and remove LUNs)                             [passed]

         runtime  1.174s  ...  1.236s

There were no issues.

Any guesses what else to look at?

The test I ran is different. I did not run any of the ip link / ip addr / rdma link commands since the blktests scripts already run the rdma link command. The bug I reported in my previous email is reproducible and triggers a VM halt.

Are we using the same kernel config? I attached my kernel config to my previous email. The source code location of the crash address is as follows:

(gdb) list *(rxe_completer+0x96d)
0x228d is in rxe_completer (drivers/infiniband/sw/rxe/rxe_comp.c:149). 144              */
145             wqe = queue_head(qp->sq.queue, QUEUE_TYPE_FROM_CLIENT); 146             *wqe_p = wqe;
147
148             /* no WQE or requester has not started it yet */ 149             if (!wqe || wqe->state == wqe_state_posted) 150                     return pkt ? COMPST_DONE : COMPST_EXIT; 151
152             /* WQE does not require an ack */
153             if (wqe->state == wqe_state_done)

The disassembly output is as follows:

drivers/infiniband/sw/rxe/rxe_comp.c:
149             if (!wqe || wqe->state == wqe_state_posted) 0x0000000000002277 <+2391>:  test   %r12,%r12 0x000000000000227a <+2394>:  je     0x2379 <rxe_completer+2649> 0x0000000000002280 <+2400>:  lea    0x94(%r12),%rdi 0x0000000000002288 <+2408>:  call   0x228d <rxe_completer+2413> 0x000000000000228d <+2413>:  mov    0x94(%r12),%eax 0x0000000000002295 <+2421>:  test   %eax,%eax 0x0000000000002297 <+2423>:  je     0x237c <rxe_completer+2652>

Quoted 7 lines

So the instruction that triggers the crash is "mov 0x94(%r12),%eax". Does consumer_addr() perhaps return an invalid address under certain circumstances?

Thanks,

Bart.

By the way I did rebuild the kernel with your config file. No change. - Bob