Skip to contentSkip to conversation

RDMA/rxe: Implement support for nonlinear packets

PATCH In review Bob Pearson 3 participants 20 messages Jul 27, 2023 – Aug 15, 2023

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

v3 cover RDMA/rxe: Implement support for nonlinear packets

This patch set is a revised version of an older set which implements support for nonlinear or fragmented packets. This avoids extra copies in both the send and receive paths and gives significant performance improvement for large messages such as are used in storage applications.

This patch set has been heavily tested at large system scale and demonstrated a 2X improvement in file system read performance on a 200 Gb/sec network.

The patch set is rebased to the current for-next branch with the following previous patch sets applied:
RDMA/rxe: Fix incomplete state save in rxe_requester RDMA/rxe: Misc fixes and cleanups
Enable rcu locking of verbs objects
RDMA/rxe: Misc cleanups

Bob Pearson (10):
RDMA/rxe: Add sg fragment ops
RDMA/rxe: Extend rxe_mr_copy to support skb frags RDMA/rxe: Extend copy_data to support skb frags RDMA/rxe: Extend rxe_init_packet() to support frags RDMA/rxe: Extend rxe_icrc.c to support frags RDMA/rxe: Extend rxe_init_req_packet() for frags RDMA/rxe: Extend response packets for frags RDMA/rxe: Extend send/write_data_in() for frags RDMA/rxe: Extend do_read() in rxe_comp.c for frags RDMA/rxe: Enable sg code in rxe

 drivers/infiniband/sw/rxe/rxe.c        |   5 +
 drivers/infiniband/sw/rxe/rxe.h        |   3 +
 drivers/infiniband/sw/rxe/rxe_comp.c   |  46 +++-
 drivers/infiniband/sw/rxe/rxe_icrc.c   |  65 ++++-
 drivers/infiniband/sw/rxe/rxe_loc.h    |  27 +-
 drivers/infiniband/sw/rxe/rxe_mr.c     | 348 +++++++++++++++++++------
 drivers/infiniband/sw/rxe/rxe_net.c    | 109 +++++++-
 drivers/infiniband/sw/rxe/rxe_opcode.c |   2 +
 drivers/infiniband/sw/rxe/rxe_recv.c   |   1 +
 drivers/infiniband/sw/rxe/rxe_req.c    |  88 ++++++-
 drivers/infiniband/sw/rxe/rxe_resp.c   | 172 +++++++-----
 drivers/infiniband/sw/rxe/rxe_verbs.h  |   8 +-
 12 files changed, 672 insertions(+), 202 deletions(-)

base-commit: 693e1cdebb50d2aa67406411ca6d5be195d62771
prerequisite-patch-id: c3994e7a93e37e0ce4f50e0c768f3c1a0059a02f
prerequisite-patch-id: 48e13f6ccb560fdeacbd20aaf6696782c23d1190
prerequisite-patch-id: da75fb8eaa863df840e7b392b5048fcc72b0bef3
prerequisite-patch-id: d0877649e2edaf00585a0a6a80391fe0d7bbc13b
prerequisite-patch-id: 6495b1d1f664f8ab91ed9ef9d2ca5b3b27d7df35
prerequisite-patch-id: a6367b8fedd0d8999139c8b857ebbd3ce5c72245
prerequisite-patch-id: 78c95e90a5e49b15b7af8ef57130739c143e88b5
prerequisite-patch-id: 7c65a01066c0418de6897bc8b5f44d078d21b0ec
prerequisite-patch-id: 8ab09f93c23c7875e56c597e69236c30464723b6
prerequisite-patch-id: ca9d84b34873b49048e42fb4c13a2a097c215c46
prerequisite-patch-id: 0f6a587501c8246e1185dfd0cbf5e2044c5f9b13
prerequisite-patch-id: 5246df93137429916d76e75b9a13a4ad5ceb0bad
prerequisite-patch-id: 41b0e4150794dd914d9fcb4cd106fe4cf4227611
prerequisite-patch-id: 02b08ec037bc35b9c7771640c89c66504cdf38a6
prerequisite-patch-id: dfccc06c16454d7fe8e6fcba064d4e471d314666
prerequisite-patch-id: 7459a6e5cdd46efd53ba27f9b3e9028af6e0863b
prerequisite-patch-id: 36d49f9303f5cb276a5601c1ab568eea6eca7d3a
prerequisite-patch-id: 6359a681e40832694f81ca003c10e5327996bf7d
prerequisite-patch-id: 558175db657f374dbd3e0a57ac4c5fb77a56b6c6
prerequisite-patch-id: d6b811de06c8900be5840dd29715161d26db66cf

Signature · 1 line
2.39.2
#1 lore raw

PATCH v3 1/10 RDMA/rxe: Add sg fragment ops

Rename rxe_mr_copy_dir to rxe_mr_copy_op. This allows adding new fragment operations later.

This is in preparation for supporting fragmented skbs.

---
 drivers/infiniband/sw/rxe/rxe_comp.c  |  4 ++--
 drivers/infiniband/sw/rxe/rxe_loc.h   |  4 ++--
 drivers/infiniband/sw/rxe/rxe_mr.c    | 22 +++++++++++-----------
 drivers/infiniband/sw/rxe/rxe_req.c   |  2 +-
 drivers/infiniband/sw/rxe/rxe_resp.c  |  6 +++---
 drivers/infiniband/sw/rxe/rxe_verbs.h |  6 +++---
 6 files changed, 22 insertions(+), 22 deletions(-)
drivers/infiniband/sw/rxe/rxe_comp.c +2 -2
index 5111735aafae..e3f8dfc9b8bf 100644
@@ -368,7 +368,7 @@ static inline enum comp_state do_read(struct rxe_qp *qp,
 
 	ret = copy_data(qp->pd, IB_ACCESS_LOCAL_WRITE,
 			&wqe->dma, payload_addr(pkt),
-			payload_size(pkt), RXE_TO_MR_OBJ);
+			payload_size(pkt), RXE_COPY_TO_MR);
 	if (ret) {
 		wqe->status = IB_WC_LOC_PROT_ERR;
 		return COMPST_ERROR;
@@ -390,7 +390,7 @@ static inline enum comp_state do_atomic(struct rxe_qp *qp,
 
 	ret = copy_data(qp->pd, IB_ACCESS_LOCAL_WRITE,
 			&wqe->dma, &atomic_orig,
-			sizeof(u64), RXE_TO_MR_OBJ);
+			sizeof(u64), RXE_COPY_TO_MR);
 	if (ret) {
 		wqe->status = IB_WC_LOC_PROT_ERR;
 		return COMPST_ERROR;
drivers/infiniband/sw/rxe/rxe_loc.h +2 -2
index cf38f4dcff78..532026cdd49e 100644
@@ -64,9 +64,9 @@ int rxe_mr_init_user(struct rxe_dev *rxe, u64 start, u64 length, u64 iova,
 int rxe_mr_init_fast(int max_pages, struct rxe_mr *mr);
 int rxe_flush_pmem_iova(struct rxe_mr *mr, u64 iova, unsigned int length);
 int rxe_mr_copy(struct rxe_mr *mr, u64 iova, void *addr,
-		unsigned int length, enum rxe_mr_copy_dir dir);
+		unsigned int length, enum rxe_mr_copy_op op);
 int copy_data(struct rxe_pd *pd, int access, struct rxe_dma_info *dma,
-	      void *addr, int length, enum rxe_mr_copy_dir dir);
+	      void *addr, int length, enum rxe_mr_copy_op op);
 int rxe_map_mr_sg(struct ib_mr *ibmr, struct scatterlist *sg,
 		  int sg_nents, unsigned int *sg_offset);
 int rxe_mr_do_atomic_op(struct rxe_mr *mr, u64 iova, int opcode,
drivers/infiniband/sw/rxe/rxe_mr.c +11 -11
index f54042e9aeb2..812c85cad463 100644
@@ -243,7 +243,7 @@ int rxe_map_mr_sg(struct ib_mr *ibmr, struct scatterlist *sgl,
 }
 
 static int rxe_mr_copy_xarray(struct rxe_mr *mr, u64 iova, void *addr,
-			      unsigned int length, enum rxe_mr_copy_dir dir)
+			      unsigned int length, enum rxe_mr_copy_op op)
 {
 	unsigned int page_offset = rxe_mr_iova_to_page_offset(mr, iova);
 	unsigned long index = rxe_mr_iova_to_index(mr, iova);
@@ -259,7 +259,7 @@ static int rxe_mr_copy_xarray(struct rxe_mr *mr, u64 iova, void *addr,
 		bytes = min_t(unsigned int, length,
 				mr_page_size(mr) - page_offset);
 		va = kmap_local_page(page);
-		if (dir == RXE_FROM_MR_OBJ)
+		if (op == RXE_COPY_FROM_MR)
 			memcpy(addr, va + page_offset, bytes);
 		else
 			memcpy(va + page_offset, addr, bytes);
@@ -275,7 +275,7 @@ static int rxe_mr_copy_xarray(struct rxe_mr *mr, u64 iova, void *addr,
 }
 
 static void rxe_mr_copy_dma(struct rxe_mr *mr, u64 dma_addr, void *addr,
-			    unsigned int length, enum rxe_mr_copy_dir dir)
+			    unsigned int length, enum rxe_mr_copy_op op)
 {
 	unsigned int page_offset = dma_addr & (PAGE_SIZE - 1);
 	unsigned int bytes;
@@ -288,10 +288,10 @@ static void rxe_mr_copy_dma(struct rxe_mr *mr, u64 dma_addr, void *addr,
 				PAGE_SIZE - page_offset);
 		va = kmap_local_page(page);
 
-		if (dir == RXE_TO_MR_OBJ)
-			memcpy(va + page_offset, addr, bytes);
-		else
+		if (op == RXE_COPY_FROM_MR)
 			memcpy(addr, va + page_offset, bytes);
+		else
+			memcpy(va + page_offset, addr, bytes);
 
 		kunmap_local(va);
 		page_offset = 0;
@@ -302,7 +302,7 @@ static void rxe_mr_copy_dma(struct rxe_mr *mr, u64 dma_addr, void *addr,
 }
 
 int rxe_mr_copy(struct rxe_mr *mr, u64 iova, void *addr,
-		unsigned int length, enum rxe_mr_copy_dir dir)
+		unsigned int length, enum rxe_mr_copy_op op)
 {
 	int err;
 
@@ -313,7 +313,7 @@ int rxe_mr_copy(struct rxe_mr *mr, u64 iova, void *addr,
 		return -EINVAL;
 
 	if (mr->ibmr.type == IB_MR_TYPE_DMA) {
-		rxe_mr_copy_dma(mr, iova, addr, length, dir);
+		rxe_mr_copy_dma(mr, iova, addr, length, op);
 		return 0;
 	}
 
@@ -323,7 +323,7 @@ int rxe_mr_copy(struct rxe_mr *mr, u64 iova, void *addr,
 		return err;
 	}
 
-	return rxe_mr_copy_xarray(mr, iova, addr, length, dir);
+	return rxe_mr_copy_xarray(mr, iova, addr, length, op);
 }
 
 /* copy data in or out of a wqe, i.e. sg list
@@ -335,7 +335,7 @@ int copy_data(
 	struct rxe_dma_info	*dma,
 	void			*addr,
 	int			length,
-	enum rxe_mr_copy_dir	dir)
+	enum rxe_mr_copy_op	op)
 {
 	int			bytes;
 	struct rxe_sge		*sge	= &dma->sge[dma->cur_sge];
@@ -395,7 +395,7 @@ int copy_data(
 
 		if (bytes > 0) {
 			iova = sge->addr + offset;
-			err = rxe_mr_copy(mr, iova, addr, bytes, dir);
+			err = rxe_mr_copy(mr, iova, addr, bytes, op);
 			if (err)
 				goto err2;
 
drivers/infiniband/sw/rxe/rxe_req.c +1 -1
index 51b781ac2844..f3653234cf32 100644
@@ -327,7 +327,7 @@ static int rxe_init_payload(struct rxe_qp *qp, struct rxe_send_wqe *wqe,
 		wqe->dma.sge_offset += payload;
 	} else {
 		err = copy_data(qp->pd, 0, &wqe->dma, payload_addr(pkt),
-				payload, RXE_FROM_MR_OBJ);
+				payload, RXE_COPY_FROM_MR);
 	}
 
 	return err;
drivers/infiniband/sw/rxe/rxe_resp.c +3 -3
index 8a25c56dfd86..596615c515ad 100644
@@ -565,7 +565,7 @@ static enum resp_states send_data_in(struct rxe_qp *qp, void *data_addr,
 	int err;
 
 	err = copy_data(qp->pd, IB_ACCESS_LOCAL_WRITE, &qp->resp.wqe->dma,
-			data_addr, data_len, RXE_TO_MR_OBJ);
+			data_addr, data_len, RXE_COPY_TO_MR);
 	if (unlikely(err))
 		return (err == -ENOSPC) ? RESPST_ERR_LENGTH
 					: RESPST_ERR_MALFORMED_WQE;
@@ -581,7 +581,7 @@ static enum resp_states write_data_in(struct rxe_qp *qp,
 	int data_len = payload_size(pkt);
 
 	err = rxe_mr_copy(qp->resp.mr, qp->resp.va + qp->resp.offset,
-			  payload_addr(pkt), data_len, RXE_TO_MR_OBJ);
+			  payload_addr(pkt), data_len, RXE_COPY_TO_MR);
 	if (err) {
 		rc = RESPST_ERR_RKEY_VIOLATION;
 		goto out;
@@ -928,7 +928,7 @@ static enum resp_states read_reply(struct rxe_qp *qp,
 	}
 
 	err = rxe_mr_copy(mr, res->read.va, payload_addr(&ack_pkt),
-			  payload, RXE_FROM_MR_OBJ);
+			  payload, RXE_COPY_FROM_MR);
 	if (err) {
 		kfree_skb(skb);
 		state = RESPST_ERR_RKEY_VIOLATION;
drivers/infiniband/sw/rxe/rxe_verbs.h +3 -3
index ccb9d19ffe8a..d9c44bd30da4 100644
@@ -275,9 +275,9 @@ enum rxe_mr_state {
 	RXE_MR_STATE_VALID,
 };
 
-enum rxe_mr_copy_dir {
-	RXE_TO_MR_OBJ,
-	RXE_FROM_MR_OBJ,
+enum rxe_mr_copy_op {
+	RXE_COPY_TO_MR,
+	RXE_COPY_FROM_MR,
 };
 
 enum rxe_mr_lookup_type {
Signature · 1 line
2.39.2
#2 lore raw

Re: [PATCH for-next v3 01/10] RDMA/rxe: Add sg fragment ops

Quoted 2 lines · Bob Pearson

Rename rxe_mr_copy_dir to rxe_mr_copy_op. This allows adding new fragment operations later.

In this commit, only renaming a enum from rxe_mr_copy_dir to rxe_mr_copy_op. And no bug fix and performance enhancing. I am not sure if it is good or not. Please Jason and Leon also check it.

Zhu Yanjun

Quoted 202 lines

This is in preparation for supporting fragmented skbs.

---
  drivers/infiniband/sw/rxe/rxe_comp.c  |  4 ++--
  drivers/infiniband/sw/rxe/rxe_loc.h   |  4 ++--
  drivers/infiniband/sw/rxe/rxe_mr.c    | 22 +++++++++++-----------
  drivers/infiniband/sw/rxe/rxe_req.c   |  2 +-
  drivers/infiniband/sw/rxe/rxe_resp.c  |  6 +++---
  drivers/infiniband/sw/rxe/rxe_verbs.h |  6 +++---
  6 files changed, 22 insertions(+), 22 deletions(-)
drivers/infiniband/sw/rxe/rxe_comp.c +2 -2
index 5111735aafae..e3f8dfc9b8bf 100644
@@ -368,7 +368,7 @@ static inline enum comp_state do_read(struct rxe_qp *qp,
 
  	ret = copy_data(qp->pd, IB_ACCESS_LOCAL_WRITE,
  			&wqe->dma, payload_addr(pkt),
-			payload_size(pkt), RXE_TO_MR_OBJ);
+			payload_size(pkt), RXE_COPY_TO_MR);
  	if (ret) {
  		wqe->status = IB_WC_LOC_PROT_ERR;
  		return COMPST_ERROR;
@@ -390,7 +390,7 @@ static inline enum comp_state do_atomic(struct rxe_qp *qp,
 
  	ret = copy_data(qp->pd, IB_ACCESS_LOCAL_WRITE,
  			&wqe->dma, &atomic_orig,
-			sizeof(u64), RXE_TO_MR_OBJ);
+			sizeof(u64), RXE_COPY_TO_MR);
  	if (ret) {
  		wqe->status = IB_WC_LOC_PROT_ERR;
  		return COMPST_ERROR;
drivers/infiniband/sw/rxe/rxe_loc.h +2 -2
index cf38f4dcff78..532026cdd49e 100644
@@ -64,9 +64,9 @@ int rxe_mr_init_user(struct rxe_dev *rxe, u64 start, u64 length, u64 iova,
  int rxe_mr_init_fast(int max_pages, struct rxe_mr *mr);
  int rxe_flush_pmem_iova(struct rxe_mr *mr, u64 iova, unsigned int length);
  int rxe_mr_copy(struct rxe_mr *mr, u64 iova, void *addr,
-		unsigned int length, enum rxe_mr_copy_dir dir);
+		unsigned int length, enum rxe_mr_copy_op op);
  int copy_data(struct rxe_pd *pd, int access, struct rxe_dma_info *dma,
-	      void *addr, int length, enum rxe_mr_copy_dir dir);
+	      void *addr, int length, enum rxe_mr_copy_op op);
  int rxe_map_mr_sg(struct ib_mr *ibmr, struct scatterlist *sg,
  		  int sg_nents, unsigned int *sg_offset);
  int rxe_mr_do_atomic_op(struct rxe_mr *mr, u64 iova, int opcode,
drivers/infiniband/sw/rxe/rxe_mr.c +11 -11
index f54042e9aeb2..812c85cad463 100644
@@ -243,7 +243,7 @@ int rxe_map_mr_sg(struct ib_mr *ibmr, struct scatterlist *sgl,
  }
 
  static int rxe_mr_copy_xarray(struct rxe_mr *mr, u64 iova, void *addr,
-			      unsigned int length, enum rxe_mr_copy_dir dir)
+			      unsigned int length, enum rxe_mr_copy_op op)
  {
  	unsigned int page_offset = rxe_mr_iova_to_page_offset(mr, iova);
  	unsigned long index = rxe_mr_iova_to_index(mr, iova);
@@ -259,7 +259,7 @@ static int rxe_mr_copy_xarray(struct rxe_mr *mr, u64 iova, void *addr,
  		bytes = min_t(unsigned int, length,
  				mr_page_size(mr) - page_offset);
  		va = kmap_local_page(page);
-		if (dir == RXE_FROM_MR_OBJ)
+		if (op == RXE_COPY_FROM_MR)
  			memcpy(addr, va + page_offset, bytes);
  		else
  			memcpy(va + page_offset, addr, bytes);
@@ -275,7 +275,7 @@ static int rxe_mr_copy_xarray(struct rxe_mr *mr, u64 iova, void *addr,
  }
 
  static void rxe_mr_copy_dma(struct rxe_mr *mr, u64 dma_addr, void *addr,
-			    unsigned int length, enum rxe_mr_copy_dir dir)
+			    unsigned int length, enum rxe_mr_copy_op op)
  {
  	unsigned int page_offset = dma_addr & (PAGE_SIZE - 1);
  	unsigned int bytes;
@@ -288,10 +288,10 @@ static void rxe_mr_copy_dma(struct rxe_mr *mr, u64 dma_addr, void *addr,
  				PAGE_SIZE - page_offset);
  		va = kmap_local_page(page);
 
-		if (dir == RXE_TO_MR_OBJ)
-			memcpy(va + page_offset, addr, bytes);
-		else
+		if (op == RXE_COPY_FROM_MR)
  			memcpy(addr, va + page_offset, bytes);
+		else
+			memcpy(va + page_offset, addr, bytes);
 
  		kunmap_local(va);
  		page_offset = 0;
@@ -302,7 +302,7 @@ static void rxe_mr_copy_dma(struct rxe_mr *mr, u64 dma_addr, void *addr,
  }
 
  int rxe_mr_copy(struct rxe_mr *mr, u64 iova, void *addr,
-		unsigned int length, enum rxe_mr_copy_dir dir)
+		unsigned int length, enum rxe_mr_copy_op op)
  {
  	int err;
 
@@ -313,7 +313,7 @@ int rxe_mr_copy(struct rxe_mr *mr, u64 iova, void *addr,
  		return -EINVAL;
 
  	if (mr->ibmr.type == IB_MR_TYPE_DMA) {
-		rxe_mr_copy_dma(mr, iova, addr, length, dir);
+		rxe_mr_copy_dma(mr, iova, addr, length, op);
  		return 0;
  	}
 
@@ -323,7 +323,7 @@ int rxe_mr_copy(struct rxe_mr *mr, u64 iova, void *addr,
  		return err;
  	}
 
-	return rxe_mr_copy_xarray(mr, iova, addr, length, dir);
+	return rxe_mr_copy_xarray(mr, iova, addr, length, op);
  }
 
  /* copy data in or out of a wqe, i.e. sg list
@@ -335,7 +335,7 @@ int copy_data(
  	struct rxe_dma_info	*dma,
  	void			*addr,
  	int			length,
-	enum rxe_mr_copy_dir	dir)
+	enum rxe_mr_copy_op	op)
  {
  	int			bytes;
  	struct rxe_sge		*sge	= &dma->sge[dma->cur_sge];
@@ -395,7 +395,7 @@ int copy_data(
 
  		if (bytes > 0) {
  			iova = sge->addr + offset;
-			err = rxe_mr_copy(mr, iova, addr, bytes, dir);
+			err = rxe_mr_copy(mr, iova, addr, bytes, op);
  			if (err)
  				goto err2;
 
drivers/infiniband/sw/rxe/rxe_req.c +1 -1
index 51b781ac2844..f3653234cf32 100644
@@ -327,7 +327,7 @@ static int rxe_init_payload(struct rxe_qp *qp, struct rxe_send_wqe *wqe,
  		wqe->dma.sge_offset += payload;
  	} else {
  		err = copy_data(qp->pd, 0, &wqe->dma, payload_addr(pkt),
-				payload, RXE_FROM_MR_OBJ);
+				payload, RXE_COPY_FROM_MR);
  	}
 
  	return err;
drivers/infiniband/sw/rxe/rxe_resp.c +3 -3
index 8a25c56dfd86..596615c515ad 100644
@@ -565,7 +565,7 @@ static enum resp_states send_data_in(struct rxe_qp *qp, void *data_addr,
  	int err;
 
  	err = copy_data(qp->pd, IB_ACCESS_LOCAL_WRITE, &qp->resp.wqe->dma,
-			data_addr, data_len, RXE_TO_MR_OBJ);
+			data_addr, data_len, RXE_COPY_TO_MR);
  	if (unlikely(err))
  		return (err == -ENOSPC) ? RESPST_ERR_LENGTH
  					: RESPST_ERR_MALFORMED_WQE;
@@ -581,7 +581,7 @@ static enum resp_states write_data_in(struct rxe_qp *qp,
  	int data_len = payload_size(pkt);
 
  	err = rxe_mr_copy(qp->resp.mr, qp->resp.va + qp->resp.offset,
-			  payload_addr(pkt), data_len, RXE_TO_MR_OBJ);
+			  payload_addr(pkt), data_len, RXE_COPY_TO_MR);
  	if (err) {
  		rc = RESPST_ERR_RKEY_VIOLATION;
  		goto out;
@@ -928,7 +928,7 @@ static enum resp_states read_reply(struct rxe_qp *qp,
  	}
 
  	err = rxe_mr_copy(mr, res->read.va, payload_addr(&ack_pkt),
-			  payload, RXE_FROM_MR_OBJ);
+			  payload, RXE_COPY_FROM_MR);
  	if (err) {
  		kfree_skb(skb);
  		state = RESPST_ERR_RKEY_VIOLATION;
drivers/infiniband/sw/rxe/rxe_verbs.h +3 -3
index ccb9d19ffe8a..d9c44bd30da4 100644
@@ -275,9 +275,9 @@ enum rxe_mr_state {
  	RXE_MR_STATE_VALID,
  };
 
-enum rxe_mr_copy_dir {
-	RXE_TO_MR_OBJ,
-	RXE_FROM_MR_OBJ,
+enum rxe_mr_copy_op {
+	RXE_COPY_TO_MR,
+	RXE_COPY_FROM_MR,
  };
 
  enum rxe_mr_lookup_type {
#3 lore raw

Re: [PATCH for-next v3 01/10] RDMA/rxe: Add sg fragment ops

Quoted 10 lines · Zhu Yanjun
Quoted 2 lines · Bob Pearson

Rename rxe_mr_copy_dir to rxe_mr_copy_op. This allows adding new fragment operations later.

In this commit, only renaming a enum from rxe_mr_copy_dir to rxe_mr_copy_op. And no bug fix and performance enhancing. I am not sure if it is good or not. Please Jason and Leon also check it.

Zhu Yanjun

Quoted 2 lines

This is in preparation for supporting fragmented skbs.

Read the next one.

Quoted 181 lines
Quoted 180 lines

---
  drivers/infiniband/sw/rxe/rxe_comp.c  |  4 ++-- drivers/infiniband/sw/rxe/rxe_loc.h   |  4 ++-- drivers/infiniband/sw/rxe/rxe_mr.c    | 22 +++++++++++----------- drivers/infiniband/sw/rxe/rxe_req.c   |  2 +- drivers/infiniband/sw/rxe/rxe_resp.c  |  6 +++--- drivers/infiniband/sw/rxe/rxe_verbs.h |  6 +++---

  6 files changed, 22 insertions(+), 22 deletions(-)
drivers/infiniband/sw/rxe/rxe_comp.c +2 -2
index 5111735aafae..e3f8dfc9b8bf 100644
@@ -368,7 +368,7 @@ static inline enum comp_state do_read(struct rxe_qp *qp,
         ret = copy_data(qp->pd, IB_ACCESS_LOCAL_WRITE,
               &wqe->dma, payload_addr(pkt),
-            payload_size(pkt), RXE_TO_MR_OBJ);
+            payload_size(pkt), RXE_COPY_TO_MR);
       if (ret) {
           wqe->status = IB_WC_LOC_PROT_ERR;
           return COMPST_ERROR;
@@ -390,7 +390,7 @@ static inline enum comp_state do_atomic(struct rxe_qp *qp,
         ret = copy_data(qp->pd, IB_ACCESS_LOCAL_WRITE,
               &wqe->dma, &atomic_orig,
-            sizeof(u64), RXE_TO_MR_OBJ);
+            sizeof(u64), RXE_COPY_TO_MR);
       if (ret) {
           wqe->status = IB_WC_LOC_PROT_ERR;
           return COMPST_ERROR;
drivers/infiniband/sw/rxe/rxe_loc.h +2 -2
index cf38f4dcff78..532026cdd49e 100644
@@ -64,9 +64,9 @@ int rxe_mr_init_user(struct rxe_dev *rxe, u64 start, u64 length, u64 iova,
   int rxe_mr_init_fast(int max_pages, struct rxe_mr *mr);
   int rxe_flush_pmem_iova(struct rxe_mr *mr, u64 iova, unsigned int length);
   int rxe_mr_copy(struct rxe_mr *mr, u64 iova, void *addr,
-        unsigned int length, enum rxe_mr_copy_dir dir);
+        unsigned int length, enum rxe_mr_copy_op op);
   int copy_data(struct rxe_pd *pd, int access, struct rxe_dma_info *dma,
-          void *addr, int length, enum rxe_mr_copy_dir dir);
+          void *addr, int length, enum rxe_mr_copy_op op);
   int rxe_map_mr_sg(struct ib_mr *ibmr, struct scatterlist *sg,
             int sg_nents, unsigned int *sg_offset);
   int rxe_mr_do_atomic_op(struct rxe_mr *mr, u64 iova, int opcode,
drivers/infiniband/sw/rxe/rxe_mr.c +2 -2
index f54042e9aeb2..812c85cad463 100644
@@ -243,7 +243,7 @@ int rxe_map_mr_sg(struct ib_mr *ibmr, struct scatterlist *sgl,
   }
     static int rxe_mr_copy_xarray(struct rxe_mr *mr, u64 iova, void *addr,
-                  unsigned int length, enum rxe_mr_copy_dir dir)
+                  unsigned int length, enum rxe_mr_copy_op op)
   {
       unsigned int page_offset = rxe_mr_iova_to_page_offset(mr, iova);
       unsigned long index = rxe_mr_iova_to_index(mr, iova);
@@ -259,7 +259,7 @@ static int rxe_mr_copy_xarray(struct rxe_mr *mr, u64 iova, void *addr,
           bytes = min_t(unsigned int, length,
                   mr_page_size(mr) - page_offset);
           va = kmap_local_page(page);
-        if (dir == RXE_FROM_MR_OBJ)
+        if (op == RXE_COPY_FROM_MR)
               memcpy(addr, va + page_offset, bytes);

          else
              memcpy(va + page_offset, addr, bytes);

drivers/infiniband/sw/rxe/rxe_mr.c +5 -4
@@ -275,7 +275,7 @@ static int rxe_mr_copy_xarray(struct rxe_mr *mr, u64 iova, void *addr,
   }
     static void rxe_mr_copy_dma(struct rxe_mr *mr, u64 dma_addr, void *addr,
-                unsigned int length, enum rxe_mr_copy_dir dir)
+                unsigned int length, enum rxe_mr_copy_op op)
   {
       unsigned int page_offset = dma_addr & (PAGE_SIZE - 1);
       unsigned int bytes;
@@ -288,10 +288,10 @@ static void rxe_mr_copy_dma(struct rxe_mr *mr, u64 dma_addr, void *addr,
                   PAGE_SIZE - page_offset);
           va = kmap_local_page(page);
   -        if (dir == RXE_TO_MR_OBJ)
-            memcpy(va + page_offset, addr, bytes);
-        else
+        if (op == RXE_COPY_FROM_MR)
               memcpy(addr, va + page_offset, bytes);
+        else
+            memcpy(va + page_offset, addr, bytes);
             kunmap_local(va);
           page_offset = 0;
@@ -302,7 +302,7 @@ static void rxe_mr_copy_dma(struct rxe_mr *mr, u64 dma_addr, void *addr,
   }
     int rxe_mr_copy(struct rxe_mr *mr, u64 iova, void *addr,
-        unsigned int length, enum rxe_mr_copy_dir dir)
+        unsigned int length, enum rxe_mr_copy_op op)
   {
       int err;
   @@ -313,7 +313,7 @@ int rxe_mr_copy(struct rxe_mr *mr, u64 iova, void *addr,
           return -EINVAL;

        if (mr->ibmr.type == IB_MR_TYPE_DMA) {
-        rxe_mr_copy_dma(mr, iova, addr, length, dir);
+        rxe_mr_copy_dma(mr, iova, addr, length, op);
          return 0;
      }
  @@ -323,7 +323,7 @@ int rxe_mr_copy(struct rxe_mr *mr, u64 iova, void *addr, return err;
      }
  -    return rxe_mr_copy_xarray(mr, iova, addr, length, dir);
+    return rxe_mr_copy_xarray(mr, iova, addr, length, op);
  }
    /* copy data in or out of a wqe, i.e. sg list

drivers/infiniband/sw/rxe/rxe_mr.c +2 -2
@@ -335,7 +335,7 @@ int copy_data(
       struct rxe_dma_info    *dma,
       void            *addr,
       int            length,
-    enum rxe_mr_copy_dir    dir)
+    enum rxe_mr_copy_op    op)
   {
       int            bytes;
       struct rxe_sge        *sge    = &dma->sge[dma->cur_sge];
@@ -395,7 +395,7 @@ int copy_data(
             if (bytes > 0) {
               iova = sge->addr + offset;
-            err = rxe_mr_copy(mr, iova, addr, bytes, dir);
+            err = rxe_mr_copy(mr, iova, addr, bytes, op);
               if (err)
                   goto err2;

  diff --git a/drivers/infiniband/sw/rxe/rxe_req.c b/drivers/infiniband/sw/rxe/rxe_req.c index 51b781ac2844..f3653234cf32 100644

drivers/infiniband/sw/rxe/rxe_req.c +1 -1
@@ -327,7 +327,7 @@ static int rxe_init_payload(struct rxe_qp *qp, struct rxe_send_wqe *wqe,
           wqe->dma.sge_offset += payload;
       } else {
           err = copy_data(qp->pd, 0, &wqe->dma, payload_addr(pkt),
-                payload, RXE_FROM_MR_OBJ);
+                payload, RXE_COPY_FROM_MR);
       }
         return err;
drivers/infiniband/sw/rxe/rxe_resp.c +1 -1
index 8a25c56dfd86..596615c515ad 100644
@@ -565,7 +565,7 @@ static enum resp_states send_data_in(struct rxe_qp *qp, void *data_addr,
       int err;
         err = copy_data(qp->pd, IB_ACCESS_LOCAL_WRITE, &qp->resp.wqe->dma,
-            data_addr, data_len, RXE_TO_MR_OBJ);
+            data_addr, data_len, RXE_COPY_TO_MR);
       if (unlikely(err))

          return (err == -ENOSPC) ? RESPST_ERR_LENGTH : RESPST_ERR_MALFORMED_WQE;

drivers/infiniband/sw/rxe/rxe_resp.c +2 -2
@@ -581,7 +581,7 @@ static enum resp_states write_data_in(struct rxe_qp *qp,
       int data_len = payload_size(pkt);
         err = rxe_mr_copy(qp->resp.mr, qp->resp.va + qp->resp.offset,
-              payload_addr(pkt), data_len, RXE_TO_MR_OBJ);
+              payload_addr(pkt), data_len, RXE_COPY_TO_MR);
       if (err) {
           rc = RESPST_ERR_RKEY_VIOLATION;
           goto out;
@@ -928,7 +928,7 @@ static enum resp_states read_reply(struct rxe_qp *qp,
       }
         err = rxe_mr_copy(mr, res->read.va, payload_addr(&ack_pkt),
-              payload, RXE_FROM_MR_OBJ);
+              payload, RXE_COPY_FROM_MR);
       if (err) {
           kfree_skb(skb);
           state = RESPST_ERR_RKEY_VIOLATION;
drivers/infiniband/sw/rxe/rxe_verbs.h +3 -2
index ccb9d19ffe8a..d9c44bd30da4 100644
@@ -275,9 +275,9 @@ enum rxe_mr_state {
       RXE_MR_STATE_VALID,
   };
   -enum rxe_mr_copy_dir {
-    RXE_TO_MR_OBJ,
-    RXE_FROM_MR_OBJ,
+enum rxe_mr_copy_op {
+    RXE_COPY_TO_MR,
+    RXE_COPY_FROM_MR,
   };
     enum rxe_mr_lookup_type {
#4 lore raw

PATCH v3 2/10 RDMA/rxe: Extend rxe_mr_copy to support skb frags

In reply to Bob Pearson

rxe_mr_copy() currently supports copying between an mr and a contiguous region of kernel memory.

Rename rxe_mr_copy() to rxe_copy_mr_data().
Extend the operations to support copying between an mr and an skb fragment list. Fixup calls to rxe_mr_copy() to support the new API. Add two APIs rxe_add_frag() and rxe_num_mr_frags() to add a fragment to and skb and count the total number of fragments needed.

---
 drivers/infiniband/sw/rxe/rxe_loc.h   |  10 +-
 drivers/infiniband/sw/rxe/rxe_mr.c    | 170 +++++++++++++++++++++++---
 drivers/infiniband/sw/rxe/rxe_resp.c  |  14 ++-
 drivers/infiniband/sw/rxe/rxe_verbs.h |   2 +
 4 files changed, 173 insertions(+), 23 deletions(-)
drivers/infiniband/sw/rxe/rxe_loc.h +7 -3
index 532026cdd49e..77661e0ccab7 100644
@@ -62,11 +62,15 @@ void rxe_mr_init_dma(int access, struct rxe_mr *mr);
 int rxe_mr_init_user(struct rxe_dev *rxe, u64 start, u64 length, u64 iova,
 		     int access, struct rxe_mr *mr);
 int rxe_mr_init_fast(int max_pages, struct rxe_mr *mr);
-int rxe_flush_pmem_iova(struct rxe_mr *mr, u64 iova, unsigned int length);
-int rxe_mr_copy(struct rxe_mr *mr, u64 iova, void *addr,
-		unsigned int length, enum rxe_mr_copy_op op);
+int rxe_add_frag(struct sk_buff *skb, struct rxe_mr *mr, struct page *page,
+		 unsigned int length, unsigned int offset);
+int rxe_num_mr_frags(struct rxe_mr *mr, u64 iova, unsigned int length);
+int rxe_copy_mr_data(struct sk_buff *skb, struct rxe_mr *mr, u64 iova,
+		     void *addr, unsigned int skb_offset,
+		     unsigned int length, enum rxe_mr_copy_op op);
 int copy_data(struct rxe_pd *pd, int access, struct rxe_dma_info *dma,
 	      void *addr, int length, enum rxe_mr_copy_op op);
+int rxe_flush_pmem_iova(struct rxe_mr *mr, u64 iova, unsigned int length);
 int rxe_map_mr_sg(struct ib_mr *ibmr, struct scatterlist *sg,
 		  int sg_nents, unsigned int *sg_offset);
 int rxe_mr_do_atomic_op(struct rxe_mr *mr, u64 iova, int opcode,
drivers/infiniband/sw/rxe/rxe_mr.c +155 -15

235 lines not on this page · raw message

drivers/infiniband/sw/rxe/rxe_resp.c +9 -5
index 596615c515ad..87d61a462ff5 100644
@@ -576,12 +576,15 @@ static enum resp_states send_data_in(struct rxe_qp *qp, void *data_addr,
 static enum resp_states write_data_in(struct rxe_qp *qp,
 				      struct rxe_pkt_info *pkt)
 {
+	struct sk_buff *skb = PKT_TO_SKB(pkt);
 	enum resp_states rc = RESPST_NONE;
-	int	err;
 	int data_len = payload_size(pkt);
+	int err;
+	int skb_offset = 0;
 
-	err = rxe_mr_copy(qp->resp.mr, qp->resp.va + qp->resp.offset,
-			  payload_addr(pkt), data_len, RXE_COPY_TO_MR);
+	err = rxe_copy_mr_data(skb, qp->resp.mr, qp->resp.va + qp->resp.offset,
+			  payload_addr(pkt), skb_offset, data_len,
+			  RXE_COPY_TO_MR);
 	if (err) {
 		rc = RESPST_ERR_RKEY_VIOLATION;
 		goto out;
@@ -876,6 +879,7 @@ static enum resp_states read_reply(struct rxe_qp *qp,
 	int err;
 	struct resp_res *res = qp->resp.res;
 	struct rxe_mr *mr;
+	unsigned int skb_offset = 0;
 	u8 *pad_addr;
 
 	if (!res) {
@@ -927,8 +931,8 @@ static enum resp_states read_reply(struct rxe_qp *qp,
 		goto err_out;
 	}
 
-	err = rxe_mr_copy(mr, res->read.va, payload_addr(&ack_pkt),
-			  payload, RXE_COPY_FROM_MR);
+	err = rxe_copy_mr_data(skb, mr, res->read.va, payload_addr(&ack_pkt),
+			       skb_offset, payload, RXE_COPY_FROM_MR);
 	if (err) {
 		kfree_skb(skb);
 		state = RESPST_ERR_RKEY_VIOLATION;
drivers/infiniband/sw/rxe/rxe_verbs.h +2 -0
index d9c44bd30da4..89cf50b938c2 100644
@@ -278,6 +278,8 @@ enum rxe_mr_state {
 enum rxe_mr_copy_op {
 	RXE_COPY_TO_MR,
 	RXE_COPY_FROM_MR,
+	RXE_FRAG_TO_MR,
+	RXE_FRAG_FROM_MR,
 };
 
 enum rxe_mr_lookup_type {
Signature · 1 line
2.39.2
#5 lore raw

PATCH v3 3/10 RDMA/rxe: Extend copy_data to support skb frags

In reply to Bob Pearson

copy_data() currently supports copying between an mr and the scatter-gather list of a wqe.

Rename copy_data() to rxe_copy_dma_data().
Extend the operations to support copying between a sg list and an skb fragment list. Fixup calls to copy_data() to support the new API. Add a routine to count the number of skbs required for rxe_copy_dma_data().

---
 drivers/infiniband/sw/rxe/rxe_comp.c |  17 ++-
 drivers/infiniband/sw/rxe/rxe_loc.h  |  10 +-
 drivers/infiniband/sw/rxe/rxe_mr.c   | 175 +++++++++++++++++----------
 drivers/infiniband/sw/rxe/rxe_req.c  |  11 +-
 drivers/infiniband/sw/rxe/rxe_resp.c |   7 +-
 5 files changed, 139 insertions(+), 81 deletions(-)
drivers/infiniband/sw/rxe/rxe_comp.c +11 -6
index e3f8dfc9b8bf..670ee08f6f5a 100644
@@ -364,11 +364,14 @@ static inline enum comp_state do_read(struct rxe_qp *qp,
 				      struct rxe_pkt_info *pkt,
 				      struct rxe_send_wqe *wqe)
 {
+	struct sk_buff *skb = PKT_TO_SKB(pkt);
+	int skb_offset = 0;
 	int ret;
 
-	ret = copy_data(qp->pd, IB_ACCESS_LOCAL_WRITE,
-			&wqe->dma, payload_addr(pkt),
-			payload_size(pkt), RXE_COPY_TO_MR);
+	ret = rxe_copy_dma_data(skb, qp->pd, IB_ACCESS_LOCAL_WRITE,
+				&wqe->dma, payload_addr(pkt),
+				skb_offset, payload_size(pkt),
+				RXE_COPY_TO_MR);
 	if (ret) {
 		wqe->status = IB_WC_LOC_PROT_ERR;
 		return COMPST_ERROR;
@@ -384,13 +387,15 @@ static inline enum comp_state do_atomic(struct rxe_qp *qp,
 					struct rxe_pkt_info *pkt,
 					struct rxe_send_wqe *wqe)
 {
+	struct sk_buff *skb = NULL;
+	int skb_offset = 0;
 	int ret;
 
 	u64 atomic_orig = atmack_orig(pkt);
 
-	ret = copy_data(qp->pd, IB_ACCESS_LOCAL_WRITE,
-			&wqe->dma, &atomic_orig,
-			sizeof(u64), RXE_COPY_TO_MR);
+	ret = rxe_copy_dma_data(skb, qp->pd, IB_ACCESS_LOCAL_WRITE,
+				&wqe->dma, &atomic_orig,
+				skb_offset, sizeof(u64), RXE_COPY_TO_MR);
 	if (ret) {
 		wqe->status = IB_WC_LOC_PROT_ERR;
 		return COMPST_ERROR;
drivers/infiniband/sw/rxe/rxe_loc.h +7 -3
index 77661e0ccab7..fad853199b4d 100644
@@ -68,15 +68,19 @@ int rxe_num_mr_frags(struct rxe_mr *mr, u64 iova, unsigned int length);
 int rxe_copy_mr_data(struct sk_buff *skb, struct rxe_mr *mr, u64 iova,
 		     void *addr, unsigned int skb_offset,
 		     unsigned int length, enum rxe_mr_copy_op op);
-int copy_data(struct rxe_pd *pd, int access, struct rxe_dma_info *dma,
-	      void *addr, int length, enum rxe_mr_copy_op op);
+int rxe_num_dma_frags(const struct rxe_pd *pd, const struct rxe_dma_info *dma,
+		      unsigned int length);
+int rxe_copy_dma_data(struct sk_buff *skb, struct rxe_pd *pd, int access,
+		      struct rxe_dma_info *dma, void *addr,
+		      unsigned int skb_offset, unsigned int length,
+		      enum rxe_mr_copy_op op);
 int rxe_flush_pmem_iova(struct rxe_mr *mr, u64 iova, unsigned int length);
 int rxe_map_mr_sg(struct ib_mr *ibmr, struct scatterlist *sg,
 		  int sg_nents, unsigned int *sg_offset);
 int rxe_mr_do_atomic_op(struct rxe_mr *mr, u64 iova, int opcode,
 			u64 compare, u64 swap_add, u64 *orig_val);
 int rxe_mr_do_atomic_write(struct rxe_mr *mr, u64 iova, u64 value);
-struct rxe_mr *lookup_mr(struct rxe_pd *pd, int access, u32 key,
+struct rxe_mr *lookup_mr(const struct rxe_pd *pd, int access, u32 key,
 			 enum rxe_mr_lookup_type type);
 int mr_check_range(struct rxe_mr *mr, u64 iova, size_t length);
 int advance_dma_data(struct rxe_dma_info *dma, unsigned int length);
drivers/infiniband/sw/rxe/rxe_mr.c +109 -66

221 lines not on this page · raw message

drivers/infiniband/sw/rxe/rxe_req.c +7 -4
index f3653234cf32..525e704c12c2 100644
@@ -315,8 +315,10 @@ static void rxe_init_roce_hdrs(struct rxe_qp *qp, struct rxe_send_wqe *wqe,
 }
 
 static int rxe_init_payload(struct rxe_qp *qp, struct rxe_send_wqe *wqe,
-			    struct rxe_pkt_info *pkt, u32 payload)
+			    struct rxe_pkt_info *pkt, u32 payload,
+			    struct sk_buff *skb)
 {
+	int skb_offset = 0;
 	void *data;
 	int err = 0;
 
@@ -326,8 +328,9 @@ static int rxe_init_payload(struct rxe_qp *qp, struct rxe_send_wqe *wqe,
 		wqe->dma.resid -= payload;
 		wqe->dma.sge_offset += payload;
 	} else {
-		err = copy_data(qp->pd, 0, &wqe->dma, payload_addr(pkt),
-				payload, RXE_COPY_FROM_MR);
+		err = rxe_copy_dma_data(skb, qp->pd, 0, &wqe->dma,
+					payload_addr(pkt), skb_offset,
+					payload, RXE_COPY_FROM_MR);
 	}
 
 	return err;
@@ -379,7 +382,7 @@ static struct sk_buff *rxe_init_req_packet(struct rxe_qp *qp,
 
 	/* init payload if any */
 	if (pkt->mask & RXE_WRITE_OR_SEND_MASK) {
-		err = rxe_init_payload(qp, wqe, pkt, payload);
+		err = rxe_init_payload(qp, wqe, pkt, payload, skb);
 		if (unlikely(err))
 			goto err_out;
 	} else if (pkt->mask & RXE_FLUSH_MASK) {
drivers/infiniband/sw/rxe/rxe_resp.c +5 -2
index 87d61a462ff5..a6c1d67ad943 100644
@@ -562,10 +562,13 @@ static enum resp_states check_rkey(struct rxe_qp *qp,
 static enum resp_states send_data_in(struct rxe_qp *qp, void *data_addr,
 				     int data_len)
 {
+	struct sk_buff *skb = NULL;
+	int skb_offset = 0;
 	int err;
 
-	err = copy_data(qp->pd, IB_ACCESS_LOCAL_WRITE, &qp->resp.wqe->dma,
-			data_addr, data_len, RXE_COPY_TO_MR);
+	err = rxe_copy_dma_data(skb, qp->pd, IB_ACCESS_LOCAL_WRITE,
+				&qp->resp.wqe->dma, data_addr,
+				skb_offset, data_len, RXE_COPY_TO_MR);
 	if (unlikely(err))
 		return (err == -ENOSPC) ? RESPST_ERR_LENGTH
 					: RESPST_ERR_MALFORMED_WQE;
Signature · 1 line
2.39.2
#6 lore raw

PATCH v3 4/10 RDMA/rxe: Extend rxe_init_packet() to support frags

In reply to Bob Pearson

Add a subroutine rxe_can_use_sg() to determine if a packet is a candidate for a fragmented skb. Add a global variable rxe_use_sg to control whether to support nonlinear skbs. Modify rxe_init_packet() to test if the packet should use a fragmented skb. Fixup calls to rxe_init_packet() to use the new API but disable creating nonlinear skbs for now.

---
 drivers/infiniband/sw/rxe/rxe.c      |  5 +++
 drivers/infiniband/sw/rxe/rxe.h      |  3 ++
 drivers/infiniband/sw/rxe/rxe_loc.h  |  4 +-
 drivers/infiniband/sw/rxe/rxe_net.c  | 58 ++++++++++++++++++++++++++--
 drivers/infiniband/sw/rxe/rxe_req.c  |  4 +-
 drivers/infiniband/sw/rxe/rxe_resp.c |  4 +-
 6 files changed, 66 insertions(+), 12 deletions(-)
drivers/infiniband/sw/rxe/rxe.c +5 -0
index 6b55c595f8f8..800e8c0d437d 100644
@@ -13,6 +13,11 @@ MODULE_AUTHOR("Bob Pearson, Frank Zago, John Groves, Kamal Heib");
 MODULE_DESCRIPTION("Soft RDMA transport");
 MODULE_LICENSE("Dual BSD/GPL");
 
+/* if true allow using fragmented skbs */
+bool rxe_use_sg;
+module_param_named(use_sg, rxe_use_sg, bool, 0444);
+MODULE_PARM_DESC(use_sg, "Support skb frags; default false");
+
 /* free resources for a rxe device all objects created for this device must
  * have been destroyed
  */
drivers/infiniband/sw/rxe/rxe.h +3 -0
index 077e3ad8f39a..b334eda62c36 100644
@@ -30,6 +30,9 @@
 #include "rxe_verbs.h"
 #include "rxe_loc.h"
 
+/* if true allow using fragmented skbs */
+extern bool rxe_use_sg;
+
 /*
  * Version 1 and Version 2 are identical on 64 bit machines, but on 32 bit
  * machines Version 2 has a different struct layout.
drivers/infiniband/sw/rxe/rxe_loc.h +2 -2
index fad853199b4d..96b1fb79610a 100644
@@ -97,8 +97,8 @@ struct rxe_mw *rxe_lookup_mw(struct rxe_qp *qp, int access, u32 rkey);
 void rxe_mw_cleanup(struct rxe_pool_elem *elem);
 
 /* rxe_net.c */
-struct sk_buff *rxe_init_packet(struct rxe_dev *rxe, struct rxe_av *av,
-				struct rxe_pkt_info *pkt);
+struct sk_buff *rxe_init_packet(struct rxe_qp *qp, struct rxe_av *av,
+				struct rxe_pkt_info *pkt, bool *is_frag);
 int rxe_prepare(struct rxe_av *av, struct rxe_pkt_info *pkt,
 		struct sk_buff *skb);
 int rxe_xmit_packet(struct rxe_qp *qp, struct rxe_pkt_info *pkt,
drivers/infiniband/sw/rxe/rxe_net.c +54 -4

80 lines not on this page · raw message

drivers/infiniband/sw/rxe/rxe_req.c +1 -3
index 525e704c12c2..491360fef346 100644
@@ -369,14 +369,12 @@ static struct sk_buff *rxe_init_req_packet(struct rxe_qp *qp,
 			pkt->pad + RXE_ICRC_SIZE;
 
 	/* init skb */
-	skb = rxe_init_packet(rxe, av, pkt);
+	skb = rxe_init_packet(qp, av, pkt, NULL);
 	if (unlikely(!skb)) {
 		err = -ENOMEM;
 		goto err_out;
 	}
 
-	pkt->hdr = skb_put(skb, pkt->paylen);
-
 	/* init roce headers */
 	rxe_init_roce_hdrs(qp, wqe, pkt);
 
drivers/infiniband/sw/rxe/rxe_resp.c +1 -3
index a6c1d67ad943..254f2eab8d20 100644
@@ -788,12 +788,10 @@ static struct sk_buff *prepare_ack_packet(struct rxe_qp *qp,
 	ack->paylen = rxe_opcode[opcode].length + payload +
 			ack->pad + RXE_ICRC_SIZE;
 
-	skb = rxe_init_packet(rxe, &qp->pri_av, ack);
+	skb = rxe_init_packet(qp, &qp->pri_av, ack, NULL);
 	if (!skb)
 		return NULL;
 
-	ack->hdr = skb_put(skb, ack->paylen);
-
 	bth_init(ack, opcode, 0, 0, ack->pad, IB_DEFAULT_PKEY_FULL,
 		 qp->attr.dest_qp_num, 0, psn);
 
Signature · 1 line
2.39.2
#7 lore raw

PATCH v3 5/10 RDMA/rxe: Extend rxe_icrc.c to support frags

In reply to Bob Pearson

Extend the subroutines rxe_icrc_generate() and rxe_icrc_check() to support skb frags.

---
 drivers/infiniband/sw/rxe/rxe_icrc.c | 65 ++++++++++++++++++++++++----
 drivers/infiniband/sw/rxe/rxe_net.c  | 51 +++++++++++++++++-----
 drivers/infiniband/sw/rxe/rxe_recv.c |  1 +
 3 files changed, 98 insertions(+), 19 deletions(-)
drivers/infiniband/sw/rxe/rxe_icrc.c +56 -9
index c9aa0995e900..393391863350 100644
@@ -63,7 +63,7 @@ static __be32 rxe_crc32(struct rxe_dev *rxe, __be32 crc, void *next, size_t len)
 
 /**
  * rxe_icrc_hdr() - Compute the partial ICRC for the network and transport
- *		  headers of a packet.
+ *		    headers of a packet.
  * @skb: packet buffer
  * @pkt: packet information
  *
@@ -129,6 +129,56 @@ static __be32 rxe_icrc_hdr(struct sk_buff *skb, struct rxe_pkt_info *pkt)
 	return crc;
 }
 
+/**
+ * rxe_icrc_payload() - Compute the ICRC for a packet payload and also
+ *			compute the address of the icrc in the packet.
+ * @skb: packet buffer
+ * @pkt: packet information
+ * @icrc: current icrc i.e. including headers
+ * @icrcp: returned pointer to icrc in skb
+ *
+ * Return: 0 if the values match else an error
+ */
+static __be32 rxe_icrc_payload(struct sk_buff *skb, struct rxe_pkt_info *pkt,
+			       __be32 icrc, __be32 **icrcp)
+{
+	struct skb_shared_info *shinfo = skb_shinfo(skb);
+	skb_frag_t *frag;
+	u8 *addr;
+	int hdr_len;
+	int len;
+	int i;
+
+	/* handle any payload left in the linear buffer */
+	hdr_len = rxe_opcode[pkt->opcode].length;
+	addr = pkt->hdr + hdr_len;
+	len = skb_tail_pointer(skb) - skb_transport_header(skb)
+		- sizeof(struct udphdr) - hdr_len;
+	if (!shinfo->nr_frags) {
+		len -= RXE_ICRC_SIZE;
+		*icrcp = (__be32 *)(addr + len);
+	}
+	if (len > 0)
+		icrc = rxe_crc32(pkt->rxe, icrc, payload_addr(pkt), len);
+	WARN_ON(len < 0);
+
+	/* handle any payload in frags */
+	for (i = 0; i < shinfo->nr_frags; i++) {
+		frag = &shinfo->frags[i];
+		addr = page_to_virt(frag->bv_page) + frag->bv_offset;
+		len = frag->bv_len;
+		if (i == shinfo->nr_frags - 1) {
+			len -= RXE_ICRC_SIZE;
+			*icrcp = (__be32 *)(addr + len);
+		}
+		if (len > 0)
+			icrc = rxe_crc32(pkt->rxe, icrc, addr, len);
+		WARN_ON(len < 0);
+	}
+
+	return icrc;
+}
+
 /**
  * rxe_icrc_check() - Compute ICRC for a packet and compare to the ICRC
  *		      delivered in the packet.
@@ -143,13 +193,11 @@ int rxe_icrc_check(struct sk_buff *skb, struct rxe_pkt_info *pkt)
 	__be32 pkt_icrc;
 	__be32 icrc;
 
-	icrcp = (__be32 *)(pkt->hdr + pkt->paylen - RXE_ICRC_SIZE);
-	pkt_icrc = *icrcp;
-
 	icrc = rxe_icrc_hdr(skb, pkt);
-	icrc = rxe_crc32(pkt->rxe, icrc, (u8 *)payload_addr(pkt),
-				payload_size(pkt) + pkt->pad);
+	icrc = rxe_icrc_payload(skb, pkt, icrc, &icrcp);
+
 	icrc = ~icrc;
+	pkt_icrc = *icrcp;
 
 	if (unlikely(icrc != pkt_icrc))
 		return -EINVAL;
@@ -167,9 +215,8 @@ void rxe_icrc_generate(struct sk_buff *skb, struct rxe_pkt_info *pkt)
 	__be32 *icrcp;
 	__be32 icrc;
 
-	icrcp = (__be32 *)(pkt->hdr + pkt->paylen - RXE_ICRC_SIZE);
 	icrc = rxe_icrc_hdr(skb, pkt);
-	icrc = rxe_crc32(pkt->rxe, icrc, (u8 *)payload_addr(pkt),
-				payload_size(pkt) + pkt->pad);
+	icrc = rxe_icrc_payload(skb, pkt, icrc, &icrcp);
+
 	*icrcp = ~icrc;
 }
drivers/infiniband/sw/rxe/rxe_net.c +41 -10

98 lines not on this page · raw message

drivers/infiniband/sw/rxe/rxe_recv.c +1 -0
index f912a913f89a..940197199252 100644
@@ -338,6 +338,7 @@ void rxe_rcv(struct sk_buff *skb)
 	if (unlikely(err))
 		goto drop;
 
+	/* skb->data points at UDP header */
 	err = rxe_icrc_check(skb, pkt);
 	if (unlikely(err))
 		goto drop;
Signature · 1 line
2.39.2
#8 lore raw

Re: [PATCH for-next v3 05/10] RDMA/rxe: Extend rxe_icrc.c to support frags

Quoted 114 lines · Bob Pearson

Extend the subroutines rxe_icrc_generate() and rxe_icrc_check() to support skb frags.

---
  drivers/infiniband/sw/rxe/rxe_icrc.c | 65 ++++++++++++++++++++++++----
  drivers/infiniband/sw/rxe/rxe_net.c  | 51 +++++++++++++++++-----
  drivers/infiniband/sw/rxe/rxe_recv.c |  1 +
  3 files changed, 98 insertions(+), 19 deletions(-)
drivers/infiniband/sw/rxe/rxe_icrc.c +56 -9
index c9aa0995e900..393391863350 100644
@@ -63,7 +63,7 @@ static __be32 rxe_crc32(struct rxe_dev *rxe, __be32 crc, void *next, size_t len)
 
  /**
   * rxe_icrc_hdr() - Compute the partial ICRC for the network and transport
- *		  headers of a packet.
+ *		    headers of a packet.
   * @skb: packet buffer
   * @pkt: packet information
   *
@@ -129,6 +129,56 @@ static __be32 rxe_icrc_hdr(struct sk_buff *skb, struct rxe_pkt_info *pkt)
  	return crc;
  }
 
+/**
+ * rxe_icrc_payload() - Compute the ICRC for a packet payload and also
+ *			compute the address of the icrc in the packet.
+ * @skb: packet buffer
+ * @pkt: packet information
+ * @icrc: current icrc i.e. including headers
+ * @icrcp: returned pointer to icrc in skb
+ *
+ * Return: 0 if the values match else an error
+ */
+static __be32 rxe_icrc_payload(struct sk_buff *skb, struct rxe_pkt_info *pkt,
+			       __be32 icrc, __be32 **icrcp)
+{
+	struct skb_shared_info *shinfo = skb_shinfo(skb);
+	skb_frag_t *frag;
+	u8 *addr;
+	int hdr_len;
+	int len;
+	int i;
+
+	/* handle any payload left in the linear buffer */
+	hdr_len = rxe_opcode[pkt->opcode].length;
+	addr = pkt->hdr + hdr_len;
+	len = skb_tail_pointer(skb) - skb_transport_header(skb)
+		- sizeof(struct udphdr) - hdr_len;
+	if (!shinfo->nr_frags) {
+		len -= RXE_ICRC_SIZE;
+		*icrcp = (__be32 *)(addr + len);
+	}
+	if (len > 0)
+		icrc = rxe_crc32(pkt->rxe, icrc, payload_addr(pkt), len);
+	WARN_ON(len < 0);
+
+	/* handle any payload in frags */
+	for (i = 0; i < shinfo->nr_frags; i++) {
+		frag = &shinfo->frags[i];
+		addr = page_to_virt(frag->bv_page) + frag->bv_offset;
+		len = frag->bv_len;
+		if (i == shinfo->nr_frags - 1) {
+			len -= RXE_ICRC_SIZE;
+			*icrcp = (__be32 *)(addr + len);
+		}
+		if (len > 0)
+			icrc = rxe_crc32(pkt->rxe, icrc, addr, len);
+		WARN_ON(len < 0);
+	}
+
+	return icrc;
+}
+
  /**
   * rxe_icrc_check() - Compute ICRC for a packet and compare to the ICRC
   *		      delivered in the packet.
@@ -143,13 +193,11 @@ int rxe_icrc_check(struct sk_buff *skb, struct rxe_pkt_info *pkt)
  	__be32 pkt_icrc;
  	__be32 icrc;
 
-	icrcp = (__be32 *)(pkt->hdr + pkt->paylen - RXE_ICRC_SIZE);
-	pkt_icrc = *icrcp;
-
  	icrc = rxe_icrc_hdr(skb, pkt);
-	icrc = rxe_crc32(pkt->rxe, icrc, (u8 *)payload_addr(pkt),
-				payload_size(pkt) + pkt->pad);
+	icrc = rxe_icrc_payload(skb, pkt, icrc, &icrcp);
+
  	icrc = ~icrc;
+	pkt_icrc = *icrcp;
 
  	if (unlikely(icrc != pkt_icrc))
  		return -EINVAL;
@@ -167,9 +215,8 @@ void rxe_icrc_generate(struct sk_buff *skb, struct rxe_pkt_info *pkt)
  	__be32 *icrcp;
  	__be32 icrc;
 
-	icrcp = (__be32 *)(pkt->hdr + pkt->paylen - RXE_ICRC_SIZE);
  	icrc = rxe_icrc_hdr(skb, pkt);
-	icrc = rxe_crc32(pkt->rxe, icrc, (u8 *)payload_addr(pkt),
-				payload_size(pkt) + pkt->pad);
+	icrc = rxe_icrc_payload(skb, pkt, icrc, &icrcp);
+
  	*icrcp = ~icrc;
  }
drivers/infiniband/sw/rxe/rxe_net.c +0 -0
index c44ef39010f1..c43f9dd3ae6e 100644

drivers/infiniband/sw/rxe/rxe_net.c static int rxe_udp_encap_recv(struct sock *sk, struct sk_buff *skb) · line 153

Quoted 6 lines
drivers/infiniband/sw/rxe/rxe_net.c +3 -0
@@ -148,33 +148,53 @@ static int rxe_udp_encap_recv(struct sock *sk, struct sk_buff *skb)
  	struct udphdr *udph;
  	struct rxe_dev *rxe;
  	struct rxe_pkt_info *pkt = SKB_TO_PKT(skb);
+	u8 opcode;
+	u8 buf[1];
+	u8 *p;

opcode and *p duplicate.
You can use only one variable.
u8 *opcode;

Quoted 7 lines
drivers/infiniband/sw/rxe/rxe_net.c +0 -1
  	/* takes a reference on rxe->ib_dev
  	 * drop when skb is freed
  	 */
  	rxe = get_rxe_from_skb(skb);
  	if (!rxe)
-		goto drop;

drivers/infiniband/sw/rxe/rxe_net.c static int rxe_udp_encap_recv(struct sock *sk, struct sk_buff *skb) · line 166

Quoted 10 lines
drivers/infiniband/sw/rxe/rxe_net.c +6 -3
+		goto err_drop;
 
-	if (skb_linearize(skb)) {
-		ib_device_put(&rxe->ib_dev);
-		goto drop;
+	/* Get bth opcode out of skb, it may be in a fragment */
+	p = skb_header_pointer(skb, sizeof(struct udphdr), 1, buf);
+	if (!p)
+		goto err_device_put;
+	opcode = *p;
	opcode = skb_header_pointer(skb, sizeof(struct udphdr), 1, buf);
	if (!opcode)
		goto err_device_put;

;

drivers/infiniband/sw/rxe/rxe_net.c static int rxe_udp_encap_recv(struct sock *sk, struct sk_buff *skb) · line 172

Quoted 6 lines
drivers/infiniband/sw/rxe/rxe_net.c +6 -0
+
+	/* If using fragmented skbs make sure roce headers
+	 * are in linear buffer else make skb linear
+	 */
+	if (rxe_use_sg && skb_is_nonlinear(skb)) {
+		int delta = rxe_opcode[opcode].length -

int delta = rxe_opcode[(*opcode)].length -

Quoted 5 lines
drivers/infiniband/sw/rxe/rxe_net.c +5 -0
+			(skb_headlen(skb) - sizeof(struct udphdr));
+
+		if (delta > 0 && !__pskb_pull_tail(skb, delta))
+			goto err_device_put;
+	} else {

drivers/infiniband/sw/rxe/rxe_net.c static int rxe_udp_encap_recv(struct sock *sk, struct sk_buff *skb) · line 186

Quoted 10 lines
drivers/infiniband/sw/rxe/rxe_net.c +3 -1
+		if (skb_linearize(skb))
+			goto err_device_put;
  	}
 
  	udph = udp_hdr(skb);
  	pkt->rxe = rxe;
  	pkt->port_num = 1;
  	pkt->hdr = (u8 *)(udph + 1);
-	pkt->mask = RXE_GRH_MASK;
+	pkt->mask = rxe_opcode[opcode].mask | RXE_GRH_MASK;

<..>

Zhu Yanjun

Quoted 67 lines
drivers/infiniband/sw/rxe/rxe_net.c +18 -5
  	pkt->paylen = be16_to_cpu(udph->len) - sizeof(*udph);
 
-	/* remove udp header */
  	skb_pull(skb, sizeof(struct udphdr));
 
  	rxe_rcv(skb);
 
  	return 0;
-drop:
+
+err_device_put:
+	ib_device_put(&rxe->ib_dev);
+err_drop:
  	kfree_skb(skb);
 
  	return 0;
@@ -446,24 +466,35 @@ static int rxe_send(struct sk_buff *skb, struct rxe_pkt_info *pkt)
   */
  static int rxe_loopback(struct sk_buff *skb, struct rxe_pkt_info *pkt)
  {
-	memcpy(SKB_TO_PKT(skb), pkt, sizeof(*pkt));
+	struct rxe_pkt_info *newpkt;
+	int err;
 
+	/* make loopback line up with rxe_udp_encap_recv */
  	if (skb->protocol == htons(ETH_P_IP))
  		skb_pull(skb, sizeof(struct iphdr));
  	else
  		skb_pull(skb, sizeof(struct ipv6hdr));
+	skb_reset_transport_header(skb);
+
+	newpkt = SKB_TO_PKT(skb);
+	memcpy(newpkt, pkt, sizeof(*newpkt));
+	newpkt->hdr = skb_transport_header(skb) + sizeof(struct udphdr);
 
  	if (WARN_ON(!ib_device_try_get(&pkt->rxe->ib_dev))) {
  		kfree_skb(skb);
-		return -EIO;
+		err = -EINVAL;
+		goto drop;
  	}
 
  	/* remove udp header */
  	skb_pull(skb, sizeof(struct udphdr));
 
  	rxe_rcv(skb);
-
  	return 0;
+
+drop:
+	kfree_skb(skb);
+	return err;
  }
 
  int rxe_xmit_packet(struct rxe_qp *qp, struct rxe_pkt_info *pkt,
drivers/infiniband/sw/rxe/rxe_recv.c +1 -0
index f912a913f89a..940197199252 100644
@@ -338,6 +338,7 @@ void rxe_rcv(struct sk_buff *skb)
  	if (unlikely(err))
  		goto drop;
 
+	/* skb->data points at UDP header */
  	err = rxe_icrc_check(skb, pkt);
  	if (unlikely(err))
  		goto drop;
#9 lore raw

Re: [PATCH for-next v3 05/10] RDMA/rxe: Extend rxe_icrc.c to support frags

Quoted 111 lines · Zhu Yanjun
Quoted 110 lines · Bob Pearson

Extend the subroutines rxe_icrc_generate() and rxe_icrc_check() to support skb frags.

---
  drivers/infiniband/sw/rxe/rxe_icrc.c | 65 ++++++++++++++++++++++++---- drivers/infiniband/sw/rxe/rxe_net.c  | 51 +++++++++++++++++----- drivers/infiniband/sw/rxe/rxe_recv.c |  1 +

  3 files changed, 98 insertions(+), 19 deletions(-)
drivers/infiniband/sw/rxe/rxe_icrc.c +1 -1
index c9aa0995e900..393391863350 100644
@@ -63,7 +63,7 @@ static __be32 rxe_crc32(struct rxe_dev *rxe, __be32 crc, void *next, size_t len)
     /**
    * rxe_icrc_hdr() - Compute the partial ICRC for the network and transport
- *          headers of a packet.
+ *            headers of a packet.
    * @skb: packet buffer
    * @pkt: packet information
    *
@@ -129,6 +129,56 @@ static __be32 rxe_icrc_hdr(struct sk_buff *skb, struct rxe_pkt_info *pkt)
       return crc;
   }

  +/**
+ * rxe_icrc_payload() - Compute the ICRC for a packet payload and also
+ *            compute the address of the icrc in the packet.
+ * @skb: packet buffer
+ * @pkt: packet information
+ * @icrc: current icrc i.e. including headers
+ * @icrcp: returned pointer to icrc in skb
+ *
+ * Return: 0 if the values match else an error
+ */

drivers/infiniband/sw/rxe/rxe_icrc.c +40 -0
+static __be32 rxe_icrc_payload(struct sk_buff *skb, struct rxe_pkt_info *pkt,
+                   __be32 icrc, __be32 **icrcp)
+{
+    struct skb_shared_info *shinfo = skb_shinfo(skb);
+    skb_frag_t *frag;
+    u8 *addr;
+    int hdr_len;
+    int len;
+    int i;
+
+    /* handle any payload left in the linear buffer */
+    hdr_len = rxe_opcode[pkt->opcode].length;
+    addr = pkt->hdr + hdr_len;
+    len = skb_tail_pointer(skb) - skb_transport_header(skb)
+        - sizeof(struct udphdr) - hdr_len;
+    if (!shinfo->nr_frags) {
+        len -= RXE_ICRC_SIZE;
+        *icrcp = (__be32 *)(addr + len);
+    }
+    if (len > 0)
+        icrc = rxe_crc32(pkt->rxe, icrc, payload_addr(pkt), len);
+    WARN_ON(len < 0);
+
+    /* handle any payload in frags */
+    for (i = 0; i < shinfo->nr_frags; i++) {
+        frag = &shinfo->frags[i];
+        addr = page_to_virt(frag->bv_page) + frag->bv_offset;
+        len = frag->bv_len;
+        if (i == shinfo->nr_frags - 1) {
+            len -= RXE_ICRC_SIZE;
+            *icrcp = (__be32 *)(addr + len);
+        }
+        if (len > 0)
+            icrc = rxe_crc32(pkt->rxe, icrc, addr, len);
+        WARN_ON(len < 0);
+    }
+
+    return icrc;
+}
+

  /**
   * rxe_icrc_check() - Compute ICRC for a packet and compare to the ICRC *              delivered in the packet.

drivers/infiniband/sw/rxe/rxe_icrc.c +5 -6
@@ -143,13 +193,11 @@ int rxe_icrc_check(struct sk_buff *skb, struct rxe_pkt_info *pkt)
       __be32 pkt_icrc;
       __be32 icrc;
   -    icrcp = (__be32 *)(pkt->hdr + pkt->paylen - RXE_ICRC_SIZE);
-    pkt_icrc = *icrcp;
-
       icrc = rxe_icrc_hdr(skb, pkt);
-    icrc = rxe_crc32(pkt->rxe, icrc, (u8 *)payload_addr(pkt),
-                payload_size(pkt) + pkt->pad);
+    icrc = rxe_icrc_payload(skb, pkt, icrc, &icrcp);
+
       icrc = ~icrc;
+    pkt_icrc = *icrcp;
         if (unlikely(icrc != pkt_icrc))
           return -EINVAL;
@@ -167,9 +215,8 @@ void rxe_icrc_generate(struct sk_buff *skb, struct rxe_pkt_info *pkt)
       __be32 *icrcp;
       __be32 icrc;
   -    icrcp = (__be32 *)(pkt->hdr + pkt->paylen - RXE_ICRC_SIZE);
       icrc = rxe_icrc_hdr(skb, pkt);
-    icrc = rxe_crc32(pkt->rxe, icrc, (u8 *)payload_addr(pkt),
-                payload_size(pkt) + pkt->pad);
+    icrc = rxe_icrc_payload(skb, pkt, icrc, &icrcp);
+
       *icrcp = ~icrc;
   }
drivers/infiniband/sw/rxe/rxe_net.c +0 -0
index c44ef39010f1..c43f9dd3ae6e 100644
@@ -148,33 +148,53 @@ static int rxe_udp_encap_recv(struct sock *sk, struct sk_buff *skb)
       struct udphdr *udph;
Quoted 8 lines

drivers/infiniband/sw/rxe/rxe_net.c static int rxe_udp_encap_recv(struct sock *sk, struct sk_buff *skb) · line 153

Quoted 5 lines
drivers/infiniband/sw/rxe/rxe_net.c +3 -0
       struct rxe_dev *rxe;
       struct rxe_pkt_info *pkt = SKB_TO_PKT(skb);
+    u8 opcode;
+    u8 buf[1];
+    u8 *p;

opcode and *p duplicate.
You can use only one variable.

opcode is u8 p is *u8.

Quoted 107 lines

u8 *opcode;

Quoted 5 lines
drivers/infiniband/sw/rxe/rxe_net.c +0 -0
         /* takes a reference on rxe->ib_dev
        * drop when skb is freed
        */
       rxe = get_rxe_from_skb(skb);
       if (!rxe)

drivers/infiniband/sw/rxe/rxe_net.c static int rxe_udp_encap_recv(struct sock *sk, struct sk_buff *skb) · line 166

Quoted 10 lines
drivers/infiniband/sw/rxe/rxe_net.c +6 -3
-        goto drop;
+        goto err_drop;
   -    if (skb_linearize(skb)) {
-        ib_device_put(&rxe->ib_dev);
-        goto drop;
+    /* Get bth opcode out of skb, it may be in a fragment */
+    p = skb_header_pointer(skb, sizeof(struct udphdr), 1, buf);
+    if (!p)
+        goto err_device_put;
+    opcode = *p;

    opcode = skb_header_pointer(skb, sizeof(struct udphdr), 1, buf); if (!opcode)
        goto err_device_put;
;

drivers/infiniband/sw/rxe/rxe_net.c static int rxe_udp_encap_recv(struct sock *sk, struct sk_buff *skb) · line 172

Quoted 6 lines
drivers/infiniband/sw/rxe/rxe_net.c +6 -0
+
+    /* If using fragmented skbs make sure roce headers
+     * are in linear buffer else make skb linear
+     */
+    if (rxe_use_sg && skb_is_nonlinear(skb)) {
+        int delta = rxe_opcode[opcode].length -

        int delta = rxe_opcode[(*opcode)].length -

Quoted 4 lines
drivers/infiniband/sw/rxe/rxe_net.c +4 -0
+            (skb_headlen(skb) - sizeof(struct udphdr));
+
+        if (delta > 0 && !__pskb_pull_tail(skb, delta))
+            goto err_device_put;

drivers/infiniband/sw/rxe/rxe_net.c static int rxe_udp_encap_recv(struct sock *sk, struct sk_buff *skb) · line 186

Quoted 10 lines
drivers/infiniband/sw/rxe/rxe_net.c +4 -1
+    } else {
+        if (skb_linearize(skb))
+            goto err_device_put;
       }
         udph = udp_hdr(skb);
       pkt->rxe = rxe;
       pkt->port_num = 1;
       pkt->hdr = (u8 *)(udph + 1);
-    pkt->mask = RXE_GRH_MASK;
+    pkt->mask = rxe_opcode[opcode].mask | RXE_GRH_MASK;

<..>

Zhu Yanjun

Quoted 57 lines
drivers/infiniband/sw/rxe/rxe_net.c +0 -0
       pkt->paylen = be16_to_cpu(udph->len) - sizeof(*udph);

  -    /* remove udp header */
      skb_pull(skb, sizeof(struct udphdr));
        rxe_rcv(skb);
        return 0;

drivers/infiniband/sw/rxe/rxe_net.c +4 -1
-drop:
+
+err_device_put:
+    ib_device_put(&rxe->ib_dev);
+err_drop:

      kfree_skb(skb);
        return 0;

drivers/infiniband/sw/rxe/rxe_net.c +2 -1
@@ -446,24 +466,35 @@ static int rxe_send(struct sk_buff *skb, struct rxe_pkt_info *pkt)
    */
   static int rxe_loopback(struct sk_buff *skb, struct rxe_pkt_info *pkt)
   {
-    memcpy(SKB_TO_PKT(skb), pkt, sizeof(*pkt));
+    struct rxe_pkt_info *newpkt;
+    int err;

  +    /* make loopback line up with rxe_udp_encap_recv */ if (skb->protocol == htons(ETH_P_IP))
          skb_pull(skb, sizeof(struct iphdr));
      else
          skb_pull(skb, sizeof(struct ipv6hdr));
+    skb_reset_transport_header(skb);
+
+    newpkt = SKB_TO_PKT(skb);
+    memcpy(newpkt, pkt, sizeof(*newpkt));
+    newpkt->hdr = skb_transport_header(skb) + sizeof(struct udphdr); if (WARN_ON(!ib_device_try_get(&pkt->rxe->ib_dev))) { kfree_skb(skb);
-        return -EIO;
+        err = -EINVAL;
+        goto drop;
      }
        /* remove udp header */
      skb_pull(skb, sizeof(struct udphdr));
        rxe_rcv(skb);
-
      return 0;

drivers/infiniband/sw/rxe/rxe_net.c +4 -0
+
+drop:
+    kfree_skb(skb);
+    return err;

  }
    int rxe_xmit_packet(struct rxe_qp *qp, struct rxe_pkt_info *pkt,

drivers/infiniband/sw/rxe/rxe_recv.c +0 -0
index f912a913f89a..940197199252 100644
@@ -338,6 +338,7 @@ void rxe_rcv(struct sk_buff *skb)
       if (unlikely(err))
           goto drop;

  +    /* skb->data points at UDP header */
      err = rxe_icrc_check(skb, pkt);
      if (unlikely(err))
          goto drop;

#10 lore raw

Re: [PATCH for-next v3 05/10] RDMA/rxe: Extend rxe_icrc.c to support frags

Quoted 119 lines · Bob Pearson
Quoted 110 lines · Zhu Yanjun
Quoted 109 lines · Bob Pearson

Extend the subroutines rxe_icrc_generate() and rxe_icrc_check() to support skb frags.

---
   drivers/infiniband/sw/rxe/rxe_icrc.c | 65 ++++++++++++++++++++++++----
   drivers/infiniband/sw/rxe/rxe_net.c  | 51 +++++++++++++++++-----
   drivers/infiniband/sw/rxe/rxe_recv.c |  1 +
   3 files changed, 98 insertions(+), 19 deletions(-)
drivers/infiniband/sw/rxe/rxe_icrc.c +55 -7
index c9aa0995e900..393391863350 100644
@@ -63,7 +63,7 @@ static __be32 rxe_crc32(struct rxe_dev *rxe, __be32 crc, void *next, size_t len)
     /**
    * rxe_icrc_hdr() - Compute the partial ICRC for the network and transport
- *          headers of a packet.
+ *            headers of a packet.
    * @skb: packet buffer
    * @pkt: packet information
    *
@@ -129,6 +129,56 @@ static __be32 rxe_icrc_hdr(struct sk_buff *skb, struct rxe_pkt_info *pkt)
       return crc;
   }
   +/**
+ * rxe_icrc_payload() - Compute the ICRC for a packet payload and also
+ *            compute the address of the icrc in the packet.
+ * @skb: packet buffer
+ * @pkt: packet information
+ * @icrc: current icrc i.e. including headers
+ * @icrcp: returned pointer to icrc in skb
+ *
+ * Return: 0 if the values match else an error
+ */
+static __be32 rxe_icrc_payload(struct sk_buff *skb, struct rxe_pkt_info *pkt,
+                   __be32 icrc, __be32 **icrcp)
+{
+    struct skb_shared_info *shinfo = skb_shinfo(skb);
+    skb_frag_t *frag;
+    u8 *addr;
+    int hdr_len;
+    int len;
+    int i;
+
+    /* handle any payload left in the linear buffer */
+    hdr_len = rxe_opcode[pkt->opcode].length;
+    addr = pkt->hdr + hdr_len;
+    len = skb_tail_pointer(skb) - skb_transport_header(skb)
+        - sizeof(struct udphdr) - hdr_len;
+    if (!shinfo->nr_frags) {
+        len -= RXE_ICRC_SIZE;
+        *icrcp = (__be32 *)(addr + len);
+    }
+    if (len > 0)
+        icrc = rxe_crc32(pkt->rxe, icrc, payload_addr(pkt), len);
+    WARN_ON(len < 0);
+
+    /* handle any payload in frags */
+    for (i = 0; i < shinfo->nr_frags; i++) {
+        frag = &shinfo->frags[i];
+        addr = page_to_virt(frag->bv_page) + frag->bv_offset;
+        len = frag->bv_len;
+        if (i == shinfo->nr_frags - 1) {
+            len -= RXE_ICRC_SIZE;
+            *icrcp = (__be32 *)(addr + len);
+        }
+        if (len > 0)
+            icrc = rxe_crc32(pkt->rxe, icrc, addr, len);
+        WARN_ON(len < 0);
+    }
+
+    return icrc;
+}
+
   /**
    * rxe_icrc_check() - Compute ICRC for a packet and compare to the ICRC
    *              delivered in the packet.
@@ -143,13 +193,11 @@ int rxe_icrc_check(struct sk_buff *skb, struct rxe_pkt_info *pkt)
       __be32 pkt_icrc;
       __be32 icrc;
   -    icrcp = (__be32 *)(pkt->hdr + pkt->paylen - RXE_ICRC_SIZE);
-    pkt_icrc = *icrcp;
-
       icrc = rxe_icrc_hdr(skb, pkt);
-    icrc = rxe_crc32(pkt->rxe, icrc, (u8 *)payload_addr(pkt),
-                payload_size(pkt) + pkt->pad);
+    icrc = rxe_icrc_payload(skb, pkt, icrc, &icrcp);
+
       icrc = ~icrc;
+    pkt_icrc = *icrcp;
         if (unlikely(icrc != pkt_icrc))
           return -EINVAL;
@@ -167,9 +215,8 @@ void rxe_icrc_generate(struct sk_buff *skb, struct rxe_pkt_info *pkt)
       __be32 *icrcp;
       __be32 icrc;
   -    icrcp = (__be32 *)(pkt->hdr + pkt->paylen - RXE_ICRC_SIZE);
       icrc = rxe_icrc_hdr(skb, pkt);
-    icrc = rxe_crc32(pkt->rxe, icrc, (u8 *)payload_addr(pkt),
-                payload_size(pkt) + pkt->pad);
+    icrc = rxe_icrc_payload(skb, pkt, icrc, &icrcp);
+
       *icrcp = ~icrc;
   }
drivers/infiniband/sw/rxe/rxe_net.c +0 -0
index c44ef39010f1..c43f9dd3ae6e 100644
@@ -148,33 +148,53 @@ static int rxe_udp_encap_recv(struct sock *sk, struct sk_buff *skb)
Quoted 8 lines

drivers/infiniband/sw/rxe/rxe_net.c static int rxe_udp_encap_recv(struct sock *sk, struct sk_buff *skb) · line 153

Quoted 6 lines
drivers/infiniband/sw/rxe/rxe_net.c +3 -0
       struct udphdr *udph;
       struct rxe_dev *rxe;
       struct rxe_pkt_info *pkt = SKB_TO_PKT(skb);
+    u8 opcode;
+    u8 buf[1];
+    u8 *p;

opcode and *p duplicate.
You can use only one variable.

Quoted 1 line

opcode is u8 p is *u8.

I mean that you can use one variable, (for example,opcode) can complete the same functionality with the 2 variables (p and opcode).

Zhu Yanjun

Quoted 103 lines
Quoted 103 lines

u8 *opcode;

Quoted 5 lines
drivers/infiniband/sw/rxe/rxe_net.c +0 -0
         /* takes a reference on rxe->ib_dev
        * drop when skb is freed
        */
       rxe = get_rxe_from_skb(skb);
       if (!rxe)

drivers/infiniband/sw/rxe/rxe_net.c static int rxe_udp_encap_recv(struct sock *sk, struct sk_buff *skb) · line 166

Quoted 10 lines
drivers/infiniband/sw/rxe/rxe_net.c +6 -3
-        goto drop;
+        goto err_drop;
   -    if (skb_linearize(skb)) {
-        ib_device_put(&rxe->ib_dev);
-        goto drop;
+    /* Get bth opcode out of skb, it may be in a fragment */
+    p = skb_header_pointer(skb, sizeof(struct udphdr), 1, buf);
+    if (!p)
+        goto err_device_put;
+    opcode = *p;

    opcode = skb_header_pointer(skb, sizeof(struct udphdr), 1, buf); if (!opcode)
        goto err_device_put;
;

drivers/infiniband/sw/rxe/rxe_net.c static int rxe_udp_encap_recv(struct sock *sk, struct sk_buff *skb) · line 172

Quoted 6 lines
drivers/infiniband/sw/rxe/rxe_net.c +6 -0
+
+    /* If using fragmented skbs make sure roce headers
+     * are in linear buffer else make skb linear
+     */
+    if (rxe_use_sg && skb_is_nonlinear(skb)) {
+        int delta = rxe_opcode[opcode].length -

        int delta = rxe_opcode[(*opcode)].length -

Quoted 4 lines
drivers/infiniband/sw/rxe/rxe_net.c +4 -0
+            (skb_headlen(skb) - sizeof(struct udphdr));
+
+        if (delta > 0 && !__pskb_pull_tail(skb, delta))
+            goto err_device_put;

drivers/infiniband/sw/rxe/rxe_net.c static int rxe_udp_encap_recv(struct sock *sk, struct sk_buff *skb) · line 186

Quoted 10 lines
drivers/infiniband/sw/rxe/rxe_net.c +4 -1
+    } else {
+        if (skb_linearize(skb))
+            goto err_device_put;
       }
         udph = udp_hdr(skb);
       pkt->rxe = rxe;
       pkt->port_num = 1;
       pkt->hdr = (u8 *)(udph + 1);
-    pkt->mask = RXE_GRH_MASK;
+    pkt->mask = rxe_opcode[opcode].mask | RXE_GRH_MASK;

<..>

Zhu Yanjun

Quoted 57 lines
drivers/infiniband/sw/rxe/rxe_net.c +17 -4
       pkt->paylen = be16_to_cpu(udph->len) - sizeof(*udph);
   -    /* remove udp header */
       skb_pull(skb, sizeof(struct udphdr));
         rxe_rcv(skb);
         return 0;
-drop:
+
+err_device_put:
+    ib_device_put(&rxe->ib_dev);
+err_drop:
       kfree_skb(skb);
         return 0;
@@ -446,24 +466,35 @@ static int rxe_send(struct sk_buff *skb, struct rxe_pkt_info *pkt)
    */
   static int rxe_loopback(struct sk_buff *skb, struct rxe_pkt_info *pkt)
   {
-    memcpy(SKB_TO_PKT(skb), pkt, sizeof(*pkt));
+    struct rxe_pkt_info *newpkt;
+    int err;
   +    /* make loopback line up with rxe_udp_encap_recv */
       if (skb->protocol == htons(ETH_P_IP))
           skb_pull(skb, sizeof(struct iphdr));
       else
           skb_pull(skb, sizeof(struct ipv6hdr));
+    skb_reset_transport_header(skb);
+
+    newpkt = SKB_TO_PKT(skb);
+    memcpy(newpkt, pkt, sizeof(*newpkt));
+    newpkt->hdr = skb_transport_header(skb) + sizeof(struct udphdr);
         if (WARN_ON(!ib_device_try_get(&pkt->rxe->ib_dev))) {
           kfree_skb(skb);
-        return -EIO;
+        err = -EINVAL;
+        goto drop;
       }
         /* remove udp header */
       skb_pull(skb, sizeof(struct udphdr));
         rxe_rcv(skb);
-
       return 0;
+
+drop:
+    kfree_skb(skb);
+    return err;
   }
     int rxe_xmit_packet(struct rxe_qp *qp, struct rxe_pkt_info *pkt,
drivers/infiniband/sw/rxe/rxe_recv.c +0 -0
index f912a913f89a..940197199252 100644
@@ -338,6 +338,7 @@ void rxe_rcv(struct sk_buff *skb)
       if (unlikely(err))
           goto drop;
   +    /* skb->data points at UDP header */
       err = rxe_icrc_check(skb, pkt);
       if (unlikely(err))
           goto drop;
#11 lore raw

PATCH v3 6/10 RDMA/rxe: Extend rxe_init_req_packet() for frags

In reply to Bob Pearson

Add code to rxe_build_req_packet() to allocate space for the pad and icrc if the skb is fragmented.

---
 drivers/infiniband/sw/rxe/rxe_loc.h    |  5 ++
 drivers/infiniband/sw/rxe/rxe_mr.c     |  5 +-
 drivers/infiniband/sw/rxe/rxe_opcode.c |  2 +
 drivers/infiniband/sw/rxe/rxe_req.c    | 83 ++++++++++++++++++++++----
 4 files changed, 84 insertions(+), 11 deletions(-)
drivers/infiniband/sw/rxe/rxe_loc.h +5 -0
index 96b1fb79610a..40624de62288 100644
@@ -177,7 +177,12 @@ void rxe_srq_cleanup(struct rxe_pool_elem *elem);
 void rxe_dealloc(struct ib_device *ib_dev);
 
 int rxe_completer(struct rxe_qp *qp);
+
+/* rxe_req.c */
+int rxe_prepare_pad_icrc(struct rxe_pkt_info *pkt, struct sk_buff *skb,
+			 int payload, bool frag);
 int rxe_requester(struct rxe_qp *qp);
+
 int rxe_responder(struct rxe_qp *qp);
 
 /* rxe_icrc.c */
drivers/infiniband/sw/rxe/rxe_mr.c +4 -1
index 0ac71238599a..5178775f2d4e 100644
@@ -263,7 +263,10 @@ int rxe_add_frag(struct sk_buff *skb, struct rxe_mr *mr, struct page *page,
 	skb_frag_t *frag = &skb_shinfo(skb)->frags[nr_frags];
 
 	if (nr_frags >= MAX_SKB_FRAGS) {
-		rxe_dbg_mr(mr, "ran out of frags");
+		if (mr)
+			rxe_dbg_mr(mr, "ran out of frags");
+		else
+			rxe_dbg("ran out of frags");
 		return -EINVAL;
 	}
 
drivers/infiniband/sw/rxe/rxe_opcode.c +2 -0
index f358b732a751..a72e5fd4f571 100644
@@ -399,6 +399,8 @@ struct rxe_opcode_info rxe_opcode[RXE_NUM_OPCODE] = {
 			[RXE_BTH]	= 0,
 			[RXE_FETH]	= RXE_BTH_BYTES,
 			[RXE_RETH]	= RXE_BTH_BYTES + RXE_FETH_BYTES,
+			[RXE_PAYLOAD]	= RXE_BTH_BYTES + RXE_FETH_BYTES +
+					  RXE_RETH_BYTES,
 		}
 	},
 	[IB_OPCODE_RC_ATOMIC_WRITE]                        = {
drivers/infiniband/sw/rxe/rxe_req.c +73 -10

124 lines not on this page · raw message

Signature · 1 line
2.39.2
#12 lore raw

PATCH v3 7/10 RDMA/rxe: Extend response packets for frags

In reply to Bob Pearson

Extend prepare_ack_packet(), read_reply() and send_common_ack() in rxe_resp.c to support fragmented skbs. Adjust calls to these routines for the changed API.

---
 drivers/infiniband/sw/rxe/rxe_resp.c | 59 ++++++++++++++++++----------
 1 file changed, 38 insertions(+), 21 deletions(-)
drivers/infiniband/sw/rxe/rxe_resp.c +38 -21
index 254f2eab8d20..dc62e11dc448 100644
@@ -765,14 +765,11 @@ static enum resp_states atomic_write_reply(struct rxe_qp *qp,
 
 static struct sk_buff *prepare_ack_packet(struct rxe_qp *qp,
 					  struct rxe_pkt_info *ack,
-					  int opcode,
-					  int payload,
-					  u32 psn,
-					  u8 syndrome)
+					  int opcode, int payload, u32 psn,
+					  u8 syndrome, bool *fragp)
 {
 	struct rxe_dev *rxe = to_rdev(qp->ibqp.device);
 	struct sk_buff *skb;
-	int err;
 
 	ack->rxe = rxe;
 	ack->qp = qp;
@@ -788,7 +785,7 @@ static struct sk_buff *prepare_ack_packet(struct rxe_qp *qp,
 	ack->paylen = rxe_opcode[opcode].length + payload +
 			ack->pad + RXE_ICRC_SIZE;
 
-	skb = rxe_init_packet(qp, &qp->pri_av, ack, NULL);
+	skb = rxe_init_packet(qp, &qp->pri_av, ack, fragp);
 	if (!skb)
 		return NULL;
 
@@ -803,12 +800,6 @@ static struct sk_buff *prepare_ack_packet(struct rxe_qp *qp,
 	if (ack->mask & RXE_ATMACK_MASK)
 		atmack_set_orig(ack, qp->resp.res->atomic.orig_val);
 
-	err = rxe_prepare(&qp->pri_av, ack, skb);
-	if (err) {
-		kfree_skb(skb);
-		return NULL;
-	}
-
 	return skb;
 }
 
@@ -881,7 +872,8 @@ static enum resp_states read_reply(struct rxe_qp *qp,
 	struct resp_res *res = qp->resp.res;
 	struct rxe_mr *mr;
 	unsigned int skb_offset = 0;
-	u8 *pad_addr;
+	enum rxe_mr_copy_op op;
+	bool frag;
 
 	if (!res) {
 		res = rxe_prepare_res(qp, req_pkt, RXE_READ_MASK);
@@ -898,8 +890,10 @@ static enum resp_states read_reply(struct rxe_qp *qp,
 			qp->resp.mr = NULL;
 		} else {
 			mr = rxe_recheck_mr(qp, res->read.rkey);
-			if (!mr)
-				return RESPST_ERR_RKEY_VIOLATION;
+			if (!mr) {
+				state = RESPST_ERR_RKEY_VIOLATION;
+				goto err_out;
+			}
 		}
 
 		if (res->read.resid <= mtu)
@@ -926,23 +920,33 @@ static enum resp_states read_reply(struct rxe_qp *qp,
 	payload = min_t(int, res->read.resid, mtu);
 
 	skb = prepare_ack_packet(qp, &ack_pkt, opcode, payload,
-				 res->cur_psn, AETH_ACK_UNLIMITED);
+				 res->cur_psn, AETH_ACK_UNLIMITED, &frag);
 	if (!skb) {
 		state = RESPST_ERR_RNR;
 		goto err_out;
 	}
 
+	op = frag ? RXE_FRAG_FROM_MR : RXE_COPY_FROM_MR;
 	err = rxe_copy_mr_data(skb, mr, res->read.va, payload_addr(&ack_pkt),
-			       skb_offset, payload, RXE_COPY_FROM_MR);
+			       skb_offset, payload, op);
 	if (err) {
 		kfree_skb(skb);
 		state = RESPST_ERR_RKEY_VIOLATION;
 		goto err_out;
 	}
 
-	if (ack_pkt.pad) {
-		pad_addr = payload_addr(&ack_pkt) + payload;
-		memset(pad_addr, 0, ack_pkt.pad);
+	err = rxe_prepare_pad_icrc(&ack_pkt, skb, payload, frag);
+	if (err) {
+		kfree_skb(skb);
+		state = RESPST_ERR_RNR;
+		goto err_out;
+	}
+
+	err = rxe_prepare(&qp->pri_av, &ack_pkt, skb);
+	if (err) {
+		kfree_skb(skb);
+		state = RESPST_ERR_RNR;
+		goto err_out;
 	}
 
 	/* rxe_xmit_packet always consumes the skb */
@@ -1177,10 +1181,23 @@ static int send_common_ack(struct rxe_qp *qp, u8 syndrome, u32 psn,
 	struct rxe_pkt_info ack_pkt;
 	struct sk_buff *skb;
 
-	skb = prepare_ack_packet(qp, &ack_pkt, opcode, 0, psn, syndrome);
+	skb = prepare_ack_packet(qp, &ack_pkt, opcode, 0, psn,
+				 syndrome, NULL);
 	if (!skb)
 		return -ENOMEM;
 
+	err = rxe_prepare_pad_icrc(&ack_pkt, skb, 0, false);
+	if (err) {
+		kfree_skb(skb);
+		return err;
+	}
+
+	err = rxe_prepare(&qp->pri_av, &ack_pkt, skb);
+	if (err) {
+		kfree_skb(skb);
+		return err;
+	}
+
 	err = rxe_xmit_packet(qp, &ack_pkt, skb);
 	if (err)
 		rxe_dbg_qp(qp, "Failed sending %s\n", msg);
Signature · 1 line
2.39.2
#13 lore raw

PATCH v3 8/10 RDMA/rxe: Extend send/write_data_in() for frags

In reply to Bob Pearson

Extend send_data_in() and write_data_in() in rxe_resp.c to support fragmented received skbs.

This is in preparation for using fragmented skbs.

---
 drivers/infiniband/sw/rxe/rxe_resp.c | 102 +++++++++++++++++----------
 1 file changed, 64 insertions(+), 38 deletions(-)
drivers/infiniband/sw/rxe/rxe_resp.c +64 -38
index dc62e11dc448..c7153e376987 100644
@@ -559,45 +559,88 @@ static enum resp_states check_rkey(struct rxe_qp *qp,
 	return state;
 }
 
-static enum resp_states send_data_in(struct rxe_qp *qp, void *data_addr,
-				     int data_len)
+/**
+ * rxe_send_data_in() - Copy payload data into receive buffer
+ * @qp: The queue pair
+ * @pkt: Request packet info
+ *
+ * Copy the packet payload into the receive buffer at the current offset.
+ * If a UD message also copy the IP header into the receive buffer.
+ *
+ * Returns: 0 if successful else an error resp_states value.
+ */
+static enum resp_states rxe_send_data_in(struct rxe_qp *qp,
+					 struct rxe_pkt_info *pkt)
 {
-	struct sk_buff *skb = NULL;
+	struct sk_buff *skb = PKT_TO_SKB(pkt);
+	u8 *data_addr = payload_addr(pkt);
+	int data_len = payload_size(pkt);
+	union rdma_network_hdr hdr;
+	enum rxe_mr_copy_op op;
 	int skb_offset = 0;
 	int err;
 
+	/* Per IBA for UD packets copy the IP header into the receive buffer */
+	if (qp_type(qp) == IB_QPT_UD || qp_type(qp) == IB_QPT_GSI) {
+		if (skb->protocol == htons(ETH_P_IP)) {
+			memset(&hdr.reserved, 0, sizeof(hdr.reserved));
+			memcpy(&hdr.roce4grh, ip_hdr(skb), sizeof(hdr.roce4grh));
+		} else {
+			memcpy(&hdr.ibgrh, ipv6_hdr(skb), sizeof(hdr));
+		}
+		err = rxe_copy_dma_data(skb, qp->pd, IB_ACCESS_LOCAL_WRITE,
+					&qp->resp.wqe->dma, &hdr, skb_offset,
+					sizeof(hdr), RXE_COPY_TO_MR);
+		if (err)
+			goto err_out;
+	}
+
+	op = skb_is_nonlinear(skb) ? RXE_FRAG_TO_MR : RXE_COPY_TO_MR;
+	/* offset to payload from skb->data (= &bth header) */
+	skb_offset = rxe_opcode[pkt->opcode].length;
 	err = rxe_copy_dma_data(skb, qp->pd, IB_ACCESS_LOCAL_WRITE,
 				&qp->resp.wqe->dma, data_addr,
-				skb_offset, data_len, RXE_COPY_TO_MR);
-	if (unlikely(err))
-		return (err == -ENOSPC) ? RESPST_ERR_LENGTH
-					: RESPST_ERR_MALFORMED_WQE;
+				skb_offset, data_len, op);
+	if (err)
+		goto err_out;
 
 	return RESPST_NONE;
+
+err_out:
+	return (err == -ENOSPC) ? RESPST_ERR_LENGTH
+				: RESPST_ERR_MALFORMED_WQE;
 }
 
-static enum resp_states write_data_in(struct rxe_qp *qp,
-				      struct rxe_pkt_info *pkt)
+/**
+ * rxe_write_data_in() - Copy payload data to iova
+ * @qp: The queue pair
+ * @pkt: Request packet info
+ *
+ * Copy the packet payload to current iova and update iova.
+ *
+ * Returns: 0 if successful else an error resp_states value.
+ */
+static enum resp_states rxe_write_data_in(struct rxe_qp *qp,
+					  struct rxe_pkt_info *pkt)
 {
 	struct sk_buff *skb = PKT_TO_SKB(pkt);
-	enum resp_states rc = RESPST_NONE;
+	u8 *data_addr = payload_addr(pkt);
 	int data_len = payload_size(pkt);
+	enum rxe_mr_copy_op op;
+	int skb_offset;
 	int err;
-	int skb_offset = 0;
 
+	op = skb_is_nonlinear(skb) ? RXE_FRAG_TO_MR : RXE_COPY_TO_MR;
+	skb_offset = rxe_opcode[pkt->opcode].length;
 	err = rxe_copy_mr_data(skb, qp->resp.mr, qp->resp.va + qp->resp.offset,
-			  payload_addr(pkt), skb_offset, data_len,
-			  RXE_COPY_TO_MR);
-	if (err) {
-		rc = RESPST_ERR_RKEY_VIOLATION;
-		goto out;
-	}
+			  data_addr, skb_offset, data_len, op);
+	if (err)
+		return RESPST_ERR_RKEY_VIOLATION;
 
 	qp->resp.va += data_len;
 	qp->resp.resid -= data_len;
 
-out:
-	return rc;
+	return RESPST_NONE;
 }
 
 static struct resp_res *rxe_prepare_res(struct rxe_qp *qp,
@@ -991,30 +1034,13 @@ static int invalidate_rkey(struct rxe_qp *qp, u32 rkey)
 static enum resp_states execute(struct rxe_qp *qp, struct rxe_pkt_info *pkt)
 {
 	enum resp_states err;
-	struct sk_buff *skb = PKT_TO_SKB(pkt);
-	union rdma_network_hdr hdr;
 
 	if (pkt->mask & RXE_SEND_MASK) {
-		if (qp_type(qp) == IB_QPT_UD ||
-		    qp_type(qp) == IB_QPT_GSI) {
-			if (skb->protocol == htons(ETH_P_IP)) {
-				memset(&hdr.reserved, 0,
-						sizeof(hdr.reserved));
-				memcpy(&hdr.roce4grh, ip_hdr(skb),
-						sizeof(hdr.roce4grh));
-				err = send_data_in(qp, &hdr, sizeof(hdr));
-			} else {
-				err = send_data_in(qp, ipv6_hdr(skb),
-						sizeof(hdr));
-			}
-			if (err)
-				return err;
-		}
-		err = send_data_in(qp, payload_addr(pkt), payload_size(pkt));
+		err = rxe_send_data_in(qp, pkt);
 		if (err)
 			return err;
 	} else if (pkt->mask & RXE_WRITE_MASK) {
-		err = write_data_in(qp, pkt);
+		err = rxe_write_data_in(qp, pkt);
 		if (err)
 			return err;
 	} else if (pkt->mask & RXE_READ_MASK) {
Signature · 1 line
2.39.2
#14 lore raw

PATCH v3 9/10 RDMA/rxe: Extend do_read() in rxe_comp.c for frags

In reply to Bob Pearson

Extend do_read() in rxe_comp.c to support fragmented skbs.

Rename rxe_do_read(). Adjust caller's API.

---
 drivers/infiniband/sw/rxe/rxe_comp.c | 39 ++++++++++++++++++----------
 1 file changed, 26 insertions(+), 13 deletions(-)
drivers/infiniband/sw/rxe/rxe_comp.c +26 -13
index 670ee08f6f5a..ecaaed15c4eb 100644
@@ -360,22 +360,35 @@ static inline enum comp_state check_ack(struct rxe_qp *qp,
 	return COMPST_ERROR;
 }
 
-static inline enum comp_state do_read(struct rxe_qp *qp,
-				      struct rxe_pkt_info *pkt,
-				      struct rxe_send_wqe *wqe)
+/**
+ * rxe_do_read() - Process read reply packet
+ * @qp: The queue pair
+ * @pkt: Packet info
+ * @wqe: The current work request
+ *
+ * Copy payload from incoming read reply packet into current
+ * iova.
+ *
+ * Returns: 0 on success else an error comp_state
+ */
+static inline enum comp_state rxe_do_read(struct rxe_qp *qp,
+					  struct rxe_pkt_info *pkt,
+					  struct rxe_send_wqe *wqe)
 {
 	struct sk_buff *skb = PKT_TO_SKB(pkt);
-	int skb_offset = 0;
-	int ret;
+	u8 *data_addr = payload_addr(pkt);
+	int data_len = payload_size(pkt);
+	enum rxe_mr_copy_op op;
+	int skb_offset;
+	int err;
 
-	ret = rxe_copy_dma_data(skb, qp->pd, IB_ACCESS_LOCAL_WRITE,
-				&wqe->dma, payload_addr(pkt),
-				skb_offset, payload_size(pkt),
-				RXE_COPY_TO_MR);
-	if (ret) {
-		wqe->status = IB_WC_LOC_PROT_ERR;
+	op = skb_is_nonlinear(skb) ? RXE_FRAG_TO_MR : RXE_COPY_TO_MR;
+	skb_offset = rxe_opcode[pkt->opcode].length;
+	err = rxe_copy_dma_data(skb, qp->pd, IB_ACCESS_LOCAL_WRITE,
+				&wqe->dma, data_addr,
+				skb_offset, data_len, op);
+	if (err)
 		return COMPST_ERROR;
-	}
 
 	if (wqe->dma.resid == 0 && (pkt->mask & RXE_END_MASK))
 		return COMPST_COMP_ACK;
@@ -704,7 +717,7 @@ int rxe_completer(struct rxe_qp *qp)
 			break;
 
 		case COMPST_READ:
-			state = do_read(qp, pkt, wqe);
+			state = rxe_do_read(qp, pkt, wqe);
 			break;
 
 		case COMPST_ATOMIC:
Signature · 1 line
2.39.2
#15 lore raw

PATCH v3 10/10 RDMA/rxe: Enable sg code in rxe

In reply to Bob Pearson

Make changes to enable sg code in rxe.

---
 drivers/infiniband/sw/rxe/rxe.c     | 4 ++--
 drivers/infiniband/sw/rxe/rxe_req.c | 4 ++--
 2 files changed, 4 insertions(+), 4 deletions(-)
drivers/infiniband/sw/rxe/rxe.c +2 -2
index 800e8c0d437d..b52dd1704e74 100644
@@ -14,9 +14,9 @@ MODULE_DESCRIPTION("Soft RDMA transport");
 MODULE_LICENSE("Dual BSD/GPL");
 
 /* if true allow using fragmented skbs */
-bool rxe_use_sg;
+bool rxe_use_sg = true;
 module_param_named(use_sg, rxe_use_sg, bool, 0444);
-MODULE_PARM_DESC(use_sg, "Support skb frags; default false");
+MODULE_PARM_DESC(use_sg, "Support skb frags; default true");
 
 /* free resources for a rxe device all objects created for this device must
  * have been destroyed
drivers/infiniband/sw/rxe/rxe_req.c +2 -2
index cf34d1a58f85..d00c24e1a569 100644
@@ -402,7 +402,7 @@ static struct sk_buff *rxe_init_req_packet(struct rxe_qp *qp,
 	struct sk_buff *skb = NULL;
 	struct rxe_av *av;
 	struct rxe_ah *ah = NULL;
-	bool frag = false;
+	bool frag;
 	int err;
 
 	pkt->rxe = rxe;
@@ -426,7 +426,7 @@ static struct sk_buff *rxe_init_req_packet(struct rxe_qp *qp,
 			pkt->pad + RXE_ICRC_SIZE;
 
 	/* init skb */
-	skb = rxe_init_packet(qp, av, pkt, NULL);
+	skb = rxe_init_packet(qp, av, pkt, &frag);
 	if (unlikely(!skb)) {
 		err = -ENOMEM;
 		goto err_out;
Signature · 1 line
2.39.2
maintainer
#16 lore raw

Re: [PATCH for-next v3 10/10] RDMA/rxe: Enable sg code in rxe

Quoted 13 lines · Bob Pearson

Make changes to enable sg code in rxe.

---
 drivers/infiniband/sw/rxe/rxe.c     | 4 ++--
 drivers/infiniband/sw/rxe/rxe_req.c | 4 ++--
 2 files changed, 4 insertions(+), 4 deletions(-)
drivers/infiniband/sw/rxe/rxe.c +0 -0
index 800e8c0d437d..b52dd1704e74 100644

drivers/infiniband/sw/rxe/rxe.c MODULE_DESCRIPTION("Soft RDMA transport"); · line 19

Quoted 8 lines
drivers/infiniband/sw/rxe/rxe.c +2 -2
@@ -14,9 +14,9 @@ MODULE_DESCRIPTION("Soft RDMA transport");
 MODULE_LICENSE("Dual BSD/GPL");
 
 /* if true allow using fragmented skbs */
-bool rxe_use_sg;
+bool rxe_use_sg = true;
 module_param_named(use_sg, rxe_use_sg, bool, 0444);
-MODULE_PARM_DESC(use_sg, "Support skb frags; default false");
+MODULE_PARM_DESC(use_sg, "Support skb frags; default true");

I would like to avoid the module option - is it necessary? Shouldn't frags always be better?

Jason

#17 lore raw
In reply to Bob Pearson
Quoted 28 lines · Bob Pearson

This patch set is a revised version of an older set which implements support for nonlinear or fragmented packets. This avoids extra copies in both the send and receive paths and gives significant performance improvement for large messages such as are used in storage applications.

This patch set has been heavily tested at large system scale and demonstrated a 2X improvement in file system read performance on a 200 Gb/sec network.

The patch set is rebased to the current for-next branch with the following previous patch sets applied:
RDMA/rxe: Fix incomplete state save in rxe_requester RDMA/rxe: Misc fixes and cleanups
Enable rcu locking of verbs objects
RDMA/rxe: Misc cleanups

Bob Pearson (10):
RDMA/rxe: Add sg fragment ops
RDMA/rxe: Extend rxe_mr_copy to support skb frags RDMA/rxe: Extend copy_data to support skb frags RDMA/rxe: Extend rxe_init_packet() to support frags RDMA/rxe: Extend rxe_icrc.c to support frags RDMA/rxe: Extend rxe_init_req_packet() for frags RDMA/rxe: Extend response packets for frags RDMA/rxe: Extend send/write_data_in() for frags RDMA/rxe: Extend do_read() in rxe_comp.c for frags RDMA/rxe: Enable sg code in rxe

Quoted 15 lines
  drivers/infiniband/sw/rxe/rxe.c        |   5 +
  drivers/infiniband/sw/rxe/rxe.h        |   3 +
  drivers/infiniband/sw/rxe/rxe_comp.c   |  46 +++-
  drivers/infiniband/sw/rxe/rxe_icrc.c   |  65 ++++-
  drivers/infiniband/sw/rxe/rxe_loc.h    |  27 +-
  drivers/infiniband/sw/rxe/rxe_mr.c     | 348 +++++++++++++++++++------
  drivers/infiniband/sw/rxe/rxe_net.c    | 109 +++++++-
  drivers/infiniband/sw/rxe/rxe_opcode.c |   2 +
  drivers/infiniband/sw/rxe/rxe_recv.c   |   1 +
  drivers/infiniband/sw/rxe/rxe_req.c    |  88 ++++++-
  drivers/infiniband/sw/rxe/rxe_resp.c   | 172 +++++++-----
  drivers/infiniband/sw/rxe/rxe_verbs.h  |   8 +-
  12 files changed, 672 insertions(+), 202 deletions(-)

What are the following? This is the new format in linux kernel community?

Zhu Yanjun

Quoted 21 lines

base-commit: 693e1cdebb50d2aa67406411ca6d5be195d62771
prerequisite-patch-id: c3994e7a93e37e0ce4f50e0c768f3c1a0059a02f
prerequisite-patch-id: 48e13f6ccb560fdeacbd20aaf6696782c23d1190
prerequisite-patch-id: da75fb8eaa863df840e7b392b5048fcc72b0bef3
prerequisite-patch-id: d0877649e2edaf00585a0a6a80391fe0d7bbc13b
prerequisite-patch-id: 6495b1d1f664f8ab91ed9ef9d2ca5b3b27d7df35
prerequisite-patch-id: a6367b8fedd0d8999139c8b857ebbd3ce5c72245
prerequisite-patch-id: 78c95e90a5e49b15b7af8ef57130739c143e88b5
prerequisite-patch-id: 7c65a01066c0418de6897bc8b5f44d078d21b0ec
prerequisite-patch-id: 8ab09f93c23c7875e56c597e69236c30464723b6
prerequisite-patch-id: ca9d84b34873b49048e42fb4c13a2a097c215c46
prerequisite-patch-id: 0f6a587501c8246e1185dfd0cbf5e2044c5f9b13
prerequisite-patch-id: 5246df93137429916d76e75b9a13a4ad5ceb0bad
prerequisite-patch-id: 41b0e4150794dd914d9fcb4cd106fe4cf4227611
prerequisite-patch-id: 02b08ec037bc35b9c7771640c89c66504cdf38a6
prerequisite-patch-id: dfccc06c16454d7fe8e6fcba064d4e471d314666
prerequisite-patch-id: 7459a6e5cdd46efd53ba27f9b3e9028af6e0863b
prerequisite-patch-id: 36d49f9303f5cb276a5601c1ab568eea6eca7d3a
prerequisite-patch-id: 6359a681e40832694f81ca003c10e5327996bf7d
prerequisite-patch-id: 558175db657f374dbd3e0a57ac4c5fb77a56b6c6
prerequisite-patch-id: d6b811de06c8900be5840dd29715161d26db66cf

#18 lore raw
Quoted 45 lines · Zhu Yanjun
Quoted 40 lines · Bob Pearson

This patch set is a revised version of an older set which implements support for nonlinear or fragmented packets. This avoids extra copies in both the send and receive paths and gives significant performance improvement for large messages such as are used in storage applications.

This patch set has been heavily tested at large system scale and demonstrated a 2X improvement in file system read performance on a 200 Gb/sec network.

The patch set is rebased to the current for-next branch with the following previous patch sets applied:
    RDMA/rxe: Fix incomplete state save in rxe_requester RDMA/rxe: Misc fixes and cleanups
    Enable rcu locking of verbs objects
    RDMA/rxe: Misc cleanups

Bob Pearson (10):
   RDMA/rxe: Add sg fragment ops
   RDMA/rxe: Extend rxe_mr_copy to support skb frags RDMA/rxe: Extend copy_data to support skb frags RDMA/rxe: Extend rxe_init_packet() to support frags RDMA/rxe: Extend rxe_icrc.c to support frags RDMA/rxe: Extend rxe_init_req_packet() for frags RDMA/rxe: Extend response packets for frags RDMA/rxe: Extend send/write_data_in() for frags RDMA/rxe: Extend do_read() in rxe_comp.c for frags RDMA/rxe: Enable sg code in rxe

  drivers/infiniband/sw/rxe/rxe.c        |   5 + drivers/infiniband/sw/rxe/rxe.h        |   3 + drivers/infiniband/sw/rxe/rxe_comp.c   |  46 +++- drivers/infiniband/sw/rxe/rxe_icrc.c   |  65 ++++- drivers/infiniband/sw/rxe/rxe_loc.h    |  27 +- drivers/infiniband/sw/rxe/rxe_mr.c     | 348 +++++++++++++++++++------ drivers/infiniband/sw/rxe/rxe_net.c    | 109 +++++++- drivers/infiniband/sw/rxe/rxe_opcode.c |   2 + drivers/infiniband/sw/rxe/rxe_recv.c   |   1 + drivers/infiniband/sw/rxe/rxe_req.c    |  88 ++++++- drivers/infiniband/sw/rxe/rxe_resp.c   | 172 +++++++----- drivers/infiniband/sw/rxe/rxe_verbs.h  |   8 +-

Quoted 3 lines
  12 files changed, 672 insertions(+), 202 deletions(-)
Quoted 1 line

What are the following? This is the new format in linux kernel community?

If you type --base d6b811de06c8900be after git format-patch it documents what the patch was applied to. I have multiple patch sets that have to be applied in order. Three of them have already been submitted but are not upstream yet. The fourth one was submitted just before this set.

Bob

Quoted 25 lines

Zhu Yanjun

Quoted 21 lines

base-commit: 693e1cdebb50d2aa67406411ca6d5be195d62771
prerequisite-patch-id: c3994e7a93e37e0ce4f50e0c768f3c1a0059a02f
prerequisite-patch-id: 48e13f6ccb560fdeacbd20aaf6696782c23d1190
prerequisite-patch-id: da75fb8eaa863df840e7b392b5048fcc72b0bef3
prerequisite-patch-id: d0877649e2edaf00585a0a6a80391fe0d7bbc13b
prerequisite-patch-id: 6495b1d1f664f8ab91ed9ef9d2ca5b3b27d7df35
prerequisite-patch-id: a6367b8fedd0d8999139c8b857ebbd3ce5c72245
prerequisite-patch-id: 78c95e90a5e49b15b7af8ef57130739c143e88b5
prerequisite-patch-id: 7c65a01066c0418de6897bc8b5f44d078d21b0ec
prerequisite-patch-id: 8ab09f93c23c7875e56c597e69236c30464723b6
prerequisite-patch-id: ca9d84b34873b49048e42fb4c13a2a097c215c46
prerequisite-patch-id: 0f6a587501c8246e1185dfd0cbf5e2044c5f9b13
prerequisite-patch-id: 5246df93137429916d76e75b9a13a4ad5ceb0bad
prerequisite-patch-id: 41b0e4150794dd914d9fcb4cd106fe4cf4227611
prerequisite-patch-id: 02b08ec037bc35b9c7771640c89c66504cdf38a6
prerequisite-patch-id: dfccc06c16454d7fe8e6fcba064d4e471d314666
prerequisite-patch-id: 7459a6e5cdd46efd53ba27f9b3e9028af6e0863b
prerequisite-patch-id: 36d49f9303f5cb276a5601c1ab568eea6eca7d3a
prerequisite-patch-id: 6359a681e40832694f81ca003c10e5327996bf7d
prerequisite-patch-id: 558175db657f374dbd3e0a57ac4c5fb77a56b6c6
prerequisite-patch-id: d6b811de06c8900be5840dd29715161d26db66cf

maintainer
#19 lore raw
In reply to Bob Pearson
Quoted 16 lines · Bob Pearson

This patch set is a revised version of an older set which implements support for nonlinear or fragmented packets. This avoids extra copies in both the send and receive paths and gives significant performance improvement for large messages such as are used in storage applications.

This patch set has been heavily tested at large system scale and demonstrated a 2X improvement in file system read performance on a 200 Gb/sec network.

The patch set is rebased to the current for-next branch with the following previous patch sets applied:
RDMA/rxe: Fix incomplete state save in rxe_requester RDMA/rxe: Misc fixes and cleanups
Enable rcu locking of verbs objects
RDMA/rxe: Misc cleanups

Quoted 11 lines

Bob Pearson (10):
RDMA/rxe: Add sg fragment ops
RDMA/rxe: Extend rxe_mr_copy to support skb frags RDMA/rxe: Extend copy_data to support skb frags RDMA/rxe: Extend rxe_init_packet() to support frags RDMA/rxe: Extend rxe_icrc.c to support frags RDMA/rxe: Extend rxe_init_req_packet() for frags RDMA/rxe: Extend response packets for frags RDMA/rxe: Extend send/write_data_in() for frags RDMA/rxe: Extend do_read() in rxe_comp.c for frags RDMA/rxe: Enable sg code in rxe

This does not apply to the tree so it will have to be rebased and resent, it looked OK other than the module option question

Jason