Cleanup for siw
Subsystems (2)
v5 cover Cleanup for siw
V5 changes:
1. add Acked-by tags.
2. rebase to latest rdma tree and remove one obsolete patch.
V4 changes:
1. add Acked-by tags.
2. update patch 3 and patch 12 per Bernard's review.
3. update patch header in patch 18.
V3 changes:
1. add Acked-by tags.
2. drop 2 patches and address other comments.
Appreciate for Bernard's review!
V2 changes:
1. address W=1 warning in patch 12 and 19 per the report from lkp.
2. add one more patch (20th).
Hi,
This series aim to cleanup siw code, please review and comment!
Thanks,
Guoqing
Guoqing Jiang (17):
RDMA/siw: Introduce siw_get_page
RDMA/siw: Introduce siw_update_skb_rcvd
RDMA/siw: Use iov.iov_len in kernel_sendmsg RDMA/siw: Remove goto lable in siw_mmap
RDMA/siw: Remove rcu from siw_qp
RDMA/siw: No need to check term_info.valid before call
siw_send_terminate
RDMA/siw: Factor out siw_rx_data helper
RDMA/siw: Introduce SIW_STAG_MAX_INDEX
RDMA/siw: Add one parameter to siw_destroy_cpulist RDMA/siw: Introduce siw_cep_set_free_and_put RDMA/siw: Introduce siw_free_cm_id
RDMA/siw: Cleanup siw_accept
RDMA/siw: Remove siw_sk_save_upcalls
RDMA/siw: Fix typo
RDMA/siw: Only check attrs->cap.max_send_wr in siw_create_qp RDMA/siw: Introduce siw_destroy_cep_sock
RDMA/siw: Update comments for siw_qp_sq_process
drivers/infiniband/sw/siw/siw.h | 1 - drivers/infiniband/sw/siw/siw_cm.c | 145 +++++++++++--------------- drivers/infiniband/sw/siw/siw_main.c | 30 +++--- drivers/infiniband/sw/siw/siw_mem.c | 12 ++- drivers/infiniband/sw/siw/siw_qp.c | 2 +- drivers/infiniband/sw/siw/siw_qp_rx.c | 84 ++++++--------- drivers/infiniband/sw/siw/siw_qp_tx.c | 39 +++---- drivers/infiniband/sw/siw/siw_verbs.c | 23 ++-- 8 files changed, 134 insertions(+), 202 deletions(-)
base-commit: 057a30168175048be9e9b30f0cafd26f5043eb07
Signature · 1 line
2.35.3
PATCH v5 1/17 RDMA/siw: Introduce siw_get_page
Add the wrapper function to get either pbl page or umem page.
--- drivers/infiniband/sw/siw/siw_qp_tx.c | 31 +++++++++++---------------- 1 file changed, 12 insertions(+), 19 deletions(-)
drivers/infiniband/sw/siw/siw_qp_tx.c +12 -19
@@ -34,6 +34,15 @@ static struct page *siw_get_pblpage(struct siw_mem *mem, u64 addr, int *idx)return NULL ; } +static struct page *siw_get_page (struct siw_mem *mem,struct siw_sge *sge, +unsigned long offset,int *pbl_idx) +{ +if (!mem->is_pbl) +return siw_get_upage (mem->umem, sge->laddr + offset); +else +return siw_get_pblpage (mem, sge->laddr + offset, pbl_idx); +} +/* * Copy short payload at provided destination payload address */
@@ -67,11 +76,7 @@ static int siw_try_1seg(struct siw_iwarp_tx *c_tx, void *paddr)char *buffer;int pbl_idx =0 ; -if (!mem->is_pbl) - p =siw_get_upage (mem->umem, sge->laddr); -else - p =siw_get_pblpage (mem, sge->laddr, &pbl_idx); - + p =siw_get_page (mem, sge,0 , &pbl_idx);if (unlikely (!p))return -EFAULT ;
@@ -85,13 +90,7 @@ static int siw_try_1seg(struct siw_iwarp_tx *c_tx, void *paddr)memcpy (paddr, buffer + off, part);kunmap_local (buffer); -if (!mem->is_pbl) - p =siw_get_upage (mem->umem, - sge->laddr + part); -else - p =siw_get_pblpage (mem, - sge->laddr + part, - &pbl_idx); + p =siw_get_page (mem, sge, part, &pbl_idx);if (unlikely (!p))return -EFAULT ;
@@ -498,13 +497,7 @@ static int siw_tx_hdt(struct siw_iwarp_tx *c_tx, struct socket *s)if (!is_kva) {struct page *p; -if (mem->is_pbl) - p =siw_get_pblpage ( - mem, sge->laddr + sge_off, - &pbl_idx); -else - p =siw_get_upage (mem->umem, - sge->laddr + sge_off); + p =siw_get_page (mem, sge, sge_off, &pbl_idx);if (unlikely (!p)) {siw_unmap_pages (iov, kmap_mask, seg); wqe->processed -= c_tx->bytes_unsent;
Signature · 1 line
2.35.3
PATCH v5 2/17 RDMA/siw: Introduce siw_update_skb_rcvd
In reply to Guoqing JiangThere are some places share the same logic, factor a common helper for it.
--- drivers/infiniband/sw/siw/siw_qp_rx.c | 31 +++++++++++---------------- 1 file changed, 12 insertions(+), 19 deletions(-)
drivers/infiniband/sw/siw/siw_qp_rx.c +12 -19
@@ -881,6 +881,13 @@ int siw_proc_rresp(struct siw_qp *qp)return rv; } +static void siw_update_skb_rcvd (struct siw_rx_stream *srx,u16 length) +{ + srx->skb_offset += length; + srx->skb_new -= length; + srx->skb_copied += length; +} +int siw_proc_terminate (struct siw_qp *qp) {struct siw_rx_stream *srx = &qp->rx_stream;
@@ -925,9 +932,7 @@ int siw_proc_terminate(struct siw_qp *qp) goto out; infop += to_copy; - srx->skb_offset += to_copy; - srx->skb_new -= to_copy; - srx->skb_copied += to_copy; +siw_update_skb_rcvd (srx, to_copy); srx->fpdu_part_rcvd += to_copy; srx->fpdu_part_rem -= to_copy;
@@ -949,9 +954,7 @@ int siw_proc_terminate(struct siw_qp *qp) term->flag_m ? "valid" : "invalid"); } out: - srx->skb_new -= to_copy; - srx->skb_offset += to_copy; - srx->skb_copied += to_copy; +siw_update_skb_rcvd (srx, to_copy); srx->fpdu_part_rcvd += to_copy; srx->fpdu_part_rem -= to_copy;
@@ -970,9 +973,7 @@ static int siw_get_trailer(struct siw_qp *qp, struct siw_rx_stream *srx)skb_copy_bits (skb, srx->skb_offset, tbuf, avail); - srx->skb_new -= avail; - srx->skb_offset += avail; - srx->skb_copied += avail; +siw_update_skb_rcvd (srx, avail); srx->fpdu_part_rem -= avail;if (srx->fpdu_part_rem)
@@ -1023,12 +1024,8 @@ static int siw_get_hdr(struct siw_rx_stream *srx)skb_copy_bits (skb, srx->skb_offset, (char *)c_hdr + srx->fpdu_part_rcvd, bytes); +siw_update_skb_rcvd (srx, bytes); srx->fpdu_part_rcvd += bytes; - - srx->skb_new -= bytes; - srx->skb_offset += bytes; - srx->skb_copied += bytes; -if (srx->fpdu_part_rcvd <MIN_DDP_HDR )return -EAGAIN ;
@@ -1091,12 +1088,8 @@ static int siw_get_hdr(struct siw_rx_stream *srx)skb_copy_bits (skb, srx->skb_offset, (char *)c_hdr + srx->fpdu_part_rcvd, bytes); +siw_update_skb_rcvd (srx, bytes); srx->fpdu_part_rcvd += bytes; - - srx->skb_new -= bytes; - srx->skb_offset += bytes; - srx->skb_copied += bytes; -if (srx->fpdu_part_rcvd < hdrlen)return -EAGAIN ; }
Signature · 1 line
2.35.3
PATCH v5 3/17 RDMA/siw: Use iov.iov_len in kernel_sendmsg
In reply to Guoqing JiangWe can pass iov.iov_len here.
--- drivers/infiniband/sw/siw/siw_qp_tx.c | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-)
drivers/infiniband/sw/siw/siw_qp_tx.c +1 -2
@@ -292,8 +292,7 @@ static int siw_tx_ctrl(struct siw_iwarp_tx *c_tx, struct socket *s, (char *)&c_tx->pkt.ctrl + c_tx->ctrl_sent, .iov_len = c_tx->ctrl_len - c_tx->ctrl_sent }; -int rv =kernel_sendmsg (s, &msg, &iov,1 , - c_tx->ctrl_len - c_tx->ctrl_sent); +int rv =kernel_sendmsg (s, &msg, &iov,1 , iov.iov_len);if (rv >=0 ) { c_tx->ctrl_sent += rv;
Signature · 1 line
2.35.3
PATCH v5 4/17 RDMA/siw: Remove goto lable in siw_mmap
In reply to Guoqing JiangLet's remove it since the failure case only falls through to the useless label.
--- drivers/infiniband/sw/siw/siw_verbs.c | 5 +---- 1 file changed, 1 insertion(+), 4 deletions(-)
drivers/infiniband/sw/siw/siw_verbs.c +1 -4
@@ -66,12 +66,9 @@ int siw_mmap(struct ib_ucontext *ctx, struct vm_area_struct *vma) entry =to_siw_mmap_entry (rdma_entry); rv =remap_vmalloc_range (vma, entry->address,0 ); -if (rv) { +if (rv)pr_warn ("remap_vmalloc_range failed: %lu, %zu\n" , vma->vm_pgoff, size); - goto out; - } -out:rdma_user_mmap_entry_put (rdma_entry);return rv;
Signature · 1 line
2.35.3
PATCH v5 5/17 RDMA/siw: Remove rcu from siw_qp
In reply to Guoqing JiangRemove it since it is not used.
--- drivers/infiniband/sw/siw/siw.h | 1 - 1 file changed, 1 deletion(-)
drivers/infiniband/sw/siw/siw.h +0 -1
@@ -465,7 +465,6 @@ struct siw_qp { } term_info;struct rdma_user_mmap_entry *sq_entry;/* mmap info for SQE array */ struct rdma_user_mmap_entry *rq_entry;/* mmap info for RQE array */ -struct rcu_head rcu; };/* helper macros */
Signature · 1 line
2.35.3
PATCH v5 6/17 RDMA/siw: No need to check term_info.valid before call siw_send_terminate
In reply to Guoqing JiangRemove the redundate checking since siw_send_terminate check it inside.
--- drivers/infiniband/sw/siw/siw_cm.c | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-)
drivers/infiniband/sw/siw/siw_cm.c +2 -3
@@ -393,8 +393,7 @@ void siw_qp_cm_drop(struct siw_qp *qp, int schedule) }siw_dbg_cep (cep,"immediate close, state %d\n" , cep->state); -if (qp->term_info.valid) -siw_send_terminate (qp); +siw_send_terminate (qp);if (cep->cm_id) {switch (cep->state) {
@@ -1061,7 +1060,7 @@ static void siw_cm_work_handler(struct work_struct *w)/* * QP scheduled LLP close */ -if (cep->qp && cep->qp->term_info.valid) +if (cep->qp)siw_send_terminate (cep->qp);if (cep->cm_id)
Signature · 1 line
2.35.3
PATCH v5 7/17 RDMA/siw: Factor out siw_rx_data helper
In reply to Guoqing JiangRemove the redundant code given they share the same logic.
--- drivers/infiniband/sw/siw/siw_qp_rx.c | 53 ++++++++++----------------- 1 file changed, 20 insertions(+), 33 deletions(-)
drivers/infiniband/sw/siw/siw_qp_rx.c +20 -33
@@ -405,6 +405,20 @@ static struct siw_wqe *siw_rqe_get(struct siw_qp *qp)return wqe; } +static int siw_rx_data (struct siw_mem *mem_p,struct siw_rx_stream *srx, +unsigned int *pbl_idx,u64 addr,int bytes) +{ +int rv; + +if (mem_p->mem_obj ==NULL ) + rv =siw_rx_kva (srx,ib_virt_dma_to_ptr (addr), bytes); +else if (!mem_p->is_pbl) + rv =siw_rx_umem (srx, mem_p->umem, addr, bytes); +else + rv =siw_rx_pbl (srx, pbl_idx, mem_p, addr, bytes); +return rv; +} +/* * siw_proc_send: *
@@ -485,17 +499,8 @@ int siw_proc_send(struct siw_qp *qp)break ; } mem_p = *mem; -if (mem_p->mem_obj ==NULL ) - rv =siw_rx_kva (srx, -ib_virt_dma_to_ptr (sge->laddr + frx->sge_off), - sge_bytes); -else if (!mem_p->is_pbl) - rv =siw_rx_umem (srx, mem_p->umem, - sge->laddr + frx->sge_off, sge_bytes); -else - rv =siw_rx_pbl (srx, &frx->pbl_idx, mem_p, - sge->laddr + frx->sge_off, sge_bytes); - + rv =siw_rx_data (mem_p, srx, &frx->pbl_idx, + sge->laddr + frx->sge_off, sge_bytes);if (unlikely (rv != sge_bytes)) { wqe->processed += rcvd_bytes;
@@ -598,17 +603,8 @@ int siw_proc_write(struct siw_qp *qp)return -EINVAL ; } -if (mem->mem_obj ==NULL ) - rv =siw_rx_kva (srx, - (void *)(uintptr_t )(srx->ddp_to + srx->fpdu_part_rcvd), - bytes); -else if (!mem->is_pbl) - rv =siw_rx_umem (srx, mem->umem, - srx->ddp_to + srx->fpdu_part_rcvd, bytes); -else - rv =siw_rx_pbl (srx, &frx->pbl_idx, mem, - srx->ddp_to + srx->fpdu_part_rcvd, bytes); - + rv =siw_rx_data (mem, srx, &frx->pbl_idx, + srx->ddp_to + srx->fpdu_part_rcvd, bytes);if (unlikely (rv != bytes)) { siw_init_terminate(qp, TERM_ERROR_LAYER_DDP, DDP_ETYPE_CATASTROPHIC,
@@ -849,17 +845,8 @@ int siw_proc_rresp(struct siw_qp *qp) mem_p = *mem; bytes =min (srx->fpdu_part_rem, srx->skb_new); - -if (mem_p->mem_obj ==NULL ) - rv =siw_rx_kva (srx, -ib_virt_dma_to_ptr (sge->laddr + wqe->processed), - bytes); -else if (!mem_p->is_pbl) - rv =siw_rx_umem (srx, mem_p->umem, sge->laddr + wqe->processed, - bytes); -else - rv =siw_rx_pbl (srx, &frx->pbl_idx, mem_p, - sge->laddr + wqe->processed, bytes); + rv =siw_rx_data (mem_p, srx, &frx->pbl_idx, + sge->laddr + wqe->processed, bytes);if (rv != bytes) { wqe->wc_status =SIW_WC_GENERAL_ERR ; rv = -EINVAL ;
Signature · 1 line
2.35.3
PATCH v5 8/17 RDMA/siw: Introduce SIW_STAG_MAX_INDEX
In reply to Guoqing JiangAdd the macro to remove magic number in the code.
--- drivers/infiniband/sw/siw/siw_mem.c | 12 +++++++----- 1 file changed, 7 insertions(+), 5 deletions(-)
drivers/infiniband/sw/siw/siw_mem.c +7 -5
@@ -14,18 +14,20 @@#include "siw.h" #include "siw_mem.h" +/* Stag lookup is based on its index part only (24 bits). */ +#define SIW_STAG_MAX_INDEX 0x00ffffff +/* -* Stag lookup is based on its index part only (24 bits). * The code avoids special Stag of zero and tries to randomize * STag values between 1 and SIW_STAG_MAX_INDEX. */ int siw_mem_add (struct siw_device *sdev,struct siw_mem *m) { -struct xa_limit limit =XA_LIMIT (1 ,0x00ffffff ); +struct xa_limit limit =XA_LIMIT (1 ,SIW_STAG_MAX_INDEX );u32 id, next;get_random_bytes (&next,4 ); - next &=0x00ffffff ; + next &=SIW_STAG_MAX_INDEX ;if (xa_alloc_cyclic (&sdev->mem_xa, &id, m, limit, &next,GFP_KERNEL ) <0 )
@@ -81,7 +83,7 @@ int siw_mr_add_mem(struct siw_mr *mr, struct ib_pd *pd, void *mem_obj, {struct siw_device *sdev =to_siw_dev (pd->device);struct siw_mem *mem =kzalloc (sizeof (*mem),GFP_KERNEL ); -struct xa_limit limit =XA_LIMIT (1 ,0x00ffffff ); +struct xa_limit limit =XA_LIMIT (1 ,SIW_STAG_MAX_INDEX );u32 id, next;if (!mem)
@@ -97,7 +99,7 @@ int siw_mr_add_mem(struct siw_mr *mr, struct ib_pd *pd, void *mem_obj,kref_init (&mem->ref);get_random_bytes (&next,4 ); - next &=0x00ffffff ; + next &=SIW_STAG_MAX_INDEX ;if (xa_alloc_cyclic (&sdev->mem_xa, &id, mem, limit, &next,GFP_KERNEL ) <0 ) {
Signature · 1 line
2.35.3
PATCH v5 9/17 RDMA/siw: Add one parameter to siw_destroy_cpulist
In reply to Guoqing JiangWith that we can reuse it in siw_init_cpulist.
--- drivers/infiniband/sw/siw/siw_main.c | 30 +++++++++++++--------------- 1 file changed, 14 insertions(+), 16 deletions(-)
drivers/infiniband/sw/siw/siw_main.c +14 -16
@@ -109,6 +109,17 @@ static struct {int num_nodes; } siw_cpu_info; +static void siw_destroy_cpulist (int number) +{ +int i =0 ; + +while (i < number) +kfree (siw_cpu_info.tx_valid_cpus[i++]); + +kfree (siw_cpu_info.tx_valid_cpus); + siw_cpu_info.tx_valid_cpus =NULL ; +} +static int siw_init_cpulist (void ) {int i, num_nodes = nr_node_ids;
@@ -138,24 +149,11 @@ static int siw_init_cpulist(void) out_err: siw_cpu_info.num_nodes =0 ; -while (--i >=0 ) -kfree (siw_cpu_info.tx_valid_cpus[i]); -kfree (siw_cpu_info.tx_valid_cpus); - siw_cpu_info.tx_valid_cpus =NULL ; +siw_destroy_cpulist (i);return -ENOMEM ; } -static void siw_destroy_cpulist (void ) -{ -int i =0 ; - -while (i < siw_cpu_info.num_nodes) -kfree (siw_cpu_info.tx_valid_cpus[i++]); - -kfree (siw_cpu_info.tx_valid_cpus); -} -/* * Choose CPU with least number of active QP's from NUMA node of * TX interface.
@@ -558,7 +556,7 @@ static __init int siw_init_module(void)pr_info ("SoftIWARP attach failed. Error: %d\n" , rv);siw_cm_exit (); -siw_destroy_cpulist (); +siw_destroy_cpulist (siw_cpu_info.num_nodes);return rv; }
@@ -573,7 +571,7 @@ static void __exit siw_exit_module(void)siw_cm_exit (); -siw_destroy_cpulist (); +siw_destroy_cpulist (siw_cpu_info.num_nodes);if (siw_crypto_shash)crypto_free_shash (siw_crypto_shash);
Signature · 1 line
2.35.3
PATCH v5 10/17 RDMA/siw: Introduce siw_cep_set_free_and_put
In reply to Guoqing JiangAdd the helper which can be used in some places.
--- drivers/infiniband/sw/siw/siw_cm.c | 31 ++++++++++++++---------------- 1 file changed, 14 insertions(+), 17 deletions(-)
drivers/infiniband/sw/siw/siw_cm.c +14 -17
@@ -444,6 +444,12 @@ void siw_cep_put(struct siw_cep *cep)kref_put (&cep->ref, __siw_cep_dealloc); } +static void siw_cep_set_free_and_put (struct siw_cep *cep) +{ +siw_cep_set_free (cep); +siw_cep_put (cep); +} +void siw_cep_get (struct siw_cep *cep) {kref_get (&cep->ref);
@@ -1514,9 +1520,7 @@ int siw_connect(struct iw_cm_id *id, struct iw_cm_conn_param *params) cep->state =SIW_EPSTATE_CLOSED ; -siw_cep_set_free (cep); - -siw_cep_put (cep); +siw_cep_set_free_and_put (cep); }else if (s) {sock_release (s);
@@ -1564,16 +1568,14 @@ int siw_accept(struct iw_cm_id *id, struct iw_cm_conn_param *params)if (cep->state !=SIW_EPSTATE_RECVD_MPAREQ ) {siw_dbg_cep (cep,"out of state\n" ); -siw_cep_set_free (cep); -siw_cep_put (cep); +siw_cep_set_free_and_put (cep);return -ECONNRESET ; } qp =siw_qp_id2obj (sdev, params->qpn);if (!qp) {WARN (1 ,"[QP %d] does not exist\n" , params->qpn); -siw_cep_set_free (cep); -siw_cep_put (cep); +siw_cep_set_free_and_put (cep);return -EINVAL ; }
@@ -1719,8 +1721,7 @@ int siw_accept(struct iw_cm_id *id, struct iw_cm_conn_param *params) cep->qp =NULL ;siw_qp_put (qp); -siw_cep_set_free (cep); -siw_cep_put (cep); +siw_cep_set_free_and_put (cep);return rv; }
@@ -1743,8 +1744,7 @@ int siw_reject(struct iw_cm_id *id, const void *pdata, u8 pd_len)if (cep->state !=SIW_EPSTATE_RECVD_MPAREQ ) {siw_dbg_cep (cep,"out of state\n" ); -siw_cep_set_free (cep); -siw_cep_put (cep);/* put last reference */ +siw_cep_set_free_and_put (cep);/* put last reference */ return -ECONNRESET ; }
@@ -1761,8 +1761,7 @@ int siw_reject(struct iw_cm_id *id, const void *pdata, u8 pd_len) cep->state =SIW_EPSTATE_CLOSED ; -siw_cep_set_free (cep); -siw_cep_put (cep); +siw_cep_set_free_and_put (cep);return 0 ; }
@@ -1897,8 +1896,7 @@ int siw_create_listen(struct iw_cm_id *id, int backlog)siw_socket_disassoc (s); cep->state =SIW_EPSTATE_CLOSED ; -siw_cep_set_free (cep); -siw_cep_put (cep); +siw_cep_set_free_and_put (cep); }sock_release (s);
@@ -1932,8 +1930,7 @@ static void siw_drop_listeners(struct iw_cm_id *id) cep->sock =NULL ; } cep->state =SIW_EPSTATE_CLOSED ; -siw_cep_set_free (cep); -siw_cep_put (cep); +siw_cep_set_free_and_put (cep); } }
Signature · 1 line
2.35.3
PATCH v5 11/17 RDMA/siw: Introduce siw_free_cm_id
In reply to Guoqing JiangFactor out a helper to simplify code.
--- drivers/infiniband/sw/siw/siw_cm.c | 30 ++++++++++++++---------------- 1 file changed, 14 insertions(+), 16 deletions(-)
drivers/infiniband/sw/siw/siw_cm.c +14 -16
@@ -364,6 +364,15 @@ static int siw_cm_upcall(struct siw_cep *cep, enum iw_cm_event_type reason,return id->event_handler (id, &event); } +static void siw_free_cm_id (struct siw_cep *cep) +{ +if (!cep->cm_id) +return ; + + cep->cm_id->rem_ref (cep->cm_id); + cep->cm_id =NULL ; +} +/* * siw_qp_cm_drop() *
@@ -415,8 +424,7 @@ void siw_qp_cm_drop(struct siw_qp *qp, int schedule)default :break ; } - cep->cm_id->rem_ref (cep->cm_id); - cep->cm_id =NULL ; +siw_free_cm_id (cep);siw_cep_put (cep); } cep->state =SIW_EPSTATE_CLOSED ;
@@ -1180,8 +1188,7 @@ static void siw_cm_work_handler(struct work_struct *w) cep->sock =NULL ; }if (cep->cm_id) { - cep->cm_id->rem_ref (cep->cm_id); - cep->cm_id =NULL ; +siw_free_cm_id (cep);siw_cep_put (cep); } }
@@ -1710,10 +1717,7 @@ int siw_accept(struct iw_cm_id *id, struct iw_cm_conn_param *params) cep->state =SIW_EPSTATE_CLOSED ; -if (cep->cm_id) { - cep->cm_id->rem_ref (id); - cep->cm_id =NULL ; - } +siw_free_cm_id (cep);if (qp->cep) {siw_cep_put (cep); qp->cep =NULL ;
@@ -1888,10 +1892,7 @@ int siw_create_listen(struct iw_cm_id *id, int backlog)if (cep) {siw_cep_set_inuse (cep); -if (cep->cm_id) { - cep->cm_id->rem_ref (cep->cm_id); - cep->cm_id =NULL ; - } +siw_free_cm_id (cep); cep->sock =NULL ;siw_socket_disassoc (s); cep->state =SIW_EPSTATE_CLOSED ;
@@ -1920,10 +1921,7 @@ static void siw_drop_listeners(struct iw_cm_id *id)siw_cep_set_inuse (cep); -if (cep->cm_id) { - cep->cm_id->rem_ref (cep->cm_id); - cep->cm_id =NULL ; - } +siw_free_cm_id (cep);if (cep->sock) {siw_socket_disassoc (cep->sock);sock_release (cep->sock);
Signature · 1 line
2.35.3
PATCH v5 12/17 RDMA/siw: Cleanup siw_accept
In reply to Guoqing JiangWith the initialization of rv and the two added label, we can simplifiy code a bit.
--- drivers/infiniband/sw/siw/siw_cm.c | 41 ++++++++++-------------------- 1 file changed, 14 insertions(+), 27 deletions(-)
drivers/infiniband/sw/siw/siw_cm.c +14 -27
@@ -1558,7 +1558,7 @@ int siw_accept(struct iw_cm_id *id, struct iw_cm_conn_param *params)struct siw_cep *cep = (struct siw_cep *)id->provider_data;struct siw_qp *qp;struct siw_qp_attrs qp_attrs; -int rv, max_priv_data =MPA_MAX_PRIVDATA ; +int rv = -EINVAL , max_priv_data =MPA_MAX_PRIVDATA ;bool wait_for_peer_rts = false;siw_cep_set_inuse (cep);
@@ -1574,24 +1574,17 @@ int siw_accept(struct iw_cm_id *id, struct iw_cm_conn_param *params)if (cep->state !=SIW_EPSTATE_RECVD_MPAREQ ) {siw_dbg_cep (cep,"out of state\n" ); - -siw_cep_set_free_and_put (cep); - -return -ECONNRESET ; + rv = -ECONNRESET ; + goto free_cep; } qp =siw_qp_id2obj (sdev, params->qpn);if (!qp) {WARN (1 ,"[QP %d] does not exist\n" , params->qpn); -siw_cep_set_free_and_put (cep); - -return -EINVAL ; + goto free_cep; }down_write (&qp->state_lock); -if (qp->attrs.state >SIW_QP_STATE_RTR ) { - rv = -EINVAL ; -up_write (&qp->state_lock); - goto error; - } +if (qp->attrs.state >SIW_QP_STATE_RTR ) + goto error_unlock;siw_dbg_cep (cep,"[QP %d]\n" , params->qpn);if (try_gso && cep->mpa.hdr.params.bits &MPA_RR_FLAG_GSO_EXP ) {
@@ -1605,9 +1598,7 @@ int siw_accept(struct iw_cm_id *id, struct iw_cm_conn_param *params)"[QP %u]: ord %d (max %d), ird %d (max %d)\n" ,qp_id (qp), params->ord, sdev->attrs.max_ord, params->ird, sdev->attrs.max_ird); - rv = -EINVAL ; -up_write (&qp->state_lock); - goto error; + goto error_unlock; }if (cep->enhanced_rdma_conn_est) max_priv_data -=sizeof (struct mpa_v2_data );
@@ -1617,9 +1608,7 @@ int siw_accept(struct iw_cm_id *id, struct iw_cm_conn_param *params) cep,"[QP %u]: private data length: %d (max %d)\n" , qp_id(qp), params->private_data_len, max_priv_data); - rv = -EINVAL ; -up_write (&qp->state_lock); - goto error; + goto error_unlock; }if (cep->enhanced_rdma_conn_est) {if (params->ord > cep->ord) {
@@ -1628,9 +1617,7 @@ int siw_accept(struct iw_cm_id *id, struct iw_cm_conn_param *params) }else { cep->ird = params->ird; cep->ord = params->ord; - rv = -EINVAL ; -up_write (&qp->state_lock); - goto error; + goto error_unlock; } }if (params->ird < cep->ird) {
@@ -1639,8 +1626,7 @@ int siw_accept(struct iw_cm_id *id, struct iw_cm_conn_param *params) params->ird = cep->ird; else { rv = -ENOMEM ; -up_write (&qp->state_lock); - goto error; + goto error_unlock; } } if (cep->mpa.v2_ctrl.ord &
@@ -1687,7 +1673,6 @@ int siw_accept(struct iw_cm_id *id, struct iw_cm_conn_param *params)SIW_QP_ATTR_ORD |SIW_QP_ATTR_IRD | SIW_QP_ATTR_MPA);up_write (&qp->state_lock); -if (rv) goto error;
@@ -1710,6 +1695,9 @@ int siw_accept(struct iw_cm_id *id, struct iw_cm_conn_param *params)siw_cep_set_free (cep);return 0 ; + +error_unlock: +up_write (&qp->state_lock); error:siw_socket_disassoc (cep->sock);sock_release (cep->sock);
@@ -1724,9 +1712,8 @@ int siw_accept(struct iw_cm_id *id, struct iw_cm_conn_param *params) } cep->qp =NULL ;siw_qp_put (qp); - +free_cep:siw_cep_set_free_and_put (cep); -return rv; }
Signature · 1 line
2.35.3
PATCH v5 13/17 RDMA/siw: Remove siw_sk_save_upcalls
In reply to Guoqing JiangLet's move it into siw_sk_assign_cm_upcalls, then we only need to get sk_callback_lock once.
--- drivers/infiniband/sw/siw/siw_cm.c | 16 +++++----------- 1 file changed, 5 insertions(+), 11 deletions(-)
drivers/infiniband/sw/siw/siw_cm.c +5 -11
@@ -40,16 +40,6 @@ static int siw_cm_upcall(struct siw_cep *cep, enum iw_cm_event_type reason, int status);static void siw_sk_assign_cm_upcalls (struct sock *sk) -{ -write_lock_bh (&sk->sk_callback_lock); - sk->sk_state_change = siw_cm_llp_state_change; - sk->sk_data_ready = siw_cm_llp_data_ready; - sk->sk_write_space = siw_cm_llp_write_space; - sk->sk_error_report = siw_cm_llp_error_report; -write_unlock_bh (&sk->sk_callback_lock); -} - -static void siw_sk_save_upcalls (struct sock *sk) {struct siw_cep *cep =sk_to_cep (sk);
@@ -58,6 +48,11 @@ static void siw_sk_save_upcalls(struct sock *sk) cep->sk_data_ready = sk->sk_data_ready; cep->sk_write_space = sk->sk_write_space; cep->sk_error_report = sk->sk_error_report; + + sk->sk_state_change = siw_cm_llp_state_change; + sk->sk_data_ready = siw_cm_llp_data_ready; + sk->sk_write_space = siw_cm_llp_write_space; + sk->sk_error_report = siw_cm_llp_error_report;write_unlock_bh (&sk->sk_callback_lock); }
@@ -156,7 +151,6 @@ static void siw_cep_socket_assoc(struct siw_cep *cep, struct socket *s)siw_cep_get (cep); s->sk->sk_user_data = cep; -siw_sk_save_upcalls (s->sk);siw_sk_assign_cm_upcalls (s->sk); }
Signature · 1 line
2.35.3
PATCH v5 14/17 RDMA/siw: Fix typo
In reply to Guoqing JiangReplace ORRQ with ORQ.
--- drivers/infiniband/sw/siw/siw_qp.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
drivers/infiniband/sw/siw/siw_qp.c +1 -1
@@ -1183,7 +1183,7 @@ int siw_rqe_complete(struct siw_qp *qp, struct siw_rqe *rqe, u32 bytes,/* * siw_sq_flush() * -* Flush SQ and ORRQ entries to CQ. +* Flush SQ and ORQ entries to CQ. * * Must be called with QP state write lock held. * Therefore, SQ and ORQ lock must not be taken.
Signature · 1 line
2.35.3
PATCH v5 15/17 RDMA/siw: Only check attrs->cap.max_send_wr in siw_create_qp
In reply to Guoqing JiangWe can just check max_send_wr here given both max_send_wr and max_recv_wr are defined as u32 type, and we also need to ensure num_sqe (derived from max_send_wr) shouldn't be zero.
--- drivers/infiniband/sw/siw/siw_verbs.c | 18 +++++------------- 1 file changed, 5 insertions(+), 13 deletions(-)
drivers/infiniband/sw/siw/siw_verbs.c +5 -13
@@ -333,11 +333,10 @@ int siw_create_qp(struct ib_qp *ibqp, struct ib_qp_init_attr *attrs, goto err_atomic; }/* -* NOTE: we allow for zero element SQ and RQ WQE's SGL's -* but not for a QP unable to hold any WQE (SQ + RQ) +* NOTE: we don't allow for a QP unable to hold any SQ WQE */ -if (attrs->cap.max_send_wr + attrs->cap.max_recv_wr ==0 ) { -siw_dbg (base_dev,"QP must have send or receive queue\n" ); +if (attrs->cap.max_send_wr ==0 ) { +siw_dbg (base_dev,"QP must have send queue\n" ); rv = -EINVAL ; goto err_atomic; }
@@ -357,21 +356,14 @@ int siw_create_qp(struct ib_qp *ibqp, struct ib_qp_init_attr *attrs,if (rv) goto err_atomic; - num_sqe = attrs->cap.max_send_wr; - num_rqe = attrs->cap.max_recv_wr;/* All queue indices are derived from modulo operations * on a free running 'get' (consumer) and 'put' (producer) * unsigned counter. Having queue sizes at power of two * avoids handling counter wrap around. */ -if (num_sqe) - num_sqe =roundup_pow_of_two (num_sqe); -else { -/* Zero sized SQ is not supported */ - rv = -EINVAL ; - goto err_out_xa; - } + num_sqe =roundup_pow_of_two (attrs->cap.max_send_wr); + num_rqe = attrs->cap.max_recv_wr;if (num_rqe) num_rqe =roundup_pow_of_two (num_rqe);
Signature · 1 line
2.35.3
PATCH v5 16/17 RDMA/siw: Introduce siw_destroy_cep_sock
In reply to Guoqing JiangAdd one helper to simplify code a bit.
--- drivers/infiniband/sw/siw/siw_cm.c | 26 ++++++++++++-------------- 1 file changed, 12 insertions(+), 14 deletions(-)
drivers/infiniband/sw/siw/siw_cm.c +12 -14
@@ -367,6 +367,15 @@ static void siw_free_cm_id(struct siw_cep *cep) cep->cm_id =NULL ; } +static void siw_destroy_cep_sock (struct siw_cep *cep) +{ +if (cep->sock) { +siw_socket_disassoc (cep->sock); +sock_release (cep->sock); + cep->sock =NULL ; + } +} +/* * siw_qp_cm_drop() *
@@ -423,14 +432,7 @@ void siw_qp_cm_drop(struct siw_qp *qp, int schedule) } cep->state =SIW_EPSTATE_CLOSED ; -if (cep->sock) { -siw_socket_disassoc (cep->sock); -/* -* Immediately close socket -*/ -sock_release (cep->sock); - cep->sock =NULL ; - } +siw_destroy_cep_sock (cep);if (cep->qp) { cep->qp =NULL ;siw_qp_put (qp);
@@ -1693,9 +1695,7 @@ int siw_accept(struct iw_cm_id *id, struct iw_cm_conn_param *params) error_unlock:up_write (&qp->state_lock); error: -siw_socket_disassoc (cep->sock); -sock_release (cep->sock); - cep->sock =NULL ; +siw_destroy_cep_sock (cep); cep->state =SIW_EPSTATE_CLOSED ;
@@ -1740,9 +1740,7 @@ int siw_reject(struct iw_cm_id *id, const void *pdata, u8 pd_len) cep->mpa.hdr.params.bits |=MPA_RR_FLAG_REJECT ;/* reject */ siw_send_mpareqrep (cep, pdata, pd_len); } -siw_socket_disassoc (cep->sock); -sock_release (cep->sock); - cep->sock =NULL ; +siw_destroy_cep_sock (cep); cep->state =SIW_EPSTATE_CLOSED ;
Signature · 1 line
2.35.3
PATCH v5 17/17 RDMA/siw: Update comments for siw_qp_sq_process
In reply to Guoqing JiangThere is no siw_sq_work_handler in code, change it with siw_tx_thread since siw_run_sq -> siw_sq_resume -> siw_qp_sq_process.
--- drivers/infiniband/sw/siw/siw_qp_tx.c | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-)
drivers/infiniband/sw/siw/siw_qp_tx.c +2 -3
@@ -997,13 +997,12 @@ static int siw_qp_sq_proc_local(struct siw_qp *qp, struct siw_wqe *wqe)* MPA FPDUs, each containing a DDP segment. * * SQ processing may occur in user context as a result of posting -* new WQE's or from siw_sq_work_handler() context. Processing in +* new WQE's or from siw_tx_thread context. Processing in * user context is limited to non-kernel verbs users. * * SQ processing may get paused anytime, possibly in the middle of a WR * or FPDU, if insufficient send space is available. SQ processing -* gets resumed from siw_sq_work_handler(), if send space becomes -* available again. +* gets resumed from siw_tx_thread, if send space becomes available again. * * Must be called with the QP state read-locked. *
Signature · 1 line
2.35.3
Leon Romanovsky
AppliedQuoted 10 lines · Guoqing Jiang
V5 changes:
1. add Acked-by tags.
2. rebase to latest rdma tree and remove one obsolete patch.V4 changes:
1. add Acked-by tags.
2. update patch 3 and patch 12 per Bernard's review.
3. update patch header in patch 18.[...]
Applied, thanks!
[01/17] RDMA/siw: Introduce siw_get_page
https://git.kernel.org/rdma/rdma/c/3a179fe34acbd5 [02/17] RDMA/siw: Introduce siw_update_skb_rcvd https://git.kernel.org/rdma/rdma/c/a2b64565e8ea95 [03/17] RDMA/siw: Use iov.iov_len in kernel_sendmsg https://git.kernel.org/rdma/rdma/c/2109ddf032ebc5 [04/17] RDMA/siw: Remove goto lable in siw_mmap https://git.kernel.org/rdma/rdma/c/d248960941b71e [05/17] RDMA/siw: Remove rcu from siw_qp https://git.kernel.org/rdma/rdma/c/659da08ed83a67 [06/17] RDMA/siw: No need to check term_info.valid before call siw_send_terminate https://git.kernel.org/rdma/rdma/c/065186d228c528 [07/17] RDMA/siw: Factor out siw_rx_data helper https://git.kernel.org/rdma/rdma/c/60d2136db8780b [08/17] RDMA/siw: Introduce SIW_STAG_MAX_INDEX https://git.kernel.org/rdma/rdma/c/6a343cc3bf2626 [09/17] RDMA/siw: Add one parameter to siw_destroy_cpulist https://git.kernel.org/rdma/rdma/c/25680c1f261475 [10/17] RDMA/siw: Introduce siw_cep_set_free_and_put https://git.kernel.org/rdma/rdma/c/b5c91543204c34 [11/17] RDMA/siw: Introduce siw_free_cm_id https://git.kernel.org/rdma/rdma/c/08456d4db73bbb [12/17] RDMA/siw: Cleanup siw_accept https://git.kernel.org/rdma/rdma/c/77b59bd932a026 [13/17] RDMA/siw: Remove siw_sk_save_upcalls https://git.kernel.org/rdma/rdma/c/a410a732787026 [14/17] RDMA/siw: Fix typo https://git.kernel.org/rdma/rdma/c/3beced14d1998c [15/17] RDMA/siw: Only check attrs->cap.max_send_wr in siw_create_qp https://git.kernel.org/rdma/rdma/c/788bbf4c2fc6e0 [16/17] RDMA/siw: Introduce siw_destroy_cep_sock https://git.kernel.org/rdma/rdma/c/d9a5b48681315a [17/17] RDMA/siw: Update comments for siw_qp_sq_process https://git.kernel.org/rdma/rdma/c/79844118d6c1cd
Best regards,
Signature · 1 line
Leon Romanovsky <[email protected]>